From rodney_bates at lcwb.coop Sun Aug 2 02:43:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 01 Aug 2015 19:43:43 -0500 Subject: [M3devel] better typing on SUBARRAY temporaries? In-Reply-To: References: <55B03594.20800@lcwb.coop> <55B25261.5010106@lcwb.coop> <55B3AA70.5050906@lcwb.coop> <55B3B8CF.1060208@lcwb.coop> Message-ID: <55BD67BF.4040606@lcwb.coop> On 07/26/2015 02:23 AM, Peter McKinna wrote: > So far not doing anything with free_temp, declare_temp just does an alloca but does it at the end of the first basic block to avoid dominate all uses problems. ( I have since discovered that there is a flag -alloca-hoisting which does this for you). The language ref hardly mentions temps except that they are referred to as unnamed values ie %1 %2 etc whether the optimiser does anything with these I dont know. I think I'm giving the temps names like %tmp.1 etc > The workaround I have at the moment is that if in the store, (which I think is the first place one of these temps is referenced) the var is a temp ie having NoID and is not declared in the current procedure then declare it ie alloca it. Seems to work but seems pretty kludgy. > My first thought when I struck this problem was that I should put all temps on the static link (since there is no parameter in declare_temp to say its uplevel and hence say which could be nested proc material) but that didnt work and really it was a pretty dumb idea. I cant see why the front end couldnt just declare a new temp in the finally proc. > Commit 496e9be1dcdcf87bda8e72239fc90132591b4cf4 fixes this, for this test case. > Regards Peter > > On Sun, Jul 26, 2015 at 2:26 AM, Rodney M. Bates > wrote: > > More on this: > > I appears that, in the CM3 IR, "variables" declared by declare_local and one > declared by declare_temp (and a few others, too), while declared with different > operators, are accessible interchangeably. In this example, the FINALLY procedure > accesses both s and the temporary nonlocally, in the same way. The difference > is, the temp, unlike a local variable, has the possibility that its space in > the activation record is freed prior to the end of the corresponding code. > > What llvm IR are you translating declare_temp and free_temp into? Llvm might > have different rules for temps. > > > > > On 07/25/2015 10:25 AM, Rodney M. Bates wrote: > > I compiled this and looked at the cm3 IR for it. At first glance, it looks like > a bug in the front end. The FINALLY code is translated as a nested procedure, > inside the one containing the entire TRY..FINALLY statement. The temp is created by > a declare_temp in the outer procedure, freed by a free_temp in the inner one, > and used in both, the 2 uses being unrelated. > > In m3-sys/m3middle/src/M3CG_Ops.i3, I see: > ----------------------------------------------------------------------------- > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > free_temp (v: Var); > (* releases the space occupied by temp 'v' so that it may be reused by > other new temporaries. *) > ----------------------------------------------------------------------------- > > And it also seems strange for the temp to be freed inside the nested procedure, > although this does not violate what the comment says. The fact that every > temp has a unique variable number would take care of matching the free_temp to the > right declare_temp, and maybe the code generators can handle freeing a nonlocal > temp. > > Apparently, this has caused no problems with preexisting code generators. > But it certainly looks wrong, and clearly violates the comments. > > I recall that the gcc-derived code generator and the integrated x86 code generator > both unnest nested procedures, in opposite ways (one pulls them out in front, the > other in back), which might have something to do with how they handle this. > > What happens in the llvm back end for a programmer-declared nested procedure > making a nonlocal reference to a programmer-declared local variable of the > containing procedure? If you can handle this latter case, can you handle the > failing one the same way? Maybe this is what is happening in the other code > generators, and the comment is just too strict. > > > > > On 07/24/2015 08:13 PM, Peter McKinna wrote: > > On the subject of temporaries can I get your thoughts on the following? > > TRY > s := s & "a"; > FINALLY > s := s & "b"; > END; > > The front end declares a temp in the try block as part of its concat and then refers to the same temp in the finally block. The trouble is that the finally code is generated in a separate procedure and references a temp declared in the proc of the try. In llvm the first you know of the problem is a store to the temp which has not been declared. > Just wondering whether the front end should redeclare this temp? Also is the front end generating similar temps for other runtime operations? > > Regards Peter > > On Sat, Jul 25, 2015 at 12:57 AM, Rodney M. Bates >> wrote: > > > > On 07/24/2015 03:57 AM, Jay K wrote: > > I model this in C like: > > > M3C.m3: > print(self, "/*declare_open_array*/typedef struct {"); > > > Presumably, this comment means you have some way of knowing, in M3C, that this is an open array? > > print(self, element_type.text); > print(self, "* _elts; CARDINAL _size"); > IF bit_size > Target.Integer.size * 2 THEN > print(self, "s["); > print(self, IntToDec((bit_size - Target.Integer.size) DIV Target.Integer.size)); > print(self, "]"); > END; > > > that is..and this i stinky that I get the struct "size", > > > size == 2 * sizeof(integer): > > > struct { > T* elements > size_t size; > } > > > else: > N = size - sizeof(INTEGER) / sizeof(INTEGER) > T ** elements; // where the number of star is N > > > I would not do pointer to pointer to pointer ... here. Just "T* elements", regardless > of the number of open dimensions. As I understand them, C's language-supported > multidimensional arrays are like Modula3 multidimensional _fixed_ arrays, i.e., > the language does the multi-subscript address arithmetic, which means the type > system needs to provide a static element size of each dimension. And that, > except for the innermost, depends on the element count of the next inner > dimension, which is not static here. > > So, the multiple dimensions are essentially flattened into one, and access to A[I,J,K] > is lowered by the front end into explicit address arithmetic into the flattened > array, using the values in the shape. Something like A[I*Shape[0]+J*Shape[1]+K] > > size_t sizes[N] > > > It is kind of lame that the frontend just gives the overall size > and the backend is just left to assume the layout like that. > > > If you know in M3C that it's an open array, you can infer what the layout is. > But yes, it's kind of lame. This is just another of the several places we have > seen that the front end has lowered things too far, and you have to > You know it's generated by code in OpenArray.m3, and you know the general dope layout, > and open dimension count, so you can generate an appropriate type. > > > > Really, the frontend should declare a type "pointer to pointer to pointer" with > the right "depth", and then a record with that pointer and a size or fixed size array of sizes. > > > Again, really ugly how it works now where backend is just given a size and can only > assume the layout. > > I don't know what a "dope vector" is. A value used as an initializer? > > > This is a very old and rather uninformative term for any block of stuff stored at runtime > that describes some source programmer's real data and how to access it. The only alternative > term I can think of would be "metadata", although that is rather overgeneral, and is usually > used with quite different specific meanings. But it is data describing data. > > > It is even worse for subarray. In this case we aren't even told it is an open array, just > some random struct with a size. That is what I first want to fix. It should declare an open array, > assuming they do have the same layout, which I think they do. > > > What you really need to know is that it's an open array, of which subarray is a subcategory. > In our implementation, all open array values have the same dope. E.g., look for the case where > a fixed array actual parameter is passed to an open array formal. > > > subarray temporaries and jmpbufs are I believe the only place the frontend passes so little > type information. > > > For jmpbufs I'm hoping to notice their name, and, unfortunately expensive, replace them > with #include and jmpbuf, instead of just a struct with an array of bytes. > > > > > - Jay > > > > Date: Wed, 22 Jul 2015 19:30:12 -0500 > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] better typing on SUBARRAY temporaries? > > > > I'm not exactly sure what you are asking, but here is some light on what > > you are seeing. These temporaries are exactly the dope the compiler uses > > to represent all open array values. First a pointer to the zeroth > > array element, then the "shape", as defined in M3 definition, 2.2.3, i.e. > > an array of element counts for each open subscript. For an open array > > parameter, this would be the machine representation of the parameter > > itself, authored in M3. (but passed by reference.) For a heap object, > > it is stored right before the elements themselves. For a SUBARRAY > > expression, it has to be a temporary. It also has to be constructed > > at the call site, as an anonymous temporary, when passing an entire fixed > > array to an open array parameter > > > > So, a good type for it might look like: > > > > > > RECORD > > Elements : REF ARRAY [0..Max, ... ,0..Max] OF ElementType > > ; Shape : ARRAY [0..OpenDepth-1] of CARDINAL > > END > > > > Max will be like the notorious TextLiteral.MaxBytes, i.e., we don't want any > > static limit here in the type of Elements, as it will be enforced dynamically, > > using Shape. But we don't want to just say REF ARRAY OF ElementType either, > > as this would mean another open array inside the dope, resulting in infinite > > recursion. > > > > On 07/22/2015 12:42 AM, Jay K wrote: > > > In the C backend I have a notion of "weak struct types" and "strong struct types". > > > > > > > > > "Strong" types have fields with types and names corresponding to the original Modula-3. i.e. they debug well. > > > > > > > > > "Weak" types have just arrays of characters (in a struct), sized/aligned to what the front end asked for. i.e. they debug poorly. > > > > > > > > > > > > Originally I had only weak types. > > > Ideally I have no weak types. > > > I'm down to very few weak types now. > > > I'd like to finish eliminating weak types. > > > > > > > > > > > > A quick investigation shows weak types come from open arrays and jmpbufs. > > > Open array temporaries from SUBARRAY specifically. > > > > > > > > > > > > Can we fix this? > > > > > > > > > > > > We have: > > > m3front/src/types/OpenArrayType.m3: > > > > > > PROCEDURE DeclareTemp (t: Type.T): CG.Var = > > > VAR > > > p := Reduce (t); > > > size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > RETURN CG.Declare_temp (size, Target.Address.align, > > > CG.Type.Struct, in_memory := TRUE); > > > END DeclareTemp; > > > > > > > > > PROCEDURE Compiler (p: P) = > > > VAR size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > Type.Compile (p.element); > > > CG.Declare_open_array (Type.GlobalUID(p), Type.GlobalUID(p.element), size); > > > END Compiler; > > > > > > > > > DeclareTemp is used in SUBARRAY expressions -- truly temporaries, > > > not variables authored by anyone in Modula-3. > > > > > > > > > Can this be easily fixed? > > > > > > > > > Thanks, > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > Rodney Bates > rodney.m.bates at acm.org > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 2 05:36:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 03:36:05 +0000 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? Message-ID: I propose the following: - A flag to m3front indicating if the backend "knows" what a jmpbuf is. - extend M3CG.Type = { int8, uint8, ... addr ... struct } by adding a jmpbuf element - if the backend "knows" what a jmpbuf is, then, vs. the current code, the frontend shall just declare a jmpbuf per try, leaving the backend to size it, and w/o using the upcoming alloca approach. - if the backend does not "know" what a jmpbuf is, then, well, there is the current approach of making a sized struct, or my approach that is almost working where we use alloca. The frontend will not output cgtype.jmpbuf unless the backend "knows" what it is. ok? The information as to the backend's knowledge of a jmpbuf shall beyet another internal m3front flag, like what guides nested functionoutput order. I was going to propose something slightly different, likem3front always use this new jmpbuf type, and existing backendscan just change it "back" to struct. The problem with that approach..it works easily with the current code, but does not addressmy upcoming alloca(sizeof_jmpbuf) change. In this way, what the C backend will do is: #include jmp_buf buf1; jmp_buf buf2; etc. leaving the size of the jmpbuf completely absent in the generated code,and allocating in the slightly more efficient way than alloca. Exception handling is special, so accomodating this seems reasonable. Yes, it might be temporary, replaced by libunwind stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 2 10:44:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 08:44:22 +0000 Subject: [M3devel] m3core.h broken In-Reply-To: <55BBDD0F.4080206@lcwb.coop> References: <55BBDD0F.4080206@lcwb.coop> Message-ID: Sorry, yes, my mistake.I should have used the UINT64/INT64 typedefs from nearby earlier.and my change wasn't well motivated anyway. - Jay > Date: Fri, 31 Jul 2015 15:39:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: [M3devel] m3core.h broken > > On AMD64_LINUX, with gcc 4.8.1, m3-libs/src/m3core.h suffers many compile errors, > when compiling on m3core and cm3. reverting line 363 of m3core.h makes them go away. > This is probably not the right way to fix this. > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 2 22:18:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 02 Aug 2015 15:18:58 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail>, Message-ID: <55BE7B32.7000502@lcwb.coop> On 08/02/2015 04:23 AM, Jay K wrote: > Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. > > so I have to move shared code to something called...ProcedureOrFinallyProcedure? > that otherwise was just in Procedure. I'm not sure what the breakage is, but for finally, the custom procedure generation consists only of: CG.Begin_procedure (p.handler.cg_proc); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc); in two places in TryFinStmt.m3. I would expect what you need to do to happen somewhere inside Stmt.Compile. Only there would it be known whether any TRYs are inside the FINALLY procedure. I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no doubt be compiler internally generated procedures. I would expect only those emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements inside. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3commit at elegosoft.com > Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > Date: Sun, 2 Aug 2015 02:20:51 +0000 > > Rodney, Do you have a test case that was broken before and fixed afterward? > > If possible, though not required, check something into m3-sys/m3tests? > > I don't mean to imply the code was correct before. I don't know, at all. > > Thank you, > - Jay > > > > Date: Sat, 1 Aug 2015 17:36:54 -0700 > From: rodney.m.bates at acm.org > To: m3commit at elegosoft.com > Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > > Branch: refs/heads/master > Home:https://github.com/modula3/cm3 > Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 > https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 > Author: Rodney Bates > Date: 2015-08-01 (Sat, 01 Aug 2015) > > Changed paths: > M m3-sys/m3front/src/misc/CG.m3 > > Log Message: > ----------- > Fix compiler-generated FINALLY procedure using parent procedure's temporary. > > Changes to be committed: > > modified: m3-sys/m3front/src/misc/CG.m3 > > CG was not stacking global variables used for keeping track of temporaries > when entering nested procedures. In the case of a compiler-generated > nested procedure for the finally part of a TRY--FINALLY block, this > meant the finally code could use a temporary allocated by the parent, > though not in use at the time. This violated the comment in M3CG_Ops.i3: > > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > > In cases of nested procedures explicitly declared in source code, this > didn't matter because they are located ahead of any of the parent > procedure's executable code, when it has no allocated temps, and they > free their own temps at their end, restoring the previous state. > > > > > _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > _______________________________________________ > M3commit mailing list > M3commit at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 3 09:05:17 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 00:05:17 -0700 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: <55BE7B32.7000502@lcwb.coop> References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail> <55BE7B32.7000502@lcwb.coop> Message-ID: <13B01241-502D-415A-A98E-F818DD8C77CA@gmail.com> Thank you for the obvious reminder I needed -- look at all begin_procedure calls. I briefly mulled over that my changes belong in Stmt or elsewhere, but, the thing is..maybe my understanding isn't right, Stmt occurs in many places, you can say "begin" anywhere. My changes are very related to procedures -- counting TRY per procedure, calling alloca once per procedure, near the start -- before any branches or TRYs. The breakage is that I was considering finally procs to be part of the enclosing function. Very wrong. I put the alloca & setting a local pointer per TRY circa Procedure.GenBody, which isn't for the "synthetic" procedures.. I'll poke around more. Thanks, - Jay On Aug 2, 2015, at 1:18 PM, "Rodney M. Bates" wrote: > > > On 08/02/2015 04:23 AM, Jay K wrote: >> Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. >> >> so I have to move shared code to something called...ProcedureOrFinallyProcedure? >> that otherwise was just in Procedure. > > I'm not sure what the breakage is, but for finally, the custom procedure generation consists > only of: > > CG.Begin_procedure (p.handler.cg_proc); > xc := Stmt.Compile (p.finally); > CG.Exit_proc (CG.Type.Void); > CG.End_procedure (p.handler.cg_proc); > > in two places in TryFinStmt.m3. I would expect what you need to do to happen > somewhere inside Stmt.Compile. Only there would it be known whether any TRYs > are inside the FINALLY procedure. > > I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), > RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no > doubt be compiler internally generated procedures. I would expect only those > emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements > inside. > > > >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3commit at elegosoft.com >> Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> Date: Sun, 2 Aug 2015 02:20:51 +0000 >> >> Rodney, Do you have a test case that was broken before and fixed afterward? >> >> If possible, though not required, check something into m3-sys/m3tests? >> >> I don't mean to imply the code was correct before. I don't know, at all. >> >> Thank you, >> - Jay >> >> >> >> Date: Sat, 1 Aug 2015 17:36:54 -0700 >> From: rodney.m.bates at acm.org >> To: m3commit at elegosoft.com >> Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> >> Branch: refs/heads/master >> Home:https://github.com/modula3/cm3 >> Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> Author: Rodney Bates >> Date: 2015-08-01 (Sat, 01 Aug 2015) >> >> Changed paths: >> M m3-sys/m3front/src/misc/CG.m3 >> >> Log Message: >> ----------- >> Fix compiler-generated FINALLY procedure using parent procedure's temporary. >> >> Changes to be committed: >> >> modified: m3-sys/m3front/src/misc/CG.m3 >> >> CG was not stacking global variables used for keeping track of temporaries >> when entering nested procedures. In the case of a compiler-generated >> nested procedure for the finally part of a TRY--FINALLY block, this >> meant the finally code could use a temporary allocated by the parent, >> though not in use at the time. This violated the comment in M3CG_Ops.i3: >> >> declare_temp (s: ByteSize; a: Alignment; t: Type; >> in_memory: BOOLEAN): Var; >> (* declares an anonymous local variable. Temps are declared >> and freed between their containing procedure's begin_procedure and >> end_procedure calls. Temps are never referenced by nested procedures. *) >> >> In cases of nested procedures explicitly declared in source code, this >> didn't matter because they are located ahead of any of the parent >> procedure's executable code, when it has no allocated temps, and they >> free their own temps at their end, restoring the previous state. >> >> >> >> >> _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit >> >> >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 04:35:10 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 19:35:10 -0700 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? In-Reply-To: References: Message-ID: <542D0438-1C2D-463A-96BC-E7CA6D0F2122@gmail.com> The alloca approach is fairly sound, w/o this change. However in the "cake and eat it too" dept..the C backend could instead say: #include setjmp.h . . . jmp_buf x; thereby still not knowing the size of a jmp_buf but generating correct portable slightly more efficient code. Also doing a better job of telling the C compiler that setjmp is being used -- rather than e.g. declaring setjmp ourselves, which I think is somewhat dubious, though faster to compile... I admit that the efficiency still stinks & we have much greater efficiency goals here. - Jay On Aug 3, 2015, at 7:05 PM, Antony Hosking wrote: > Remind me again what the issue is with the alloca approach? > >> On Aug 2, 2015, at 1:36 PM, Jay K wrote: >> >> I propose the following: >> >> >> - A flag to m3front indicating if the backend "knows" what a jmpbuf is. >> >> - extend M3CG.Type = { int8, uint8, ... addr ... struct } >> by adding a jmpbuf element >> >> >> - if the backend "knows" what a jmpbuf is, then, vs. the current code, >> the frontend shall just declare a jmpbuf per try, leaving the backend >> to size it, and w/o using the upcoming alloca approach. >> >> >> - if the backend does not "know" what a jmpbuf is, then, well, there >> is the current approach of making a sized struct, or my approach >> that is almost working where we use alloca. The frontend >> will not output cgtype.jmpbuf unless the backend "knows" what it is. >> >> >> ok? >> >> >> The information as to the backend's knowledge of a jmpbuf shall be >> yet another internal m3front flag, like what guides nested function >> output order. >> >> >> I was going to propose something slightly different, like >> m3front always use this new jmpbuf type, and existing backends >> can just change it "back" to struct. The problem with that approach.. >> it works easily with the current code, but does not address >> my upcoming alloca(sizeof_jmpbuf) change. >> >> >> In this way, what the C backend will do is: >> #include >> jmp_buf buf1; >> jmp_buf buf2; >> etc. >> >> >> leaving the size of the jmpbuf completely absent in the generated code, >> and allocating in the slightly more efficient way than alloca. >> >> >> Exception handling is special, so accomodating this seems reasonable. >> >> >> Yes, it might be temporary, replaced by libunwind stuff. >> >> >> Thank you, >> - Jay >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 09:36:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 07:36:21 +0000 Subject: [M3devel] Modula-2 parsing Message-ID: Procedure.m3: Do we need this?I know it isn't much, but remove it? ELSIF (cur.token = TK.tSEMI) THEN t.body := NEW (Body, self := t); ProcBody.Push (t.body); (* try accepting the Modula-2 syntax *) Error.ID (id, "expecting \'=\' before procedure body"); GetToken (); (* ; *) t.syms := Scope.PushNew (TRUE, id); t.block := BlockStmt.Parse (FALSE); t.fails := BlockStmt.ExtractFails (t.block); t.end_origin := Scanner.offset; final_id := MatchID (); IF (final_id # id) THEN Error.ID (id, "Initial name doesn\'t match final name"); END; Scope.PopNew (); ProcBody.Pop (); ELSE It always errors? But I guess it errors and checks nicer than it might otherwise? Why am I looking here? Tangential: I'm looking for where/how to model counting TRYs per "procedure"and doing the alloca at the start of a "procedure". "procedure"'s definition for my context is being worked on. It appears to be three things: 1) Things called "PROCEDURE". 2) FINALLY blocks, sometimes 3) "Module main" I was looking for what they have in common already, and it looks like maybe "ProcBody.Push". So I was looking all of them. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 12:02:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 10:02:03 +0000 Subject: [M3devel] declaring/initializing variables "anywhere" Message-ID: I had: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count;BEGIN which is fairly nice and elegant.I like the static type inference. but p could be NIL so I need like: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module: Module.T; alloca: CG.Proc; size: CG.Var; try_count: INTEGER;BEGIN IF p = NIL THEN RETURN END; module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count; double the lines, and a need to state types that I didn't have to state before, OR I can use WITH to get the "best" of both, but implied extra indentation. Pretty much all other mainstream languages in use today,except C89, are "better" than this, they all let youboth delay local variable declaration until arbitrarilyinto a function, but with implying that you should indent further,or close an extra block. Most mainstream languages also share Modula-3's type inferenceand let you omit the type, except C. Can we update Modula-3 here somehow? Or just use "with"? Proposals for a syntax that works well with the existing language and compiler? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 4 18:42:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 11:42:23 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: References: Message-ID: <55C0EB6F.7050704@lcwb.coop> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually involves finding one's way around in hundreds of thousands of lines of unfamiliar code. If you're exceptionally lucky, an eventual one-line change can be figured out after vetting a mere several tens of lines. Very often, it is more like a few hundreds, and not infrequently, even more. The average line of actively maintained code is written once and read something like seven times. (I don't remember where that number came from.) Speaking for myself, even most brand new code gets read at least seven times before it gets past my own testing, before it gets passed to anybody else to either use or further test. Moreover, the initial writer already has lots of context in his head that a maintainer has to ferret out. What I am leading to is that things should be *locally explicit*, far more than we usually see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches who have to come along and figure it out later to constantly locate, open, scan another source file, every fourth token they read, is penny wise and pound foolish. It's classic case of instant-gratification immaturity. Ensuring a ton of aggravation later, to save an ounce of effort now. It's just plain laziness. (more below:) On 08/04/2015 05:02 AM, Jay K wrote: > I had: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); So, coded like this, the poor schmuck who has to come along and figure this out will have to find Module.i3 and search it for declarations of Current, GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly more steps, as with alloca, whose type, as we will find out, requires going to yet another source file CG.i3. It can be even worse, e.g.: VAR V := Ptr^.Field[I].method(X); You have to track down the type of Ptr, to find Field, to find the type of its elements, to find method, to find its result type. Sometimes it's obvious from identifier names, but more often, it is not, or at least only too vaguely for serious software work. Is a module denoted by an abstract type? An integer numbering of modules? something else? Very often, there are two or three different types that denote a module or whatever. (An aside: In C++, things often get dramatically worse when coders, overly enamored with cleverness for its own sake, define implicit type conversions among these, so the poor maintainer has even less idea what is going on. Moreover, the conversions could be declared anywhere in the tens of transitively #included files, and there is no path leading to them, only exhaustive search through all the files. And this possibility has to be considered for every single assignment and actual parameter.) Instead, I always code cases like this by putting both the type and the initializer in the declaration: VAR module : Module.T := Module.Current (); I even often do it this way when the initializer is a literal. For one thing, I may want a subrange for the type of the variable. The exception is when the initializer itself also names the type. It's shorter, but at no loss of explicitness. VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > try_count := p.try_count; > BEGIN > > > which is fairly nice and elegant. > I like the static type inference. > > > but p could be NIL so I need like: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module: Module.T; > alloca: CG.Proc; > size: CG.Var; > try_count: INTEGER; > BEGIN > IF p = NIL THEN RETURN END; > module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); > try_count := p.try_count; > > > double the lines, and a need to state types that I didn't have to state before, > This will save wasted computation calling the functions, if it turns out p=NIL. Depending on the purpose of the code. In some code, I care about efficiency at this level. > > OR I can use WITH to get the "best" of both, but implied extra indentation. > If I were to propose any language change at all, it would be to allow a WITH binding to optionally specify the type too, for exactly the same reasons. WITH w : T = Var.method(x) DO ... Many a WITH statement has been far too hard to read, requiring the side trips of checking other source files to see what this value really is. > > > Pretty much all other mainstream languages in use today, > except C89, are "better" than this, they all let you > both delay local variable declaration until arbitrarily > into a function, but with implying that you should indent further, > or close an extra block. > To be clear, I definitely do not advocate allowing declarations and statements to be arbitrarily mixed in a block, BUT... if we did it, at least the decls would be syntactically explicit with a VAR, and we would not have to "define" our language with words like "if it looks like a declaration...". > > Most mainstream languages also share Modula-3's type inference > and let you omit the type, except C. > > > Can we update Modula-3 here somehow? > > > Or just use "with"? > > > Proposals for a syntax that works well with the existing language and compiler? > > > Thank you, > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 20:07:54 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 11:07:54 -0700 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <55C0EB6F.7050704@lcwb.coop> References: <55C0EB6F.7050704@lcwb.coop> Message-ID: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> But...Modula-3 already has this in VAR and WITH. Every language has this for temporaries: F1(F2() + F3()) Are these bad? Too much explicit detail can actually get in the way of reading the code. People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. - Jay On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually > involves finding one's way around in hundreds of thousands of lines of unfamiliar code. > If you're exceptionally lucky, an eventual one-line change can be figured out after vetting > a mere several tens of lines. Very often, it is more like a few hundreds, and not > infrequently, even more. > > The average line of actively maintained code is written once and read something like > seven times. (I don't remember where that number came from.) Speaking for myself, > even most brand new code gets read at least seven times before it gets past my own testing, > before it gets passed to anybody else to either use or further test. Moreover, the initial > writer already has lots of context in his head that a maintainer has to ferret out. > > What I am leading to is that things should be *locally explicit*, far more than we usually > see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches > who have to come along and figure it out later to constantly locate, open, scan another > source file, every fourth token they read, is penny wise and pound foolish. It's classic > case of instant-gratification immaturity. Ensuring a ton of aggravation later, to > save an ounce of effort now. It's just plain laziness. > > (more below:) > > On 08/04/2015 05:02 AM, Jay K wrote: >> I had: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); > > So, coded like this, the poor schmuck who has to come along and figure this > out will have to find Module.i3 and search it for declarations of Current, > GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly > more steps, as with alloca, whose type, as we will find out, requires > going to yet another source file CG.i3. It can be even worse, e.g.: > > VAR V := Ptr^.Field[I].method(X); > > You have to track down the type of Ptr, to find Field, to find the type of > its elements, to find method, to find its result type. > > Sometimes it's obvious from identifier names, but more often, it is not, or at > least only too vaguely for serious software work. Is a module denoted by an > abstract type? An integer numbering of modules? something else? Very often, > there are two or three different types that denote a module or whatever. > > (An aside: In C++, things often get dramatically worse when coders, overly > enamored with cleverness for its own sake, define implicit type conversions > among these, so the poor maintainer has even less idea what is going on. > Moreover, the conversions could be declared anywhere in the tens of transitively > #included files, and there is no path leading to them, only exhaustive search > through all the files. And this possibility has to be considered for every > single assignment and actual parameter.) > > Instead, I always code cases like this by putting both the type and the initializer > in the declaration: > VAR module : Module.T := Module.Current (); > > I even often do it this way when the initializer is a literal. For one thing, > I may want a subrange for the type of the variable. > > The exception is when the initializer itself also names the type. It's shorter, > but at no loss of explicitness. > > VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > >> try_count := p.try_count; >> BEGIN >> >> >> which is fairly nice and elegant. >> I like the static type inference. >> >> >> but p could be NIL so I need like: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module: Module.T; >> alloca: CG.Proc; >> size: CG.Var; >> try_count: INTEGER; >> BEGIN >> IF p = NIL THEN RETURN END; >> module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); >> try_count := p.try_count; >> >> >> double the lines, and a need to state types that I didn't have to state before, > > This will save wasted computation calling the functions, if it turns out p=NIL. > Depending on the purpose of the code. In some code, I care about efficiency > at this level. > >> >> OR I can use WITH to get the "best" of both, but implied extra indentation. > > If I were to propose any language change at all, it would be to allow a WITH > binding to optionally specify the type too, for exactly the same reasons. > > WITH w : T = Var.method(x) DO ... > > Many a WITH statement has been far too hard to read, requiring the side trips of > checking other source files to see what this value really is. > > > >> >> >> Pretty much all other mainstream languages in use today, >> except C89, are "better" than this, they all let you >> both delay local variable declaration until arbitrarily >> into a function, but with implying that you should indent further, >> or close an extra block. > > To be clear, I definitely do not advocate allowing declarations and statements > to be arbitrarily mixed in a block, BUT... if we did it, at least the decls > would be syntactically explicit with a VAR, and we would not have to "define" > our language with words like "if it looks like a declaration...". > >> >> Most mainstream languages also share Modula-3's type inference >> and let you omit the type, except C. >> >> >> Can we update Modula-3 here somehow? >> >> >> Or just use "with"? >> >> >> Proposals for a syntax that works well with the existing language and compiler? >> >> >> Thank you, >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:18:15 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C101E7.60106@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:30:47 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:30:47 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C104D7.4040601@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > It looks to me like the comment "accepting" is misleading, since it produces an error message. My guess is that this is a specialized syntax error recovery technique, designed to do a better job of continuing when this particular error occurs, probably based on experience. Modula-3 evolved from Modula-2+, which evolved from Modula-2, and I'll bet there was a lot of code converted from one language to another. This would have been a common leftover from that process. I'd change the comment, but otherwise leave it. > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > Are you counting TRYs in the front end, or on the back side of the cm3 IR? If the former, try looking at the way "fails" is collected from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, to a field of Procedure.T. But it has to come out of TryStmt.Parse, to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not a list? If the latter, aren't you already doing two passes over the IR? > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 21:02:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 14:02:53 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> References: <55C0EB6F.7050704@lcwb.coop> <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> Message-ID: <55C10C5D.40402@lcwb.coop> On 08/04/2015 01:07 PM, Jay wrote: > But...Modula-3 already has this in VAR and WITH. Huh? WithSt = WITH Binding {"," Binding} DO S END. Binding = Id "=" Expr. ^ no explicit type allowed here. > > Every language has this for temporaries: > > F1(F2() + F3()) > > Are these bad? > > Too much explicit detail can actually get in the way of reading the code. Yes, it is a balance. But much of the world is waaaay to far on the implicit side. Explicitly typing proper subexpressions would be far too tedious. But that is exactly why the rules for inferring should not be highly complex, multi-pronged, or depend on a huge amount of context. > > > People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. > Yes, if such tools were ubiquitous and dependable, it would go a long way to help. > > "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. > Yes, there is a tension in making things self-adapt to one-place changes. But if it's as complicated as is often the case, the time saved editing will be overwhelmed by the time figuring out what to change. And trusting a language/compiler to make the deduction that works right while the human can't understand it, or just doesn't have the time and gumption, is insane. > > - Jay > > On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > >> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually >> involves finding one's way around in hundreds of thousands of lines of unfamiliar code. >> If you're exceptionally lucky, an eventual one-line change can be figured out after vetting >> a mere several tens of lines. Very often, it is more like a few hundreds, and not >> infrequently, even more. >> >> The average line of actively maintained code is written once and read something like >> seven times. (I don't remember where that number came from.) Speaking for myself, >> even most brand new code gets read at least seven times before it gets past my own testing, >> before it gets passed to anybody else to either use or further test. Moreover, the initial >> writer already has lots of context in his head that a maintainer has to ferret out. >> >> What I am leading to is that things should be *locally explicit*, far more than we usually >> see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches >> who have to come along and figure it out later to constantly locate, open, scan another >> source file, every fourth token they read, is penny wise and pound foolish. It's classic >> case of instant-gratification immaturity. Ensuring a ton of aggravation later, to >> save an ounce of effort now. It's just plain laziness. >> >> (more below:) >> >> On 08/04/2015 05:02 AM, Jay K wrote: >>> I had: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >> >> So, coded like this, the poor schmuck who has to come along and figure this >> out will have to find Module.i3 and search it for declarations of Current, >> GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly >> more steps, as with alloca, whose type, as we will find out, requires >> going to yet another source file CG.i3. It can be even worse, e.g.: >> >> VAR V := Ptr^.Field[I].method(X); >> >> You have to track down the type of Ptr, to find Field, to find the type of >> its elements, to find method, to find its result type. >> >> Sometimes it's obvious from identifier names, but more often, it is not, or at >> least only too vaguely for serious software work. Is a module denoted by an >> abstract type? An integer numbering of modules? something else? Very often, >> there are two or three different types that denote a module or whatever. >> >> (An aside: In C++, things often get dramatically worse when coders, overly >> enamored with cleverness for its own sake, define implicit type conversions >> among these, so the poor maintainer has even less idea what is going on. >> Moreover, the conversions could be declared anywhere in the tens of transitively >> #included files, and there is no path leading to them, only exhaustive search >> through all the files. And this possibility has to be considered for every >> single assignment and actual parameter.) >> >> Instead, I always code cases like this by putting both the type and the initializer >> in the declaration: >> VAR module : Module.T := Module.Current (); >> >> I even often do it this way when the initializer is a literal. For one thing, >> I may want a subrange for the type of the variable. >> >> The exception is when the initializer itself also names the type. It's shorter, >> but at no loss of explicitness. >> >> VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; >> >>> try_count := p.try_count; >>> BEGIN >>> >>> >>> which is fairly nice and elegant. >>> I like the static type inference. >>> >>> >>> but p could be NIL so I need like: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module: Module.T; >>> alloca: CG.Proc; >>> size: CG.Var; >>> try_count: INTEGER; >>> BEGIN >>> IF p = NIL THEN RETURN END; >>> module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >>> try_count := p.try_count; >>> >>> >>> double the lines, and a need to state types that I didn't have to state before, >> >> This will save wasted computation calling the functions, if it turns out p=NIL. >> Depending on the purpose of the code. In some code, I care about efficiency >> at this level. >> >>> >>> OR I can use WITH to get the "best" of both, but implied extra indentation. >> >> If I were to propose any language change at all, it would be to allow a WITH >> binding to optionally specify the type too, for exactly the same reasons. >> >> WITH w : T = Var.method(x) DO ... >> >> Many a WITH statement has been far too hard to read, requiring the side trips of >> checking other source files to see what this value really is. >> >> >> >>> >>> >>> Pretty much all other mainstream languages in use today, >>> except C89, are "better" than this, they all let you >>> both delay local variable declaration until arbitrarily >>> into a function, but with implying that you should indent further, >>> or close an extra block. >> >> To be clear, I definitely do not advocate allowing declarations and statements >> to be arbitrarily mixed in a block, BUT... if we did it, at least the decls >> would be syntactically explicit with a VAR, and we would not have to "define" >> our language with words like "if it looks like a declaration...". >> >>> >>> Most mainstream languages also share Modula-3's type inference >>> and let you omit the type, except C. >>> >>> >>> Can we update Modula-3 here somehow? >>> >>> >>> Or just use "with"? >>> >>> >>> Proposals for a syntax that works well with the existing language and compiler? >>> >>> >>> Thank you, >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:47:16 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:47:16 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: <55C104D7.4040601@lcwb.coop> References: <55C104D7.4040601@lcwb.coop> Message-ID: Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. This change is for all backends. No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. If a backend wants to know about jmp_buf, that ability will be added soon thereafter. I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? There are obvious counter points but most seem moot these days. Yes, it means a larger working set and more likely-traced allocations. Ignoring the tracing though, this is how cm3cg has always worked. And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. At least one of the transforms M3C does could be moved to m3front -- removing unused locals. And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. Given the M3C code, this is easy at this point. Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. I'll send diffs later. I think I'm quite close. Thoughts? - Jay On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > On 08/04/2015 02:36 AM, Jay K wrote: >> Procedure.m3: >> >> Do we need this? >> I know it isn't much, but remove it? > > It looks to me like the comment "accepting" is misleading, since it produces > an error message. My guess is that this is a specialized syntax error recovery > technique, designed to do a better job of continuing when this particular > error occurs, probably based on experience. Modula-3 evolved from Modula-2+, > which evolved from Modula-2, and I'll bet there was a lot of code converted > from one language to another. This would have been a common leftover from > that process. > > I'd change the comment, but otherwise leave it. > >> >> ELSIF (cur.token = TK.tSEMI) THEN >> t.body := NEW (Body, self := t); >> ProcBody.Push (t.body); >> (* try accepting the Modula-2 syntax *) >> Error.ID (id, "expecting \'=\' before procedure body"); >> GetToken (); (* ; *) >> t.syms := Scope.PushNew (TRUE, id); >> t.block := BlockStmt.Parse (FALSE); >> t.fails := BlockStmt.ExtractFails (t.block); >> t.end_origin := Scanner.offset; >> final_id := MatchID (); >> IF (final_id # id) THEN >> Error.ID (id, "Initial name doesn\'t match final name"); >> END; >> Scope.PopNew (); >> ProcBody.Pop (); >> ELSE >> >> >> It always errors? >> >> But I guess it errors and checks nicer than it might otherwise? >> >> Why am I looking here? >> >> Tangential: >> >> I'm looking for where/how to model counting TRYs per "procedure" >> and doing the alloca at the start of a "procedure". > > Are you counting TRYs in the front end, or on the back side of the > cm3 IR? If the former, try looking at the way "fails" is collected > from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, > to a field of Procedure.T. But it has to come out of TryStmt.Parse, > to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not > a list? > > If the latter, aren't you already doing two passes over the IR? > >> >> "procedure"'s definition for my context is being worked on. >> It appears to be three things: >> 1) Things called "PROCEDURE". >> 2) FINALLY blocks, sometimes >> 3) "Module main" >> >> I was looking for what they have in common already, >> and it looks like maybe "ProcBody.Push". >> >> So I was looking all of them. >> >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:56:44 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:56:44 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: <55C104D7.4040601@lcwb.coop> Message-ID: Ps: yes just a count. A list could be useful but I achieved similar end another way. Each try records the current count as its index before incrementing it. Then we allocate an array of that many vars, which are initialized (perhaps too preemptively) to point into the bulk alloca. Then each TRY uses the corresponding variable. With a list I could poke each var into each Try*.T -- introducing a common base. The difference is negligible -- in m3front an extra array per procedure and extra indirection when compiling TRY, vs. walking the list. The generated code would be the same either way. The "spreading out of the division of labor" also similar. I introduced module Jmpbufs in m3front to hold most of this. I consider Marker, even existing functions in it, but it didn't seem to fit well. - Jay On Aug 4, 2015, at 12:47 PM, Jay wrote: > Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. > > > "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. > > > This change is for all backends. > > No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. > > If a backend wants to know about jmp_buf, that ability will be added soon thereafter. > > > I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). > > > So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? > > > There are obvious counter points but most seem moot these days. > > > Yes, it means a larger working set and more likely-traced allocations. > > > Ignoring the tracing though, this is how cm3cg has always worked. > > > And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. > > > At least one of the transforms M3C does could be moved to m3front -- removing unused locals. > > > And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. > > > Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. > > > Given the M3C code, this is easy at this point. > > Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. > > I'll send diffs later. I think I'm quite close. > > Thoughts? > > - Jay > > On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > >> On 08/04/2015 02:36 AM, Jay K wrote: >>> Procedure.m3: >>> >>> Do we need this? >>> I know it isn't much, but remove it? >> >> It looks to me like the comment "accepting" is misleading, since it produces >> an error message. My guess is that this is a specialized syntax error recovery >> technique, designed to do a better job of continuing when this particular >> error occurs, probably based on experience. Modula-3 evolved from Modula-2+, >> which evolved from Modula-2, and I'll bet there was a lot of code converted >> from one language to another. This would have been a common leftover from >> that process. >> >> I'd change the comment, but otherwise leave it. >> >>> >>> ELSIF (cur.token = TK.tSEMI) THEN >>> t.body := NEW (Body, self := t); >>> ProcBody.Push (t.body); >>> (* try accepting the Modula-2 syntax *) >>> Error.ID (id, "expecting \'=\' before procedure body"); >>> GetToken (); (* ; *) >>> t.syms := Scope.PushNew (TRUE, id); >>> t.block := BlockStmt.Parse (FALSE); >>> t.fails := BlockStmt.ExtractFails (t.block); >>> t.end_origin := Scanner.offset; >>> final_id := MatchID (); >>> IF (final_id # id) THEN >>> Error.ID (id, "Initial name doesn\'t match final name"); >>> END; >>> Scope.PopNew (); >>> ProcBody.Pop (); >>> ELSE >>> >>> >>> It always errors? >>> >>> But I guess it errors and checks nicer than it might otherwise? >>> >>> Why am I looking here? >>> >>> Tangential: >>> >>> I'm looking for where/how to model counting TRYs per "procedure" >>> and doing the alloca at the start of a "procedure". >> >> Are you counting TRYs in the front end, or on the back side of the >> cm3 IR? If the former, try looking at the way "fails" is collected >> from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, >> to a field of Procedure.T. But it has to come out of TryStmt.Parse, >> to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not >> a list? >> >> If the latter, aren't you already doing two passes over the IR? >> >>> >>> "procedure"'s definition for my context is being worked on. >>> It appears to be three things: >>> 1) Things called "PROCEDURE". >>> 2) FINALLY blocks, sometimes >>> 3) "Module main" >>> >>> I was looking for what they have in common already, >>> and it looks like maybe "ProcBody.Push". >>> >>> So I was looking all of them. >>> >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 5 09:17:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 07:17:42 +0000 Subject: [M3devel] names in (hopefully) upcoming changes Message-ID: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 11:15:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:15:22 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review Message-ID: Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 22.txt URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.i3 Type: application/octet-stream Size: 1138 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.m3 Type: application/octet-stream Size: 5020 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Aug 5 11:49:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:49:15 +0000 Subject: [M3devel] setjmp/jmpbuf declaration vs. #include setjmp.h Message-ID: fyi...having updated tools, I now get: cm3.d/Main.m3.c:757:1: warning: declaration of built-in function '_setjmp' requires inclusion of the header [-Wbuiltin-requires-header] which I think helps vindicate the upcoming addition of CGType.Jmpbuf. That is, we shouldn't declare setjmp ourselves.And therefore, we also can't pretend jmpbuf is just a struct with a size, but must use whatever setjmp.h says.I hope to fix this after the first round of "alloca(jmpbuf)". Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 5 17:54:48 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 5 Aug 2015 15:54:48 +0000 (UTC) Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: Message-ID: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Hi all:long time ago, a proposal for supporting more higher level R like Aimer Diwan et al, would benefit, support C -> Fortran compilation to further parallelize in big machines (Virtual Speculative Parallel Machine), see [1]:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear Anyway, hearing of what you are trying to do especially support OPenVMS, etc, is nice Thanks in advance [1]R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of Language Transcription?, en Languages and Compilers for Parallel Computing, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. El Mi?rcoles 5 de agosto de 2015 2:17, Jay K escribi?: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called?Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed? "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks,?- Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 20:53:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 18:53:32 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 Message-ID: Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 6 01:45:02 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> References: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Message-ID: Just wondering if the OpenVMS port is being used or if indeed its been tested. I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter On Thu, Aug 6, 2015 at 1:54 AM, Daniel Alejandro Benavides D. < dabenavidesd at yahoo.es> wrote: > Hi all: > long time ago, a proposal for supporting more higher level R like Aimer > Diwan et al, would benefit, support C -> Fortran compilation to further > parallelize in big machines (Virtual Speculative Parallel Machine), see [1]: > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear > > > > > Anyway, hearing of what you are trying to do especially support OPenVMS, > etc, is nice > > Thanks in advance > > [1] > R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of > Language Transcription?, en *Languages and Compilers for Parallel > Computing*, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. > Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. > > > > El Mi?rcoles 5 de agosto de 2015 2:17, Jay K > escribi?: > > > I need some names. > > C has a type "jmp_buf" -- with no "u" and with an underscore. > > > I need a module in m3front for counting tries and managing jmp_bufs. > I call it Jmpbufs. > It could be Jumpbufs. > Or JumpBuffers. > Perhaps that is too direct. > Or sjljeh (setjmp/longjmp exception handling) > Or worked into the existing Marker. > > > A "constant variable" in C to hold sizeof(jmp_buf). > So far this was called Csetjmp__Jumpbuf_size. > However this "constant variable" should never be used from Modula-3. > Putting in a Modula-3 interface (Csetjmp), with the two level naming and a > double underscore is unnecessary and I suggest should not be done. > > > This name is an "interface" between m3front and m3core, i.e. > m3core/src/unix/Common/Uconstants.c. > It is always statically linked. > > > Potential names here are: > > > m3_jmpbuf_size my favorate at the moment > jmpbuf_size > Csetjmp_Jumpbuf_size already present by this name, but can be changed > > > > "alloca" > > > This is (will be) part of the interface between m3front and every backend. > While it is *almost* pass-through, it isn't. > The C backend targeting non-Win32, non-VMS, can just pass through the name. > But every other case in every other backend must treat the name specially. > cm3cg has to convert it to "builtin_alloca". > LLVM probably is like cm3cg. > Win32 non-C has to change it to __chkstk or perhaps _chkstk. > Win32 C has to change it to _alloca. > OpenVMS has to change it to __ALLOCA or such. > That is, the symbol "alloca" is almost but not quite portable in C. > > > Again the function is never called from Modula-3 and need not be in an > interface. > > > Potential names here are: > > "alloca" what I'm using currently and remains my favorate. > "m3_alloca" > "m3_allocate_jmpbufs" > > > So far I'm calling it "alloca". > > > Thoughts/suggestions/criticisms? > > > Just wait for the larger diff? > > > As well, some of this could be more than "just random string function > names". > We could make them CONSTs in M3CG_Ops.i3. > > > We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" > and leave the "lowering" always to the backend -- this isn't likely, as > many backends can share part of the lowering. > > > But adding alloca to M3CG_Ops.i3 might be reasonable. > > > Thanks, > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:06:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:06:02 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 In-Reply-To: References: , Message-ID: So far the problems aren't in anything you did, and likely won't be. They are all subtle minor things. I gave a detailed commit. I can repeat it if needed. - Jay Subject: Re: [M3devel] cm3 on Mac OSX 10.10.4 From: hosking at purdue.edu Date: Thu, 6 Aug 2015 09:51:54 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Keep me posted on the problems. I did the original OS X port. On Aug 6, 2015, at 4:53 AM, Jay K wrote:Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:12:18 +0000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: , <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com>, Message-ID: I doubt the OpenVMS port is being used.It may or may not be complete. I managed to construct a Mac-hosted OpenVMS-targeted gcc.I built a native gcc with it. I didn't set the environment variables on the OpenVMS machinefor it to work. The biggest problem in all that is building the "sysroot".i.e. the headers and libraries. I built a bootstrap archive with cm3cg.I went to the trouble of building a ".com" file (makefile/scripty thing).So I might have gotten a "cm3" out of it. I don't remember. If someone has an OpenVMS machine available via ssh I might give it a try.I was working on an Alpha. I can see if it is still available. Also of note -- IA64 would need a little work, for all operating systems -- OpenVMS, HP-UX, Linux, FreeBSD, NT -- to scan the upper half of the stack. Thanks, - Jay From: peter.mckinna at gmail.com Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: Re: [M3devel] names in (hopefully) upcoming changes CC: jay.krell at cornell.edu; m3devel at elegosoft.com Just wondering if the OpenVMS port is being used or if indeed its been tested.I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 10:46:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 08:46:23 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review In-Reply-To: References: Message-ID: Any objections? criticisms? suggestions?I'd like to commit this. My bootstrapping procedure (boot1.py) precludes cm3 incompatible with m3core/libm3 -- so this is holding up moving to 10.10.4, unless I do some git gyrations. Bootstrapping really needs to produce the matching m3core and libm3, so target can have older/newer source. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: alloca(sizeof(jmp_buf)) changes for review Date: Wed, 5 Aug 2015 09:15:22 +0000 Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:31:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:31:14 +0000 Subject: [M3devel] alloca(jmpbuf_size) is in In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, Message-ID: I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:33:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:33:00 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? Message-ID: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:01:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:01:30 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , Message-ID: And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:02:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:02:42 +0000 Subject: [M3devel] updating m3cg? Message-ID: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:05:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:05:27 +0000 Subject: [M3devel] multi-threaded m3front? Message-ID: Is anyone interested in updating m3front to be multi-threaded? I haven't seen a single core system in a while. Surely each module can be compiled separately, possibly with some serialization around compiling interfaces? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 18:18:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 16:18:59 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> Message-ID: https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Aug 9 18:58:41 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 09 Aug 2015 09:58:41 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150809165841.BA38C1A2062@async.async.caltech.edu> I think this is a great idea. I made the back-end parallel already (at least for the version of the compiler that calls the gcc-based backend). To enable that you have to set "M3_PARALLEL_BACK" to the number of threads you want to spawn in your cm3.cfg . Mika Jay K writes: >--===============7278135725256123965== >Content-Type: multipart/alternative; > boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Is anyone interested in updating m3front to be multi-threaded? >I haven't seen a single core system in a while. >Surely each module can be compiled separately=2C possibly with some seriali= >zation around compiling interfaces? >Thanks=2C - Jay > > = > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
Is anyone interested in updating= > m3front to be multi-threaded?

I haven't seen a single c= >ore system in a while.

Surely each module can be c= >ompiled separately=2C possibly with some serialization around compiling int= >erfaces?

Thanks=2C
 =3B- Jay

= >
>= > >--_9e876ef9-e673-486b-befa-abff52e19a99_-- > >--===============7278135725256123965== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============7278135725256123965==-- From rodney_bates at lcwb.coop Sun Aug 9 21:05:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:05:57 -0500 Subject: [M3devel] updating m3cg? In-Reply-To: References: Message-ID: <55C7A495.6090409@lcwb.coop> Not high on my priority list, but I wouldn't object. On 08/08/2015 10:02 PM, Jay K wrote: > Is there any interesting in updating the gcc backend from 4.7, like to 5.2? > I'm guessing not really. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 9 21:14:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:14:48 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150809165841.BA38C1A2062@async.async.caltech.edu> References: <20150809165841.BA38C1A2062@async.async.caltech.edu> Message-ID: <55C7A6A8.2050805@lcwb.coop> On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > I think this is a great idea. I made the back-end parallel already (at > least for the version of the compiler that calls the gcc-based backend). I believe this works for any executable(s) separate from cm3 itself. > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > you want to spawn in your cm3.cfg . > > Mika > > Jay K writes: >> --===============7278135725256123965== >> Content-Type: multipart/alternative; >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> Is anyone interested in updating m3front to be multi-threaded? >> I haven't seen a single core system in a while. >> Surely each module can be compiled separately=2C possibly with some seriali= >> zation around compiling interfaces? >> Thanks=2C - Jay >> >> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
Is anyone interested in updating= >> m3front to be multi-threaded?

I haven't seen a single c= >> ore system in a while.

Surely each module can be c= >> ompiled separately=2C possibly with some serialization around compiling int= >> erfaces?

Thanks=2C
 =3B- Jay

= >>
>> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- >> >> --===============7278135725256123965== >> Content-Type: text/plain; charset="us-ascii" >> MIME-Version: 1.0 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: inline >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> --===============7278135725256123965==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 9 21:18:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:18:12 +0000 Subject: [M3devel] updating m3cg? In-Reply-To: <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> References: , <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> Message-ID: The level of change of supported targets would likely be small either way.Maybe a little lost, maybe a little gained.The C or C++ backend to me has the most reach. - Jay Subject: Re: [M3devel] updating m3cg? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:05:02 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu If it gets us running on newer targets then yes. Sent from my iPad On 9 Aug 2015, at 1:02 pm, Jay K wrote: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 21:20:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:20:01 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <55C7A6A8.2050805@lcwb.coop> References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: Right, agreed -- this should work for LLVM, and the C compiler.LLVM should be multi-threadable in-proc too. Fixing m3front itself -- we'd have to identify and deal with all the globals.For the alloca/setjmp work I put the "globals" in Module.T. - Jay > Date: Sun, 9 Aug 2015 14:14:48 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > I think this is a great idea. I made the back-end parallel already (at > > least for the version of the compiler that calls the gcc-based backend). > > I believe this works for any executable(s) separate from cm3 itself. > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > you want to spawn in your cm3.cfg . > > > > Mika > > > > Jay K writes: > >> --===============7278135725256123965== > >> Content-Type: multipart/alternative; > >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> Is anyone interested in updating m3front to be multi-threaded? > >> I haven't seen a single core system in a while. > >> Surely each module can be compiled separately=2C possibly with some seriali= > >> zation around compiling interfaces? > >> Thanks=2C - Jay > >> > >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
Is anyone interested in updating= > >> m3front to be multi-threaded?

I haven't seen a single c= > >> ore system in a while.

Surely each module can be c= > >> ompiled separately=2C possibly with some serialization around compiling int= > >> erfaces?

Thanks=2C
 =3B- Jay

= > >>
> >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- > >> > >> --===============7278135725256123965== > >> Content-Type: text/plain; charset="us-ascii" > >> MIME-Version: 1.0 > >> Content-Transfer-Encoding: 7bit > >> Content-Disposition: inline > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> --===============7278135725256123965==-- > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:19:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:19:27 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, Message-ID: The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:23:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:23:05 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? Message-ID: I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 07:02:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 05:02:57 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, , , Message-ID: Right -- I would have just said, add a name or such parameter to load/store, but I'm aware of LLVM GetElementPointer so came up with a name like it.But, I'm not sure we want a separate call -- GetElementPointer, followed by a load/store of whatever is stacked -- or build it into the load/store. I've always found the presence of the offset parameter in load/store "surprising".I'd'a'naively'thunk that'd a be a separate add operation.But I also suspect the original authors knew what they were doing and had possiblydone it several times already, and this parameter's presence was an easy lesson they learnedfrom experience. Heck, perhaps they had my current desires in mind? Then again, I was also /initially/ surprised by both the low level nature of the IR, all the layout done already, and that I was able to get it to work generating C. But ultimately it makes a lot of sense in the current form. If you look at the "level" and sophistication of M3x86 -- it doesn't do layout. Even M3C doesn't really do layout -- it echoes/renders the information in another form, leaving the C compiler to do the layout. > something that suits LLVM and C And presumably for the time being, gcc and M3x86? :) My actual extremely long term goal that I might never get to is that once we have something very portable -- C. And I guess something everyone likes -- LLVM.Perhaps try to write more of our backends. Like starting with M3x86.m3.I know I'll never achieve the reach of C/gcc/LLVM though. Merely targeting AMD64 across NT/Linux/Mac will be difficult -- three file formats and at least two ABIs. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:31:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Ideally, we would also converge on something that suits both LLVM and C.Your notion of get_member_reference is similar to LLVM getelementptr, which allows the backend to do layout for structs.It would be very nice if the front-end used this level of abstraction.I assume that is what you are talking about. On Aug 10, 2015, at 2:19 PM, Jay K wrote:The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay_______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 08:55:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 06:55:01 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , Message-ID: Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? > BYTESIZE(INTEGER) I know this is trickier but I consider it almost the same problem. Imagine I have this C code: typedef struct T1 { size_t a,b; } T1;T1* pt1; pt1->b (*size_t*)(((char*)pt1) + sizeof(size_t)) while the second is not idiomatic, and may skirt alignment/padding rules (imagine if a was a char and b a size_t), it is just about as portable. And you can write Modula-3 that way, with LOOPHOLE, ADR, BITESIZE, etc. I think "expressions" or "constant expressions" need to be maintained in "string" or "structural" form and passed to the backend..."string" would be, like, the original Modula-3."structural" would be parse them and make an "expression tree" with nodes like plus, minus, multiplication, division. I think, perhaps, the expression tree would be optional. Backends that ignore them could say so and save that cost. I believe most/all Modula-3 constant expressions render trivially in C. Really, like,RTIO.PutInt(BYTESIZE(INTEGER)); ideally translates to C involving the string/type "INTEGER", and not INT32 or INT64. I have to go. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:37:03 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 2:23 PM, Jay K wrote:I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. A cleaner solution is to use types and request the nth field of the type, etc. That way we don?t need symbolic names for the fields. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? That is a bit trickier if not impossible. Modula-3 inherently exposes these as compile-time integer constants. Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 19:34:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 17:34:34 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , , , Message-ID: > BYTESIZE(INTEGER) I have other ideas here. 1) Give up. Be like the 3.6 release and have target-specific boot archives, and target specific intermediate C, like current. 2) kinda of compile everything twice. The generated C would be: #ifdef _LP64_ // or such ... #else ... almost but not quite the same .. #endif To cut down cost somewhat, rereading dead code, it could be: foo32.c foo64.c foo.c #ifdef _LP64_ #include "foo64.c" #else #include "foo32.c" #endif 3) Sort of compile as always/only 64bit, when using C backend. Specifically, in m3middle/Target.m3, when using C backend, setup 64bit types. And in M3C output something like: #ifdef _LP64_ /* or such -- not quite*/ #define PAD64(x) /* nothing */ #else #define PAD64(x) int x##pad; #endif RECORD T1 = a: REF INTEGER; END; => struct T1 { INTEGER* a; PAD64(a); /* after any pointer field in a record */ }; This is an efficiency loss on 32bit platforms but maybe ok. It would gain a certain compatibility, i.e. in persistance formats. Local variables not in a record..would possibly be put into such a record, depending on if we could guarantee the entire pointer doesn't get overwritten..consider unusual valid Modula-3 such as: Cstring.memset(ADR(ptr), 0, BYTESIZE(ptr)); so we would have to pad out locals also. Possibly subject to #ifdef _LP64_. Big problem here would be interfacing with actual C code. For example Win32 and X Windows. If not for that, I was thinking this is a very viable approach. Given that, I'm not sure. Maybe as I was saying really kind of preserve all constant expressions symbolically and let the C backend rerender them.including stuff like: a: ARRAY [0.. BYTESIZE(INTEGER)] OF CHAR. or a: ARRAY [0.. BYTESIZE(INTEGER) * 2] OF CHAR. or a: ARRAY [0.. (BYTESIZE(INTEGER) = 32) * 4 + (BYTESIZE(INTEGER) = 64) * 8] OF CHAR. would have to be certain basically what is constant in Modula-3 is constant in C. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 17:02:37 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 4:55 PM, Jay K wrote:Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. The names are in the type. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? Yes. I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 11 00:13:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 10 Aug 2015 17:13:00 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: <55C921EC.7070303@lcwb.coop> On 08/09/2015 02:20 PM, Jay K wrote: > Right, agreed -- this should work for LLVM, and the C compiler. > LLVM should be multi-threadable in-proc too. > > Fixing m3front itself -- we'd have to identify and deal with all the globals. > For the alloca/setjmp work I put the "globals" in Module.T. > There are also sequence constraints on order of compilation that the back ends don't have. At a minimum, a unit can't be front-ended (yes, I am told that any word can be verbed, although I try to resist unless it has a really useful conciseness benefit. ;-|) until all its imported and exported interfaces have been front-ended, particularly, their .M3EXPORTS files created. > - Jay > > > > Date: Sun, 9 Aug 2015 14:14:48 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > > > I think this is a great idea. I made the back-end parallel already (at > > > least for the version of the compiler that calls the gcc-based backend). > > > > I believe this works for any executable(s) separate from cm3 itself. > > > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > > you want to spawn in your cm3.cfg . > > > > > > Mika > > > ..... -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Tue Aug 11 06:01:50 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:50 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 06:01:51 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:51 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 07:00:17 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 22:00:17 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: A more fruitful approach might be pipelining the compiler: - files enter the pipeline in reverse dependency order - have a thread (stage) to read those files into memory - have a thread to tokenize and parse the files and form the data structure - have a thread to do intermediary processing - have a thread to generate the output representation for the back end You'll only get a maximum 4x speedup and you'll only be as fast as the slowest thread, but you can reliably tune the divisions based on simple benchmarking of each thread. You're very likely to get somewhat close to the 4x speedup. This works best when there is a 1:1 between pipeline stages and cores. If you were ambitious you could attempt an 8 stage pipeline for 8 core processors. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Aug 11 09:13:25 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 11 Aug 2015 09:13:25 +0200 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Aug 11 12:47:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , Message-ID: I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 13:44:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , , Message-ID: Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 19:20:26 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 10:20:26 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 20:46:35 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 14:46:35 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811184635.GA3909@topoi.pooq.com> On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. I'm even pleased how *fast* the compiler is. If I could make it generate portable C, it would satisy most of the uses I can imagine for it, not just the uses I actually have. -- hendrik From jay.krell at cornell.edu Tue Aug 11 20:56:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 18:56:03 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, Message-ID: The problem with cross compiling is you need: a working cross-compiler for C/C++ a working cross-assembler a working cross-linker If you have those, it is fairly easy. * Getting those is often not easy. Esp. the C/C+ compiler as you need headers/libraries/startup code. "sysroot" or "buildroot" it is often called. * fairly easy: for NT386 -- nothing extra to do, the backend is in cm3 There are endianness bugs in mklib, but it works on the majority of systems -- little endian hosts. I can fix it. For the C backend -- nothing extra to do, the backend is in cm3. For the gcc-based backend -- fairly easy -- you need to rebuild a different cm3cg and point the config at it. It kind of is already setup to work. For the LLVM backend, probably also easy. I think on Debian and Gentoo, cross C compilers at least to every Linux architecture are readily available. But I don't know about otherwise. In as much as the compiler is gcc, the assembler GNU as, and the linker GNU ld, that gets you far, but you still need the headers/libraries/startup code. If your C compiler is LLVM, that might also help for compiler/assembler/linker. But again, the headers/libraries/startup code. There are various systems out there that try to automate this but I don't have much experience with them, and they often are slightly different than what we want. Otherwise, what we have setup is we cross compile to assembly files, copy them to the target, which typically does have the compiler/assembler/linker, and finish there. What remains there is that I only have this automated to build cm3. It should also at least also build m3core and libm3 statically. And therefore reach approximately parity with the 3.6 "boot" archives and install process. Has anyone here installed 3.6? And consider there method a decent goal and stopping point? Today is slightly different since quake has been rewritten from C to Modula-3 in the 4.0 timeframe. And *possibly* boot archives should contain everything -- going beyond what 3.6 did. This is just a matter of work in scripts/python/pylib.py. Copying stuff into sub directories and generating a hierarhical recursive or not make system in there. My make skills aren't great, and I've been hung up on details like depending on GNU make or trying to be more portable. On the threading area, I believe there is a simple almost ideal design. - parse and mostly "execute" all the quake code almost unchanged, single thread - difference starts at about the last line of quake where it says library or program - possibly though queueing every file to a "prefetcher" thread, it just reads every file into a reused buffer and throws out the data, populating the operating system's file system cache; or possibly mmaping every file and keeping it mapped, and touching every page; doing this in a somewhat thread aware/safe way for the upcoming actual accesses - once quake is done, one of two choices: simple but not optimal: do a single threaded parse of every interface less simple: parsing here can be in parallel, depending on the dependency graph; you'd just start up a thread per interface, but block on parsing its dependencies; as you get away from the root (RT0.i3), the tree should get ever wider You would either manually throttle the number of threads, or rely on an underlying threadpool. NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be simplar. Maybe that wasn't good. Simpler as in, including, thread id is now just the underlying Win32 thread id. It wasn't before. Win32 has had a thread pool since circa Windows 2000. That might be profitable to use. - once parsing interfaces is done, I believe codegen of every interface, and parsing and codegen of every module can proceed with arbitrary parallelism Fetching interface/module contents might synchronize with the prefetcher depending on which of the two approaches -- if the prefetcher is just prefetch and discard, populating the file system cache, then later compilation just refetches oblibviously/unchanged. If the prefetcher thread mmaps and keeps everything, then serialize with it. Also, you might have multiple prefetcher threads. What you really want is..difficult. You want a prefetcher thread per spindle. How to count them up? If you have an SSD, then prefetching might not buy much and just forget about it. The thing is, if you have spindles, this might be the most gain, and if you have an SSD, the cost might be negligible or slightly beneficial. I/O is so often the bottleneck, at least in the days of rotating storage. In the event that the file system cache is small, or the source for a package really large, prefetching can be counterproductive -- moving stuff through the cache only to flush it before it is used and fetching it a second time. I expect on most systems this is not a concern though. One of the pieces of work here, I believe, is to move most globals in m3front to be in Module.T. This would actually make things cleaner imho. Yes, it consolidates knowledge that you might want distributed. And accessing context'ed data is slightly slower than globals. But imho globals are bad and everything should be placed in *some* context other than the process or thread. I would not advocate compiling functions within a module in parallel, only modules. Similarly, code generators must have no globals, and put all the state in the cg object. - Jay Date: Tue, 11 Aug 2015 10:20:26 -0700 From: lists at darko.org To: wagner at elegosoft.com CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] multi-threaded m3front? I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 21:04:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 19:04:23 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811184635.GA3909@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: There are clearly three speeds to the compiler in my informal observation. The NT386 backend is fastest. The m3cg backend is medium. The C backed is a little slower. But if this can all be divided by number of cores, and the I/O overlapped,they might all be fast. :) I would like to write more integrated backends, but the C backend needs a bit more work,and we want to remove setjmp/longjmp for most targets still,and the existing M3x86.m3 has a terrible debugging story -- only line numbers. - Jay > Date: Tue, 11 Aug 2015 14:46:35 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > I couldn't agree with you more. I think being able to compile or cross > > compile the system without spending hours (or days) hacking > > scripts/environments would be a huge step forward for the project. Or > > having compiler binaries for more than two platforms (four if you count > > different word sizes). Meanwhile I've never heard anyone complain that the > > compiler is too slow. > > I'm even pleased how *fast* the compiler is. > > If I could make it generate portable C, it would satisy most of the > uses I can imagine for it, not just the uses I actually have. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 21:32:37 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 12:32:37 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: Jay I can see the complexity of the task but can we have an option for users who cannot invest time in this? What would be great is if you could set up the scripts in a static environment available to all that builds various platforms and back ends that are requested or that you feel like maintaining. Having it so it could auto build based on a particular version from github would be optimal. Surely this would help with developing and debugging your scripts and backend and would solve the problem of other people getting the environment right. I have dozens of servers (x86 and ARM) with ample memory and storage plus PDUs that allow powering the servers up and down over the net. If you want to set up the environments and scripts I can setup the access and install Mac OS and whatever Unixes on some of them. You can have carte blanche as to what you do with them, I don't care as long as working compilers pop out somewhere. Maybe others are interested in helping with this too. On Tue, Aug 11, 2015 at 11:56 AM, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ > a working cross-assembler > a working cross-linker > > > If you have those, it is fairly easy. * > Getting those is often not easy. > Esp. the C/C+ compiler as you need headers/libraries/startup code. > "sysroot" or "buildroot" > it is often called. > > > * fairly easy: > for NT386 -- nothing extra to do, the backend is in cm3 > There are endianness bugs in mklib, but it works on the majority of > systems -- little endian hosts. I can fix it. > > For the C backend -- nothing extra to do, the backend is in cm3. > > For the gcc-based backend -- fairly easy -- you need to rebuild a > different cm3cg > and point the config at it. It kind of is already setup to work. > > > For the LLVM backend, probably also easy. > > > I think on Debian and Gentoo, cross C compilers at least to every Linux > architecture are readily available. > But I don't know about otherwise. > > > In as much as the compiler is gcc, the assembler GNU as, and the linker > GNU ld, that gets you far, > but you still need the headers/libraries/startup code. > > > If your C compiler is LLVM, that might also help for > compiler/assembler/linker. > But again, the headers/libraries/startup code. > > > There are various systems out there that try to automate this but I > don't have much experience > with them, and they often are slightly different than what we want. > > > Otherwise, what we have setup is we cross compile to assembly files, copy > them to the target, > which typically does have the compiler/assembler/linker, and finish there. > > > What remains there is that I only have this automated to build cm3. > It should also at least also build m3core and libm3 statically. And > therefore reach > approximately parity with the 3.6 "boot" archives and install process. > > > Has anyone here installed 3.6? And consider there method a decent goal > and stopping point? > Today is slightly different since quake has been rewritten from C to > Modula-3 in the 4.0 timeframe. > > > And *possibly* boot archives should contain everything -- going beyond > what 3.6 did. > > > > This is just a matter of work in scripts/python/pylib.py. > Copying stuff into sub directories and generating a hierarhical recursive > or not make system in there. > My make skills aren't great, and I've been hung up on details like > depending on GNU make or trying to > be more portable. > > > On the threading area, I believe there is a simple almost ideal design. > > - parse and mostly "execute" all the quake code almost unchanged, single > thread > > - difference starts at about the last line of quake where it says library > or program > > - possibly though queueing every file to a "prefetcher" thread, it just > reads > every file into a reused buffer and throws out the data, populating the > operating > system's file system cache; or possibly mmaping every file and keeping > it mapped, > and touching every page; doing this in a somewhat thread aware/safe way > for the > upcoming actual accesses > > - once quake is done, one of two choices: > > simple but not optimal: do a single threaded parse of every interface > less simple: parsing here can be in parallel, depending on the dependency > graph; > you'd just start up a thread per interface, but block on parsing its > dependencies; > as you get away from the root (RT0.i3), the tree should get ever wider > > > You would either manually throttle the number of threads, or rely on an > underlying threadpool. > > > NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be > simplar. > Maybe that wasn't good. Simpler as in, including, thread id is now just > the underlying > Win32 thread id. It wasn't before. > > Win32 has had a thread pool since circa Windows 2000. That might be > profitable to use. > > - once parsing interfaces is done, I believe codegen of every interface, > and parsing and codegen > of every module can proceed with arbitrary parallelism > > > Fetching interface/module contents might synchronize with the prefetcher > depending > on which of the two approaches -- if the prefetcher is just prefetch and > discard, > populating the file system cache, then later compilation just refetches > oblibviously/unchanged. > If the prefetcher thread mmaps and keeps everything, then serialize with > it. > > > Also, you might have multiple prefetcher threads. What you really want > is..difficult. > You want a prefetcher thread per spindle. How to count them up? > If you have an SSD, then prefetching might not buy much and just forget > about it. > The thing is, if you have spindles, this might be the most gain, and if > you have an SSD, > the cost might be negligible or slightly beneficial. > > > I/O is so often the bottleneck, at least in the days of rotating storage. > > > In the event that the file system cache is small, or the source for a > package > really large, prefetching can be counterproductive -- moving stuff > through the cache > only to flush it before it is used and fetching it a second time. > > > I expect on most systems this is not a concern though. > > > One of the pieces of work here, I believe, is to move most globals in > m3front > to be in Module.T. This would actually make things cleaner imho. > Yes, it consolidates knowledge that you might want distributed. > And accessing context'ed data is slightly slower than globals. > But imho globals are bad and everything should be placed in *some* context > other than the process or thread. > > > I would not advocate compiling functions within a module in parallel, > only modules. > > > Similarly, code generators must have no globals, and put all the state in > the cg object. > > > - Jay > > > > > ------------------------------ > Date: Tue, 11 Aug 2015 10:20:26 -0700 > From: lists at darko.org > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] multi-threaded m3front? > > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. > > But of course everyone has different priorities, different interests and > different opinions in a volunteer project so any discussion on this subject > will inevitably boil down to "he who writes the code determines the > priorities." > > > > > On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner > wrote: > > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 22:57:21 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 16:57:21 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811205721.GA9712@topoi.pooq.com> On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ Slightly clumsier is just haveing a C/C++ compiler on the target system. But it's often easier to get that than a cross-compiler. -- hendrik From rodney_bates at lcwb.coop Tue Aug 11 23:01:40 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 11 Aug 2015 16:01:40 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: <55CA62B4.5000205@lcwb.coop> In my informal observation, it is additionally true that: On 08/11/2015 02:04 PM, Jay K wrote: > There are clearly three speeds to the compiler in my informal observation. > > The NT386 backend is fastest. > The m3cg backend is medium. > The C backed is a little slower. > gcc compiling m3cc is quite a bit slower than cm3 with m3cc backend. gcc compiling llvm/clang is a lot slower than that. clang compiling individual C++ units is noticeably slower than gcc on the same units. All of which is leading to my feeling that compiling Modula-3 code is already so much faster than compiling C or C++, that it's a pretty minor need to speed it up even more. > > But if this can all be divided by number of cores, and the I/O overlapped, > they might all be fast. :) > > > I would like to write more integrated backends, but the C backend needs a bit more work, > and we want to remove setjmp/longjmp for most targets still, > and the existing M3x86.m3 has a terrible debugging story -- only line numbers. > > > - Jay > > > > > Date: Tue, 11 Aug 2015 14:46:35 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > > I couldn't agree with you more. I think being able to compile or cross > > > compile the system without spending hours (or days) hacking > > > scripts/environments would be a huge step forward for the project. Or > > > having compiler binaries for more than two platforms (four if you count > > > different word sizes). Meanwhile I've never heard anyone complain that the > > > compiler is too slow. > > > > I'm even pleased how *fast* the compiler is. > > > > If I could make it generate portable C, it would satisy most of the > > uses I can imagine for it, not just the uses I actually have. > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 12 00:33:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811205721.GA9712@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , <20150811205721.GA9712@topoi.pooq.com> Message-ID: Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:37:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:37:10 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , Message-ID: It should be ok now. Have you been using Modula-3 on it? How?I can imagine a preexisting path of low resistance was to use gcc everywhere and reject Apple's tools.gcc as the "assembler driver" would have worked as it'd pick the working assembler instead of the LLVM assembler.As well as using (/usr/bin/)as directly, which is what I do now (it actually calls out to yet another as, but I didn'twant to hardcode that path -- /applications/xcode and all that .. which reminds me I need to uninstall xcodeand see what things do -- as there is an option to install command line tools w/o xcode).. But Apple's tools should work now. - Jay > Subject: Re: [M3devel] Mac 10.10? > From: hosking at purdue.edu > Date: Wed, 12 Aug 2015 07:49:58 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I am and I suspect it'll be a popular target! > > Sent from my iPad > > > On 11 Aug 2015, at 9:44 pm, Jay K wrote: > > > > Anyone else running 10.10? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:44:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:44:19 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , , , , <20150811205721.GA9712@topoi.pooq.com>, Message-ID: I suspect using Debian or Ubuntu is the quickest route to a suite of cross tools. https://wiki.debian.org/CrossToolchains at least to Linux, not necessarily to BSD/NT/Solaris/Mac. We could add "support" to that -- having the default config noticethat host = is debian or ubuntu and host != target and target is linux and invoke those toolsand tell you to apt-get install whatever if they aren't present.. But I still think the "fair" / "flat" / "consistent" story is to make boot archives something like 3.6and user then completes the build on the target with native tools. We'd distribute like three archives per target: boot -- no binaries, all assembly/C source, to just cm3/libm3/m3core (not just cm3 like current) binmin -- binaries -- cm3, cm3cg (except for NT386), m3core (maybe just static), libm3 (ditto) binall -- all binaries, nothing to build The C source might be somewhat target-independent, but getting it to be fully target-independentI can't yet commit to. binmin has a pretty narrow use case, maybe strike it - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: Re: [M3devel] multi-threaded m3front? Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 08:14:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 06:14:02 +0000 Subject: [M3devel] cm3cg coverage/profiling features? Message-ID: Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 12 08:40:40 2015 From: wagner at elego.de (Olaf Wagner) Date: Wed, 12 Aug 2015 08:40:40 +0200 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812084040.665db643b5756d863fb2eb2a@elego.de> On Wed, 12 Aug 2015 06:14:02 +0000 Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. I'm not sure what exactly you're going to remove, but keeping coverage and profiling support would be great. Though it's been years since I used them... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From hendrik at topoi.pooq.com Wed Aug 12 14:09:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 12 Aug 2015 08:09:19 -0400 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812120918.GA578@topoi.pooq.com> On Wed, Aug 12, 2015 at 06:14:02AM +0000, Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. > Thank you, - Jay Didn't know they were there. How to use them? -- hendrik From lists at darko.org Wed Aug 12 17:17:54 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 08:17:54 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: Why does it have to be removed? Is there some pressing reason that justifies removing functionality? How does it improve the compiler? Also, how does asking in the mailing list justify its removal? Not all users follow the mailing list, and future users do not get a say. On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > Does anyone use the coverage or profiling or > optimization-via-profiling-feedback features of cm3cg? > I'm removing dead stuff. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 12 17:21:55 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 12 Aug 2015 08:21:55 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812152155.6D9891A2062@async.async.caltech.edu> Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- From jay.krell at cornell.edu Wed Aug 12 17:33:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 15:33:24 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: , , <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: gcc is so big, I like to trim it down. For example, I removed the C preprocessor, libgcc (adding back a subsetted alternative when using Sun CC, in another place), the driver, most of the frontends, LTO which I assume nobody uses, all the dependency on gmp/mpfr/mpc, lots of "builtins" that we don't expose so you can't use in Modula-3, etc. This is stuff in gcc, not the stuff in cm3/m3front. But arguably we could (have) left it all alone, just apply a minimal patch. Note that anything here is going to be somewhat specific to this backend.Though LLVM may very well have similar functionality.The integrated backend does not -- though for coverage there exists post-link instrumentation options at least on Windows in Visual Studio.The C backend could get it via the underlying C compiler, e.g. gcc. Someone should try them out and report.The config files have long offered profile/-pg.I'll leave it for now. Thanks, - Jay > To: lists at darko.org > Date: Wed, 12 Aug 2015 08:21:55 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing rea= > >son that justifies removing functionality? How does it improve the compiler= > >?

Also, how does asking in the mailing list justify its = > >removal? Not all users follow the mailing list, and future users do not get= > > a say.

>">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= > >t; wrote:
> .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= > >uff.

Thank you,
=C2=A0- Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > >=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 18:11:10 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 09:11:10 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 19:41:26 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 17:41:26 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, Message-ID: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 20:21:42 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 11:21:42 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top; TYPE T <: ROOT; END Top. INTERFACE Pub; IMPORT Top; TYPE TPub = OBJECT [...] END; REVEAL Top.T <: TPub; END Pub. INTERFACE Pvt; IMPORT Top, Pub; TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt; END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 20:56:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 18:56:12 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , , Message-ID: > Error recovery is hard but the original authors laboured to implement it and it worked Within a module it is unbounded problem and such labor will only slightly work. That said, I know of no changes within modules or across modules wrt error recovery.We can/should investigate and fix. I don't think a compiler switch is a good idea here, as we should have a smaller matrixof behaviors and the behavior you desire is probably the one and only way it should be. Making switches for too many things is its own madness. Nobody runs the same thingand the test matrix explodes. Across modules we should probably just proceed, just being sure to error at the end (i.e. so we don't ship, and maybe so scripts don't proceed, thoughactually, scripts probably should proceed, but again error at the end -- builds tend to contain highly independent graph pieces, even if thereare some heavily shared nodes at the root like m3core) There are at least two schools of thought in software in general. There is "fail fast" and "muddle along". Generally older thinking is "muddle along", make a best effort attempt to work in the face of errors. Newer thinking is "fail fast", and generally acknowledging that once there is an error, some?many?most?all "bets"are off and best to stop, vs. doing the wrong thing. But there is a middle ground, which is to try to think about dependencies and if errors in one placereally affect another. Often times, for a simple example, iterations of a loop are independent and failurewithin one shouldn't stop the loop. Imagine you specify to copy many files, but a few fail. Should you stop the copy at the first failure?Should you rollback the earlier successful copies? Should you have some level of "transactionality" so that no intermediate state is visible unlessthe entire operation is guanteed to succeed? Anyway, compilers, even within a module, seem popular to place in the middle and muddle along.Their output is eventuall mission critical, but how they handle errors is not.People like to get as many errors as possible, fix them all, try again.Instead of the compiler stopping at the first. You know, to try to divide the quadratic natureof fixing errors down to almost linear. But it depends on the nature of the error. "breaking" a declaration is presumably removing it or changing it to a different valid declaration.Changing it to a declaration that doesn't parse is risky -- you have to assume error recovery works a certain way. I also discovered an incrementality problem -- if you move a module across packages.That is probably not a new regression though. I don't think we are losing functionality and all changes still come through m3commit. Imho review can occur before or after change, but I know, not everyone in the world agrees.Many development environments require review before commit. There is the matter of if rate of change exceeds rate of review and needs to be throttled down,but rate of change is really quite low (even if rate of review is even lower). - Jay Date: Wed, 12 Aug 2015 11:21:42 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top;TYPE T <: ROOT;END Top. INTERFACE Pub;IMPORT Top;TYPE TPub = OBJECT [...] END;REVEAL Top.T <: TPub;END Pub. INTERFACE Pvt;IMPORT Top, Pub;TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt;END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *)REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END;BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 21:41:43 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 12:41:43 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The error recovery doesn't matter. For instance I just rename a declaration so it no longer exists. The compiler will then produce an error for each usage of that symbol, saying it's no longer found, and it will report it at least once for every module where it used (I've noticed that the compiler will often only report one error if it is repeated multiple times in a module). I'm not sure the rate of change is that low. I think the best idea would be put changes in branches so each set can be reviewed and tested before being merged. This would also allow people to choose what they want in their build. With the new GitHub repository this should be a lot easier than with CVS. On Wed, Aug 12, 2015 at 11:56 AM, Jay K wrote: > > Error recovery is hard but the original authors laboured to implement > it and it worked > > > Within a module it is unbounded problem and such labor will only slightly > work. > > > That said, I know of no changes within modules or across modules wrt error > recovery. > We can/should investigate and fix. > > I don't think a compiler switch is a good idea here, as we should have a > smaller matrix > of behaviors and the behavior you desire is probably the one and only way > it should be. > > Making switches for too many things is its own madness. Nobody runs the > same thing > and the test matrix explodes. > > > Across modules we should probably just proceed, just being sure to error > at the end (i.e. so we don't ship, and maybe so scripts don't proceed, > though > actually, scripts probably should proceed, but again error at the end -- > builds tend to contain *highly* *independent* graph pieces, even if there > are some heavily shared nodes at the root like m3core) > > > There are at least two schools of thought in software in general. > > There is "fail fast" and "muddle along". > > > Generally older thinking is "muddle along", make a best effort attempt to > work in the face of errors. > > > Newer thinking is "fail fast", and generally acknowledging that once there > is an error, some?many?most?all "bets" > are off and best to stop, vs. doing the wrong thing. > > > But there is a middle ground, which is to try to think about dependencies > and if errors in one place > really affect another. Often times, for a simple example, iterations of a > loop are independent and failure > within one shouldn't stop the loop. > > > Imagine you specify to copy many files, but a few fail. Should you stop > the copy at the first failure? > Should you rollback the earlier successful copies? > > Should you have some level of "transactionality" so that no intermediate > state is visible unless > the entire operation is guanteed to succeed? > > > Anyway, compilers, even within a module, seem popular to place in the > middle and muddle along. > Their output is eventuall mission critical, but how they handle errors is > not. > People like to get as many errors as possible, fix them all, try again. > Instead of the compiler stopping at the first. You know, to try to divide > the quadratic nature > of fixing errors down to almost linear. But it depends on the nature of > the error. > > > "breaking" a declaration is presumably removing it or changing it to a > different valid declaration. > Changing it to a declaration that doesn't parse is risky -- you have to > assume error recovery works a certain way. > > > I also discovered an incrementality problem -- if you move a module across > packages. > That is probably not a new regression though. > > > I don't think we are losing functionality and all changes still come > through m3commit. > > Imho review can occur before or after change, but I know, not everyone in > the world agrees. > Many development environments require review before commit. > > > There is the matter of if rate of change exceeds rate of review and needs > to be throttled down, > but rate of change is *really quite low *(even if rate of review is even > lower). > > > - Jay > > > ------------------------------ > Date: Wed, 12 Aug 2015 11:21:42 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > The larger point is that is that changes are being made to the compiler > ad-hoc. There seems to be no consideration of the consequence of those > changes and how it affects users. These changes are being made on the trunk > instead of a branch and it may be hard to untangle them. If we're losing > functionality accidently then something is wrong with the development > process. > > On this issue, ignoring cascading errors is perfectly legitimate but so is > not ignoring them. I (used to) purposely break a declaration so see the > consequences of it throughout the code and to reveal where the declaration > was used. It's also very slow having to recompile after fixing a change in > each client module to reveal the next place it needs fixing, just because > it's in a different file. > > Error recovery is hard but the original authors laboured to implement it > and it worked, it shouldn't be lost. At the very least a compiler switch is > in order here. > > Either way this seems to be a bug because the compiler is also failing to > track dependencies in some causes causing files not to be recompiled. For > example it seems to be happening somewhere around using this sort of idiom: > > INTERFACE Top; > TYPE T <: ROOT; > END Top. > > INTERFACE Pub; > IMPORT Top; > TYPE TPub = OBJECT [...] END; > REVEAL Top.T <: TPub; > END Pub. > > INTERFACE Pvt; > IMPORT Top, Pub; > TYPE TPvt = Pub.TPub OBJECT [...] END; > REVEAL Top.T <: TPvt; > END Pvt. > > MODULE Top; > IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) > REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; > BEGIN > END Top. > > I suspect the problem has something to do with the "injection" of the > partial revelations through importing interfaces. > > On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:27:36 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:27:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: > I wasn?t aware this had changed. > When was this regression introduced? > > On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:29:03 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:29:03 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > I'm using: > > Critical Mass Modula-3 version d5.9.0 > > last updated: 2010-07-21 > > compiled: 2014-08-27 19:15:04 > > configuration: /usr/local/cm3/bin/cm3.cfg > > host: AMD64_DARWIN > > target: AMD64_DARWIN > > > > On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: > >> I wasn?t aware this had changed. >> When was this regression introduced? >> >> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >> >> For instance, cm3 used to keep on compiling files after errors were >> found. Now it seems to stop after one module. That loss of functionality >> seriously reduces productivity. I couldn't find any switch to reverse the >> change. When was this change decided? Is there a way to restore it? >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 13 03:00:22 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do. Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: > It's the one from the Releases section on GitHub. > > On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >> wrote: >> >>> I wasn?t aware this had changed. >>> When was this regression introduced? >>> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >>> >>> >>> >> > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 04:05:17 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 19:05:17 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Sorry, I was distracted by a phone call before - that is the compiler that exhibits the behaviour. Previous to that I was using something that was compiled in 2010, so not very helpful for pinpointing. On 8/12/15, Antony Hosking wrote: > Are you saying that the behavior was still working as of d5.9.0? > Or is that the version you are currently using in which it no longer does > what you want? > We should be able to figure out what commit broke things if we can pinpoint > a time when it worked. > >> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >> >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > > wrote: >> I wasn?t aware this had changed. >> When was this regression introduced? >> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >> > wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >> >> > > From jay.krell at cornell.edu Thu Aug 13 05:43:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Aug 2015 20:43:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> Message-ID: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? > >> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >> >> Sorry, I was distracted by a phone call before - that is the compiler >> that exhibits the behaviour. Previous to that I was using something >> that was compiled in 2010, so not very helpful for pinpointing. >> >> On 8/12/15, Antony Hosking wrote: >>> Are you saying that the behavior was still working as of d5.9.0? >>> Or is that the version you are currently using in which it no longer does >>> what you want? >>> We should be able to figure out what commit broke things if we can pinpoint >>> a time when it worked. >>> >>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>> >>>> I'm using: >>>> >>>> Critical Mass Modula-3 version d5.9.0 >>>> >>>> last updated: 2010-07-21 >>>> >>>> compiled: 2014-08-27 19:15:04 >>>> >>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>> >>>> host: AMD64_DARWIN >>>> >>>> target: AMD64_DARWIN >>>> >>>> >>>> >>>> >>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>> > wrote: >>>> I wasn?t aware this had changed. >>>> When was this regression introduced? >>>> >>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>> > wrote: >>>>> >>>>> For instance, cm3 used to keep on compiling files after errors were >>>>> found. Now it seems to stop after one module. That loss of functionality >>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>> change. When was this change decided? Is there a way to restore it? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 10:50:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 08:50:57 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , Message-ID: Note that this probably isn't the coverage/profiling I was talking about.I was talking about specifically in the gcc backend. I wonder if coverage -- a boolean per basic block; or maybe a counter,is cheap enough that we can't have it always enabled. And then some way of exposing it. I don't like having to recompile/relink everything multiple differentways to get all the features. - Jay From: peter.mckinna at gmail.com Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: Re: [M3devel] cm3cg coverage/profiling features? To: lists at darko.org CC: hosking at purdue.edu; m3devel at elegosoft.com; jay.krell at cornell.edu Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do.Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: I wasn?t aware this had changed.When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 11:23:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 09:23:46 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , ,,, , , , , , Message-ID: Eh, seems like some off by one in my testing -- things work now, including X apps. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: Re: [M3devel] Mac 10.10? Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 19:34:26 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Aug 2015 10:34:26 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> Message-ID: Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: > "Yes". > > I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. > > > I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) > > > - Jay > > On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > >> Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? >> >>> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >>> >>> Sorry, I was distracted by a phone call before - that is the compiler >>> that exhibits the behaviour. Previous to that I was using something >>> that was compiled in 2010, so not very helpful for pinpointing. >>> >>> On 8/12/15, Antony Hosking wrote: >>>> Are you saying that the behavior was still working as of d5.9.0? >>>> Or is that the version you are currently using in which it no longer does >>>> what you want? >>>> We should be able to figure out what commit broke things if we can pinpoint >>>> a time when it worked. >>>> >>>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>>> >>>>> I'm using: >>>>> >>>>> Critical Mass Modula-3 version d5.9.0 >>>>> >>>>> last updated: 2010-07-21 >>>>> >>>>> compiled: 2014-08-27 19:15:04 >>>>> >>>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>>> >>>>> host: AMD64_DARWIN >>>>> >>>>> target: AMD64_DARWIN >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>>> > wrote: >>>>> I wasn?t aware this had changed. >>>>> When was this regression introduced? >>>>> >>>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>>> > wrote: >>>>>> >>>>>> For instance, cm3 used to keep on compiling files after errors were >>>>>> found. Now it seems to stop after one module. That loss of functionality >>>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>>> change. When was this change decided? Is there a way to restore it? >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 20:47:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 18:47:13 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , , <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu>, <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com>, Message-ID: (Note: This was meant to report the unit path but I failed to commit that. This is good enough for today. Later...) From: jay.krell at cornell.edu CC: hosking at purdue.edu; lists at darko.org; jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? Date: Thu, 13 Aug 2015 10:34:26 -0700 To: jay.krell at cornell.edu Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote:Sorry, I was distracted by a phone call before - that is the compilerthat exhibits the behaviour. Previous to that I was using somethingthat was compiled in 2010, so not very helpful for pinpointing.On 8/12/15, Antony Hosking wrote:Are you saying that the behavior was still working as of d5.9.0? Or is that the version you are currently using in which it no longer does what you want? We should be able to figure out what commit broke things if we can pinpoint a time when it worked. On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: I wasn?t aware this had changed. When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric > wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 22:27:54 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 13:27:54 -0700 Subject: [M3devel] Build Server - Plan Message-ID: I'm setting up a server for building CM3 that takes a "minimalist" approach. It's a machine running several virtual machines, one for each platform supported by CM3. Each VM will contain clean install of the OS plus any external tool dependencies. It will have a minimal compiler install, basically enough to compile itself for the host target. I'm going create one VM for each target I have a bootstrap compiler for, so if you think you'll find this useful and you want a target supported PLEASE CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. Users can request any version of the compiler from the github repository and all new commits will be automatically built for all platforms. The publicly available build products will be: - minimal executables for bootstrap, eg the frontend and a backend - model compiler config files - compilation logs for bootstrap executables - compilation logs for most modules in the github repository - logs for certain tests Packages, libraries, scripts and non-essential tools or executables will not be built or used, the idea being that people take the minimal bootstraps and build from there. I'll post the URL when it's up and running and any suggestions are welcome. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 13 22:31:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 13 Aug 2015 16:31:29 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150813203129.GA20453@topoi.pooq.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. > > I'm going create one VM for each target I have a bootstrap compiler for, so > if you think you'll find this useful and you want a target supported PLEASE > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > Users can request any version of the compiler from the github repository > and all new commits will be automatically built for all platforms. > > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests > > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. > > I'll post the URL when it's up and running and any suggestions are welcome. Will the C backend be one of the platforms (or several, if they're not yet all compatible?) -- hendrik From jay.krell at cornell.edu Fri Aug 14 00:26:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 22:26:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: , <20150813203129.GA20453@topoi.pooq.com> Message-ID: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 02:14:16 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 17:14:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: I want to try to support every sort of backend I can configure. That's the focus, frontend and backend executables. In particular the idea is to be able to test backends against each other for correctness with a particular front end (and the other way around). On Thu, Aug 13, 2015 at 1:31 PM, Hendrik Boom wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, > so > > if you think you'll find this useful and you want a target supported > PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first > up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 04:11:08 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 19:11:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > > minimal executables for bootstrap, eg the frontend and a backend > > > While I have fiddled with this being just cm3, I currently propose it is > the "min" output of make-dist.py (which does output an "all" variant as > well, > it'll take just a bit of change to make that optional). > > > Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), > libm3 (static suffices). > > > make-dist.py also already outputs make-dist.py.log or such next to > itself I believe. > See "Tee" in pylib.py. > > > cm3 is a valid minimal bootstrap, and I have used it a number of times > all on its own, > but only IF you also have a matching source tree -- > m3cc/config/libm3/m3core. > It doesn't work if it is some "random old working" cm3 and not the rest. > > > "min" can be slightly more min I believe -- it has all the .m3 files for > m3core/libm3, > for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > > > > > Will the C backend be one of the platforms (or several, > > if they're not yet all compatible?) > > > Unfortunately, the C backend output is not compatible with the gcc > backend output. > They vary in how they pass the "display" to nested procedures (oh, if > only we didn't > have nested procedures...) > So we might want to do something about this. > I have considered appending "C" to BUILD_DIR, and have done so with mixed > success. > Alternatively, do everything twice. > > > Also, if isn't clear, the C backend output is still very > target-specific. > I'd like to fix that, but I don't have a full plan in mind and might > give up. > > > Note that the minimal bootstrap can be used no matter which backend > you are next going to use, i.e. any bootstrap can feed into C backend, > gcc backend, LLVM backend. > > > Can you really do Mac? > And if so, multiple versions? > > There is unfortunate combinatorial explosion beyond all this. > Specifically, you could n different C compiler per platform. > gcc x, gcc y, clang x, clang y, etc. > > > > logs for certain tests > > > We have been using m3-sys/m3tests as a start. Just a start. > > > Thank you, > - Jay > > > > > Date: Thu, 13 Aug 2015 16:31:29 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > > > It's a machine running several virtual machines, one for each platform > > > supported by CM3. Each VM will contain clean install of the OS plus any > > > external tool dependencies. It will have a minimal compiler install, > > > basically enough to compile itself for the host target. > > > > > > I'm going create one VM for each target I have a bootstrap compiler > for, so > > > if you think you'll find this useful and you want a target supported > PLEASE > > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be > first up. > > > > > > Users can request any version of the compiler from the github > repository > > > and all new commits will be automatically built for all platforms. > > > > > > The publicly available build products will be: > > > > > > - minimal executables for bootstrap, eg the frontend and a backend > > > - model compiler config files > > > - compilation logs for bootstrap executables > > > - compilation logs for most modules in the github repository > > > - logs for certain tests > > > > > > Packages, libraries, scripts and non-essential tools or executables > will > > > not be built or used, the idea being that people take the minimal > > > bootstraps and build from there. > > > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > > > Will the C backend be one of the platforms (or several, if they're not > > yet all compatible?) > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 08:36:29 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 06:36:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150814063629.GA3086@zoho.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. From jay.krell at cornell.edu Fri Aug 14 09:02:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 07:02:51 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: , <20150814063629.GA3086@zoho.com> Message-ID: It is very easy to install. "We" run make-dist.py and get out a "min" install and an "all" install (per platform).You pick one, extract it, set $PATH optionally, and you are done -- run cm3 either via $PATH or by full path. There is some chance our binaries built on a "new" system won't run on an "old" system.We can mitigate that by deferring some of the build to install time.I hope to have a better story here, a more "sourcey" distribution, that is easy to install.But the binary distributions usually work well. Sometimes you have to edit the config files, but I have endeavored to make that not necessary. There is another install path that prompts for path to compiler, linker, etc., even tries to figure it out. But I don't use that. I think it isn't needed. Yes we support significantly more than x86/amd64. We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw machines -- very recently even.In the 5.8.6 timeframe, I was setup to build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built tonight -- the gcc-based backend. Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Posix or NT for I/O, networking, threads, etc., and X windows or Win32.That is, while we don't run in a browser, we can run almost anywhere else, generally because other people have ported their underlying systems everywhere. (Run in a browser though: LLVM w/ emcriptem backend) The system was already very portable. I have made it easier to port. I assume the "difficult" goals are meant to be ensure that people don't have to jump through the hoops of "install lots of extra stuff" to get it to work. Most people will overshoot the minimal install requirements, but they will overshoot differently.I - Jay > Date: Fri, 14 Aug 2015 06:36:29 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 16:49:56 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 14:49:56 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150814144956.GA4391@zoho.com> Hi, please see comments within: On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > It is very easy to install. Ok, from what I read here I misunderstood. But I am glad to hear it. > Yes we support significantly more than x86/amd64. Thanks for the explanation. I am following projects that run on non-Intel platforms and I get a little concerned when there is movement to only run on Intel or when there seems to be the position Intel is all there is. While I concede Intel is overwhelmingly what people have there is also some very nice and increasingly cheap really good hardware based on other, better architectures and it's a shame to see that ignored into obvlivion. I was happy to see the broad range of hardware and OS when I looked at the CM3 download page so when somebody noted he was going to tackle the Big 3 I was concerned because many projects and OS are now pretty much Intel only which I think is both unfortunate and also technically limiting in terms of code quality. > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > tonight -- the gcc-based backend. I remember we talked about the confusion of the various flavors of SPARC listed in the downloads section. If people are in a consolidating mood it might be good to just use the Solaris Studio version instead of gcc on that platform. > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Well, LLVM is a significant inhibitor to running anywhere but Intel as far as I know. LLVM SPARC does not seem to even be in the alpha stage and doesn't look like it's on anybody's list of things to do. LLVM MIPS is also not working well from the looks of things on the Debian MIPS list. Other projects that have adopted LLVM seem to use it only on Intel and have stayed with gcc on other platforms. There is a concern that will eventually be too difficult to maintain and the natural solution will be to abandon non-Intel hardware. I understand and agree with the value of LLVM from the standpoint of reducing code complexity over gcc and also the more liberal license but it's disappointing to see only the Intel support is probably ever going to be worth anything as far as LLVM goes. > The system was already very portable. I have made it easier to port. > > I assume the "difficult" goals are meant to be ensure that people don't > have to jump through the hoops of "install lots of extra stuff" to get it > to work. I really don't know but I would agree with that. The install should go in standard paths and not require any prereqs, if possible. Thank you. From lists at darko.org Fri Aug 14 18:22:08 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:22:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 18:30:16 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:30:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 19:18:10 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 10:18:10 -0700 Subject: [M3devel] Build Server - Access and Architecture Message-ID: Anyone who wants to setup their own build VMs is welcome to do so and SSH access will be provided. The VMs are controlled from the host command line and you can jump into the command line of any VM. The host will have a local, full, and up to date copy of the github repository. There will be VMs with pristine installs of each of the supported OSes available on the host. These will include MacOS, Windows and various Unixes. To make a build VM you copy an OS VM and setup the build environment by installing the compiler and any required tools. This becomes the baseline for any actual compiling/building. The VM must also have a "build" script that performs the work. This script must pull the requested source version from the local git repository before building. The host build driver performs the following steps for each build VM: - it starts the build VM and calls the build script - when the build script has completed it copies the products (results) from the build VM onto the host where they are consequently uploaded to the publically accessible download server. - the host then rolls back/restores the build VM to its original state and stops it. This means there can be no contamination between builds and you always start building from a known, working state. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 20:15:37 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 11:15:37 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: > The server will be running MacOS 10.10 as the host. The VMs can handle any > MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need > for anything else? > > If you mean can we have multiple VMs for deploying different compiler > configs, then the answer is yes. Each VM will be suspended until it's > needed for a build. Once it's built the requested products and they are > uploaded to the download server, it shuts down. So there aren't any real > performance or memory limitations, all disks are SSDs. > > > On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > >> > minimal executables for bootstrap, eg the frontend and a backend >> >> >> While I have fiddled with this being just cm3, I currently propose it is >> the "min" output of make-dist.py (which does output an "all" variant as >> well, >> it'll take just a bit of change to make that optional). >> >> >> Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), >> libm3 (static suffices). >> >> >> make-dist.py also already outputs make-dist.py.log or such next to >> itself I believe. >> See "Tee" in pylib.py. >> >> >> cm3 is a valid minimal bootstrap, and I have used it a number of times >> all on its own, >> but only IF you also have a matching source tree -- >> m3cc/config/libm3/m3core. >> It doesn't work if it is some "random old working" cm3 and not the rest. >> >> >> "min" can be slightly more min I believe -- it has all the .m3 files >> for m3core/libm3, >> for debugging. I'm not sure if the .i3 files are needed, or the >> .ig/.mg. >> >> >> >> > Will the C backend be one of the platforms (or several, >> > if they're not yet all compatible?) >> >> >> Unfortunately, the C backend output is not compatible with the gcc >> backend output. >> They vary in how they pass the "display" to nested procedures (oh, if >> only we didn't >> have nested procedures...) >> So we might want to do something about this. >> I have considered appending "C" to BUILD_DIR, and have done so with >> mixed success. >> Alternatively, do everything twice. >> >> >> Also, if isn't clear, the C backend output is still very >> target-specific. >> I'd like to fix that, but I don't have a full plan in mind and might >> give up. >> >> >> Note that the minimal bootstrap can be used no matter which backend >> you are next going to use, i.e. any bootstrap can feed into C backend, >> gcc backend, LLVM backend. >> >> >> Can you really do Mac? >> And if so, multiple versions? >> >> There is unfortunate combinatorial explosion beyond all this. >> Specifically, you could n different C compiler per platform. >> gcc x, gcc y, clang x, clang y, etc. >> >> >> > logs for certain tests >> >> >> We have been using m3-sys/m3tests as a start. Just a start. >> >> >> Thank you, >> - Jay >> >> >> >> > Date: Thu, 13 Aug 2015 16:31:29 -0400 >> > From: hendrik at topoi.pooq.com >> > To: m3devel at elegosoft.com >> > Subject: Re: [M3devel] Build Server - Plan >> >> > >> > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: >> > > I'm setting up a server for building CM3 that takes a "minimalist" >> approach. >> > > >> > > It's a machine running several virtual machines, one for each platform >> > > supported by CM3. Each VM will contain clean install of the OS plus >> any >> > > external tool dependencies. It will have a minimal compiler install, >> > > basically enough to compile itself for the host target. >> > > >> > > I'm going create one VM for each target I have a bootstrap compiler >> for, so >> > > if you think you'll find this useful and you want a target supported >> PLEASE >> > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be >> first up. >> > > >> > > Users can request any version of the compiler from the github >> repository >> > > and all new commits will be automatically built for all platforms. >> > > >> > > The publicly available build products will be: >> > > >> > > - minimal executables for bootstrap, eg the frontend and a backend >> > > - model compiler config files >> > > - compilation logs for bootstrap executables >> > > - compilation logs for most modules in the github repository >> > > - logs for certain tests >> > > >> > > Packages, libraries, scripts and non-essential tools or executables >> will >> > > not be built or used, the idea being that people take the minimal >> > > bootstraps and build from there. >> > > >> > > I'll post the URL when it's up and running and any suggestions are >> welcome. >> > >> > Will the C backend be one of the platforms (or several, if they're not >> > yet all compatible?) >> > >> > -- hendrik >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 20:46:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 18:46:46 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150813203129.GA20453@topoi.pooq.com>, , , Message-ID: I have tried PPC_DARWIN on 10.5. It doesn't *quite* work. Almost. I don't believe Rosetta has adequate compatibility, i.e in suspending threadsand getting their context. When we switch to cooperative suspend that will probably work.And then we can remove a bunch more target-specific code. Generally you don't need to support Rosetta. It just works.But I don't believe Rosetta quite fully works. PPC_DARWIN definitely had a long somewhat recent run of workingon native hardware and probably still does.(Fyi, I386_DARWIN also worked before MacOSX/Intel was public.) - Jay Date: Fri, 14 Aug 2015 11:15:37 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:07:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:07:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: ARM_DARWIN I never quite finished.It was almost working on a jail-broken iPhone 3gs.Setting up and testing just a C development environmentis often the difficult part. If you do have a C cross compiler yes that can shift slightly,easily where some things happen. Let me explain.The compilation process has a few different steps it can go.Including that we have some C code.So that goes .c => .o, or internally .c => .s => .o. Ignoring LLVM...We have the NT386 backend.Code goes right from .m3 => .obj.cm3 does "everything". We have the gcc backend.Code goes.m3 => .mc => .ms => .o .ms files are just assembly source.mc files are an internal format that cm3 writes out. We have the C backend.Code goes.m3 => .c => .s => .ocm3 writes the .c files. And then there is linking. This is all *hightly* automated.From the users point of view, cm3 seems to go from .m3 files to a .a/.dylib/executable. But cm3 only has within it some of the stages.It doesn't have a C compiler, an assembler, or a linker.We depend on a "C" toolchain for this (I'm calling assembler/linker part of the C toolchain). If you don't have the C toolchain, then you are stuck part way through those pipelines.Either with a bunch of .o files you can't link.Or a bunch of .ms files you can't assemble and link.Or a bunch of .c files you can't compile/assemble/link. Native toolchains are historically more common than cross tool chains.So we have a "boot" workflow -- that needs work -- where on the "first" machine,you stop where I say you are "stuck", you archive up the files, copy themto the target machine, which has a native toolchain, and then resume the pipeline --running C compiler/assembler/linker. Now, if you have a cross toolchain there is not quite reason for that. But, it comes down mainly to the config files and the builder, and another detail I'll get to. The "boot" workflow was well automated 20 years ago.Back when "quake" was written in C. The 3.6 release.I don't remember what the 4.0/4.1 release looked like. I have good automated flow for producing a cm3 for cross scenarios,and it is useful, but it requires a matching source tree on the other side. This is boot1.py and boot2.sh. If there is an incompatibility, like I recently added to m3core, then it doesn't work.So it dawns on me now, that boot1.py should produce m3core/libm3, and maybe cm3cg.Though cm3cg would require a cross compiler, so sometimes a pain. "another detail I'll get to".We have a small number of tools that when we build the system, we buildthem and run time. Other than cm3. There is mklib, which I already handle.But there is also m3bundle and PklFonts and some RPC stub generators. Given a cross C toolset, we don't have conventions/automation around,hey, I'm targeting x, building on y, my y-hosted tools might need an update.I need to build all of y, then all of x, and when I'm building x, what is the cm3I use? What is the C compiler I use? Again it is mostly just a thing with config files.The config files tend to run "gcc" and find it in $PATH.They assume it is for the current host/target.But there needs to be arguably some convention, I'm hosting on I386_LINUX,targeting AMD64_DARWIN -- where are the native tools and where are the cross tools?In gcc-land, this is basically you run ad64-apple-darwin-gcc for the AMD64 target toolset,instead of just "gcc" You might also run e.g. /usrl/ocal/cm3/bin//cm3cg. So, very long story short -- we largely need a file system layout. The "GNU platform" mapping I386_DARWIN => i386-apple-darwin8 for example,is somewhat encoded in our tree, but for this to work it needs more formalization. With LLVM, you get all the backends together in on tree and you givethe executable a command line switch and it finds the right .so, I imagine,so this sort of gets easier. We still have to know the switches and work that into config and such.We should just do that for native targets anyway and then cross will reuse that. Again, sorry, long story short -- paths and switches to compiler/assembler/linker is largely the thing. Apple has a good system here btw.You say gcc -arch ppc.And that combines with -x assembler.So you can cross compile with gcc -arch ppc.You can cross assemble with gcc -arch -x assembler.They put the work in the gcc driver. It then farms out to the correct cc1/cc1plus/as. If only this was the norm.. Another thing -- a cross C compiler that can handle something with no #includes andno linking, is really easy to build. It is the headers and libraries that make it more difficult.The assembler/linker are also sometimes a sticking point, because sometimes, if youread the gcc config/install manual, they prefer the native closed source assembler/linker. I have a working VMS C cross compiler from Mac, but I had to copy various filesfrom a running VMS machine to do it -- the headers and libraries. In as much as the assembler/linker is GNU, it is easier.And Apple's stuff is open source and people have ported it.But imagine targeting HP-UX or AIX, or NT using the Microsoft toolset.There you can't just conjure cross tools with arbitrary host, only what the vendor ships. - Jay Date: Fri, 14 Aug 2015 09:30:16 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:11:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:11:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:24:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:24:54 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814144956.GA4391@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com> Message-ID: > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. I already mostly solved this years ago. SOLgnu and SOLsun were always already the same, e.g. same threading,same jmpbuf, except the config files picked a different C compiler and differentoutput directory name. I decided it shouldn't be separate paths everywhere in cm3/config files,only which C compiler to run and output directory names.That is, only the config files should vary. We have a somewhat separate notion of "TARGET" and "BUILD_DIR".We might need to better about that.For example profile build get "p" appended to BUILD_DIR, but still the same TARGET.We need BUILD_DIR=SOLgnu but TARGET=SPARC32_SOLARIS. The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun compiler.I invented a separate variable with better granularity. Just look at SPARC32_SOLARIS: readonly TARGET = "SPARC32_SOLARIS"readonly C_COMPILER = "SUN" % or GNU (SOLgnu configuration)include("SPARC32_SOLARIS.common") and then SPARC32_SOLARIS.common looks for the compiler you ask for,and makes the switches match also. Rodney gave me the term for this elegance the other day -- "Cartesian factoring". Or, only factor out what varies. Where things are the same, don't duplicate. In the old days the config files were very monolithic and there was duplication all over the place.For example, in the old days, HPPA implies HP-UX, but now HPPA can be Linux, NetBSD, FreeBSD, etc.The HPPA part is one file, the Linux/NetBSD/FreeBSD parts are their own single files.We only have a combinatorial explosion of small config files, not large ones.Even this could be improved -- runtime composition? The old days had somewhat less sharing. i.e. we probably used "vendor" cc/as/ld more and GNUless. But there is still good allowance for that now. I similarly refactor I386_NT.It was called NT386.The "mingw" platform is largely the same -- Win32 threads, Win32 I/O, Win32 GUI, GNU compiler/assembler/linker.There I came up with several variables for the factoring: See NT.commonTHREAD_LIBRARY = MS or PTHREADSWINDOW_LIBRARY = MS or XLINKER = MS or GNUC_COMPILER = MS or GNU (should add "clang" or "llvm" as an option here) LINUXLIBC6 is now I386_LINUX. NT386GNU is I386_CYGWIN or I386_MINGW A problem I ran into though wrt SOLgnu/SOLsun/LINUXLIBC6/NT386 is that nobodyusing them wanted to switch the name. So I fixed n systems by renaming to m, leaving us with n + m, but they are justlayered over very common stuff. It doesn't take n + m work to implement.But the system is cluttered. Has anyone switched from LINUXLIBC6 to I386_LINUX? Can we delete LINUXLIBC6?Has anyone/everyone switched from SOLsun/SOLgnu to SPARC32_SOLARIS?FreeBSD4 to I386_FREEBSD? Well, at least all the AMD64 and DARWIN targets are well named.The others maybe aren't used much either way.. - Jay > Date: Fri, 14 Aug 2015 14:49:56 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > Hi, please see comments within: > > On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > > > It is very easy to install. > > Ok, from what I read here I misunderstood. But I am glad to hear it. > > > Yes we support significantly more than x86/amd64. > > Thanks for the explanation. I am following projects that run on non-Intel > platforms and I get a little concerned when there is movement to only run on > Intel or when there seems to be the position Intel is all there is. While I > concede Intel is overwhelmingly what people have there is also some very > nice and increasingly cheap really good hardware based on other, better > architectures and it's a shame to see that ignored into obvlivion. > > I was happy to see the broad range of hardware and OS when I looked at the > CM3 download page so when somebody noted he was going to tackle the Big 3 I > was concerned because many projects and OS are now pretty much Intel only > which I think is both unfortunate and also technically limiting in terms of > code quality. > > > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > > tonight -- the gcc-based backend. > > I remember we talked about the confusion of the various flavors of SPARC > listed in the downloads section. If people are in a consolidating mood it > might be good to just use the Solaris Studio version instead of gcc on that > platform. > > > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, > > Well, LLVM is a significant inhibitor to running anywhere but Intel as far > as I know. LLVM SPARC does not seem to even be in the alpha stage and > doesn't look like it's on anybody's list of things to do. LLVM MIPS is also > not working well from the looks of things on the Debian MIPS list. Other > projects that have adopted LLVM seem to use it only on Intel and have stayed > with gcc on other platforms. There is a concern that will eventually be too > difficult to maintain and the natural solution will be to abandon non-Intel > hardware. > > I understand and agree with the value of LLVM from the standpoint of > reducing code complexity over gcc and also the more liberal license but it's > disappointing to see only the Intel support is probably ever going to be > worth anything as far as LLVM goes. > > > The system was already very portable. I have made it easier to port. > > > > I assume the "difficult" goals are meant to be ensure that people don't > > have to jump through the hoops of "install lots of extra stuff" to get it > > to work. > > I really don't know but I would agree with that. The install should go in > standard paths and not require any prereqs, if possible. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:32:02 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:32:02 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:42:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:42:00 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , Message-ID: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:48:29 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: > Reasonable but painful. And varied. > > > Do you want the Apple-supplied tools or fetch various cc/ld source and > build them? > > > For Windows, Microsoft or Cygwin or Mingw? > > Probe the machine to see if the tools are already present? > Run cc/gcc/cl, see if it works? > > > I like the autoconf model here -- which is indeed that last point. > It tries to compile "int main() { return 0; }", if that works, great a > working C compiler. > If it doesn't, it might fish around for a few, but ultimately it gives up, > and the > user can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. > > > It doesn't know how to apt-get install build-essentials or other. > > > I must say -- autoconf -- I don't like sh or m4, and it is slow, but it > does kinda work well. > I wrestle with this. I output C and I need int8/int16/int32/int64. > I have the following choices > 1 assume char/short/int/__int64/long long > 2 #if on compiler for __int64 vs. long long > 3 try to check limits.h -- paying for the cost of #include for every > compile > 4 check for STDC_VERSION and use inttypes.h/stdint.h > 5 autoconf > 6 something like autoconf > > > For now I do some mix of 1/2/3, but I have this nagging feeling that > autoconf would > gain portability to more systems. > > > Modula-3 follows "the Perl way" and is ported everywhere. > Whereas autoconf can kind of cons up a port if the system is the > combination > of already known elements. Only individual elements need to be known about, > not combinations. > > > autoconf again is ugly and slow, but it is better in terms of "cartesian > factoring". > > And it doesn't handle Microsoft tools well or at all. > > > Also I have a similar nagging dilemma on bootstrap archives. > I want some incrementally. So I want e.g. make. > I want (currently lack) some hierarchy. So I want recursive make or maybe > GNU make-specific. > Maybe automake? Which still doesn't buy me MS make. > cmake? > I don't really want to reinvent portability beyond GNU make. > out-of-source builds? With bootstrapping? Overkill? > So I wonder if I should, gasp, generate automake input. > > make does though cater to the new generation and their IDEs, while also > catering to the old ways. > KDE's use of it is a big gote > > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 12:32:02 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > I think the Mac and Win installers should also install the required > external tools, so it all "just works", which makes it a bit more > difficult. Either way not a priority right now. > > On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:22:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:22:10 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , , , Message-ID: Good thinking...but not easy to pick the one.People will complain no matter what you chose.MS? Cygwin? Mingw? Cross inhibits MS.Cross that isn't prebuilt will take a while to build.MS probably not redistributable. Have to point people at a web page. Chasing annual versions.. - Jay Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:39:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:39:10 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? Message-ID: Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:43:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:43:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? Message-ID: air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Aug 15 22:43:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 16:43:59 -0400 Subject: [M3devel] Current releas(es) Message-ID: <20150815204359.GA15497@topoi.pooq.com> I'm currently installing stuff on an effecively new machine. Where do I find the current Modula 3 release? Is it still http://www.opencm3.net/releng/ ? If not, I'd better update the wikipedia entry. And is CM3 5.8.6 the current release? -- hendrik From jay.krell at cornell.edu Sun Aug 16 00:15:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 15 Aug 2015 22:15:30 +0000 Subject: [M3devel] Current releas(es) In-Reply-To: <20150815204359.GA15497@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: 5.8.6 is unfortunately probably current for most architectures.We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 before: foreach root in [ m3cc, bin ] after:foreach root in [ bin ] - Jay > Date: Sat, 15 Aug 2015 16:43:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: [M3devel] Current releas(es) > > I'm currently installing stuff on an effecively new machine. > > Where do I find the current Modula 3 release? > > Is it still > > http://www.opencm3.net/releng/ > > ? If not, I'd better update the wikipedia entry. > > And is CM3 5.8.6 the current release? > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Aug 16 01:42:07 2015 From: adacore at marino.st (John Marino) Date: Sun, 16 Aug 2015 01:42:07 +0200 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <55CFCE4F.2070104@marino.st> Don't forget FreeBSD has 5.10.0 easily available: http://www.freshports.org/lang/modula3 John On 8/16/2015 12:15 AM, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. > We have a newer build for MacOSX. > You will need one or two patches, if you want to run upgrade.py to get > latest. > I remember this one: > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > > before: > > foreach root in [ m3cc, bin ] > > after: > foreach root in [ bin ] > > > - Jay > >> Date: Sat, 15 Aug 2015 16:43:59 -0400 >> From: hendrik at topoi.pooq.com >> To: m3devel at elegosoft.com >> Subject: [M3devel] Current releas(es) >> >> I'm currently installing stuff on an effecively new machine. >> >> Where do I find the current Modula 3 release? >> >> Is it still >> >> http://www.opencm3.net/releng/ >> >> ? If not, I'd better update the wikipedia entry. >> >> And is CM3 5.8.6 the current release? >> >> -- hendrik >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From hendrik at topoi.pooq.com Sun Aug 16 04:15:41 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 22:15:41 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <20150816021541.GA22514@topoi.pooq.com> On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should really carry a version number. Should I try and find the official Debian syntax for version numbers? -- hendrik We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > before: > foreach root in [ m3cc, bin ] > after:foreach root in [ bin ] > > > - Jay > > > Date: Sat, 15 Aug 2015 16:43:59 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: [M3devel] Current releas(es) > > > > I'm currently installing stuff on an effecively new machine. > > > > Where do I find the current Modula 3 release? > > > > Is it still > > > > http://www.opencm3.net/releng/ > > > > ? If not, I'd better update the wikipedia entry. > > > > And is CM3 5.8.6 the current release? > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From jay.krell at cornell.edu Sun Aug 16 04:20:05 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 15 Aug 2015 19:20:05 -0700 Subject: [M3devel] Current releas(es) In-Reply-To: <20150816021541.GA22514@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: "Yes please take over packaging?" :) Start with scripts/python/make-dist.py? - Jay On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: >> 5.8.6 is unfortunately probably current for most architectures. > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > really carry a version number. Should I try and find the official > Debian syntax for version numbers? > > -- hendrik > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 >> >> before: >> foreach root in [ m3cc, bin ] >> after:foreach root in [ bin ] >> >> >> - Jay >> >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 >>> From: hendrik at topoi.pooq.com >>> To: m3devel at elegosoft.com >>> Subject: [M3devel] Current releas(es) >>> >>> I'm currently installing stuff on an effecively new machine. >>> >>> Where do I find the current Modula 3 release? >>> >>> Is it still >>> >>> http://www.opencm3.net/releng/ >>> >>> ? If not, I'd better update the wikipedia entry. >>> >>> And is CM3 5.8.6 the current release? >>> >>> -- hendrik >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> From microcode at zoho.com Sun Aug 16 09:16:08 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:16:08 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> Message-ID: <20150816071607.GA3479@zoho.com> On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > I already mostly solved this years ago. Oh, that is good news. > SOLgnu and SOLsun were always already the same, e.g. same threading,same > jmpbuf, except the config files picked a different C compiler and > differentoutput directory name. I had misunderstood there was a choice because of the download names, suggesting you actually got a compiler built with Studio or gcc depending on which file you downloaded. > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > compiler.I invented a separate variable with better granularity. That sounds much better, thanks. But for the downloadable stuff where presumably you get a working setup already built and just have to untar the tarball, which compiler(s) are you going forward with now? In other words, since you have consolidate the various Solaris versions as you listed above and it is no longer necessarily indicative of Studio v. gcc, are you going to make turnkey builds available as in the past and if so what will they be built with? And I would vote for using Studio if possible... Thanks. From microcode at zoho.com Sun Aug 16 09:27:30 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:27:30 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150816072730.GB3479@zoho.com> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. From jay.krell at cornell.edu Sun Aug 16 09:52:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 07:52:13 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com> Message-ID: The output of autoconf/automake should have lightweight dependencies.They might stress make, might require GNU make.They might stress the sh, but I think there are adequate shells out there. They are meant to be easy for people building stuff to use.They aren't meant to be easy for people developing stuff to use. Look at this way..while people complain and there are widelyused alternatives like cmake, autoconf/automake are in widespreaduse, and they do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, Aix, Irix, etc. Furthermore, consider any platform that has a native gcc, is likely buildingthat with autoconf/automake. I remain very template for "bootstrap" archives to consist of C and automake/autoconf/libtool output.Or assembly, but still automake/autoconf -- at least to generate makefiles and decide how to run the assembler. - Jay > Date: Sun, 16 Aug 2015 07:27:30 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 16 10:24:09 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 08:24:09 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150816082409.GC3479@zoho.com> On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > The output of autoconf/automake should have lightweight dependencies.They > might stress make, might require GNU make.They might stress the sh, but I > think there are adequate shells out there. That is typically one issue with autoconf, requiring gnu tools in the path. On Solaris this can be annoying since most Solaris people don't use gcc or bash or have them in their path and not all (none of?) the gnu pieces are up to date or even current by any stretch of the imagination. > They are meant to be easy for people building stuff to use.They aren't > meant to be easy for people developing stuff to use. Not sure what you meant here... > Look at this way..while people complain and there are widelyused > alternatives like cmake, autoconf/automake are in widespreaduse, and they > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > Aix, Irix, etc. They often don't "work" for Solaris as-installed but they can most often be made to work. Increasingly, as automake and its prereqs get version bumps there are problems building apps on Solaris because Solaris installs with a very back-level version of gcc and a rather incomplete set of gnu tools. I ran into a problem with the last year where Solaris awk was not good enough to install(!) an app that compiled on Solaris as part of autotools so I had to download a newish copy of gawk and install it. So really, it is much better not to go there if you can avoid it. I am not suggesting an alternative to autotools but just pointing out it is not really accurate to say they work on Linux, Solaris, BSD, etc. except for Linux. I also run BSD on several platforms and because of the same issues as with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, shells, etc. not being good enough for autotools) it is sometimes non-trivial and painful to get Linux apps built on BSD. There is obviously no good/easy solution to this, just to point out it is not the slam-dunk as might be thought. > Furthermore, consider any platform that has a native gcc, is likely > buildingthat with autoconf/automake. Conceded, yet on Solaris it is not clear why gcc is there. It is old and it is in a non-standard location and is often not used. Ideally, apps to be run on Solaris should be able to be built with native (non-gnu) tools. The Studio compilers are very good and have optimizations for SPARC that should be better than what gcc can provide and it is also somewhat of a test of C portability since Studio doesn't necessarily provide all non-standard gcc extensions. From jay.krell at cornell.edu Sun Aug 16 12:04:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:04:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816071607.GA3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com>, , <20150816071607.GA3479@zoho.com> Message-ID: It is like you thought:The SOLsun build used the Sun C compiler, and probably assembler and linker.The SOLgnu build used the GNU C compiler, and maybe assembler, and maybe linker.It isn't a big deal.Turnkey is based on what people have installed.I don't know how to acquire/install the Sun compilers. Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need configurations..just a collection of factors. - Jay > Date: Sun, 16 Aug 2015 07:16:08 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > I already mostly solved this years ago. > > Oh, that is good news. > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > jmpbuf, except the config files picked a different C compiler and > > differentoutput directory name. > > I had misunderstood there was a choice because of the download names, > suggesting you actually got a compiler built with Studio or gcc depending on > which file you downloaded. > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > compiler.I invented a separate variable with better granularity. > > That sounds much better, thanks. But for the downloadable stuff where > presumably you get a working setup already built and just have to untar the > tarball, which compiler(s) are you going forward with now? In other words, > since you have consolidate the various Solaris versions as you listed above > and it is no longer necessarily indicative of Studio v. gcc, are you going > to make turnkey builds available as in the past and if so what will they be > built with? And I would vote for using Studio if possible... > > Thanks. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:11:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:11:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816082409.GC3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com> Message-ID: Autoconf output has little/none dependency.Sun's SPARC optimization are mostly irrelevant as we have little C code,unless you use the C backend. > somewhat of a test of C portability This I appreciate. We have some C code and the C backend.More compilers have "helped".The Tru64 compiler was another.On HP-UX in-box I had only K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And now clang -- having found and worked around a bug in its assembler.Some time soon I'll expand to Metrowerks and Digital Mars.. - Jay > Date: Sun, 16 Aug 2015 08:24:09 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > The output of autoconf/automake should have lightweight dependencies.They > > might stress make, might require GNU make.They might stress the sh, but I > > think there are adequate shells out there. > > That is typically one issue with autoconf, requiring gnu tools in the > path. On Solaris this can be annoying since most Solaris people don't use > gcc or bash or have them in their path and not all (none of?) the gnu pieces > are up to date or even current by any stretch of the imagination. > > > They are meant to be easy for people building stuff to use.They aren't > > meant to be easy for people developing stuff to use. > > Not sure what you meant here... > > > Look at this way..while people complain and there are widelyused > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > Aix, Irix, etc. > > They often don't "work" for Solaris as-installed but they can most often be > made to work. Increasingly, as automake and its prereqs get version bumps > there are problems building apps on Solaris because Solaris installs with a > very back-level version of gcc and a rather incomplete set of gnu tools. I > ran into a problem with the last year where Solaris awk was not good enough > to install(!) an app that compiled on Solaris as part of autotools so I had > to download a newish copy of gawk and install it. So really, it is much > better not to go there if you can avoid it. > > I am not suggesting an alternative to autotools but just pointing out it is > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > Linux. I also run BSD on several platforms and because of the same issues as > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > shells, etc. not being good enough for autotools) it is sometimes > non-trivial and painful to get Linux apps built on BSD. There is obviously > no good/easy solution to this, just to point out it is not the slam-dunk as > might be thought. > > > Furthermore, consider any platform that has a native gcc, is likely > > buildingthat with autoconf/automake. > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > is in a non-standard location and is often not used. Ideally, apps to be run > on Solaris should be able to be built with native (non-gnu) tools. The > Studio compilers are very good and have optimizations for SPARC that should > be better than what gcc can provide and it is also somewhat of a test of C > portability since Studio doesn't necessarily provide all non-standard gcc > extensions. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:12:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? In-Reply-To: References: Message-ID: ok, so this structure_size_boundary isn't entirely bogus.In fact, as is typical, it might have been correct, but itsrelevance has drastically declined as the targets have shifted.Like so much of cm3, it is based closely on gcc. It appears that OpenBSD/68k rounds struct sizes up to a multiple of 2. Other 68k targets do not. Perhaps historical 68k targets did -- perhaps that is a lot of what cm3 was handling. ARM sometimes rounds up to a multiple of 4, but sometimes not, depending on exact platform. sh used to? and still optionally rounds up to a multiple of 4, but I think defaults to not? The only non-8 case we had in cm3 was for HPPA and that was wrong seeming. Getting this value wrong should only lead to a lack of C interop on unusual records.i.e. if you have an array of records with an odd size, the adress of than the firstmight mismatch.It can have an affect in non-array-of-record cases too, if you use BYTESIZE.Or, when the C backend does layout, possibly disagreeing with the frontend. Maybe I'm naive but this all seems like a mistake. sizeof(struct {char a;}) should be 1 for all targets. sizeof(struct {char a[3];}) should be 3 for all targets. but I guess various ABI developers have disagreed through the decadesand we are kinda somewhat stuck with it. There is a gcc command line switch, at least for ARM, to vary the rounding. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: Target.Structure_size_boundary? Date: Fri, 14 Aug 2015 20:43:18 +0000 air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 16 14:18:42 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 16 Aug 2015 08:18:42 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: <20150816121842.GA4830@topoi.pooq.com> On Sat, Aug 15, 2015 at 07:20:05PM -0700, Jay wrote: > "Yes please take over packaging?" :) :) > Start with scripts/python/make-dist.py? As far as I can tell looking at my package cache, all that's needed is to change the name of the file to cm3-LINUXLIBC6-REL_5.8.6_i386.deb When I install it with dpkg, it appears to know internally what revision it is, so not much would need changing. Havint the version number there would help me know whether to bother downloading another copy if I seem to already have one. I probably wasted somoe of your server time by redownloading. -- hendrik > > - Jay > > On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > > > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > >> 5.8.6 is unfortunately probably current for most architectures. > > > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > > really carry a version number. Should I try and find the official > > Debian syntax for version numbers? > > > > -- hendrik > > > > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > >> > >> before: > >> foreach root in [ m3cc, bin ] > >> after:foreach root in [ bin ] > >> > >> > >> - Jay > >> > >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 > >>> From: hendrik at topoi.pooq.com > >>> To: m3devel at elegosoft.com > >>> Subject: [M3devel] Current releas(es) > >>> > >>> I'm currently installing stuff on an effecively new machine. > >>> > >>> Where do I find the current Modula 3 release? > >>> > >>> Is it still > >>> > >>> http://www.opencm3.net/releng/ > >>> > >>> ? If not, I'd better update the wikipedia entry. > >>> > >>> And is CM3 5.8.6 the current release? > >>> > >>> -- hendrik > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> From microcode at zoho.com Sun Aug 16 15:39:31 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:39:31 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> <20150816071607.GA3479@zoho.com> Message-ID: <20150816133931.GA5008@zoho.com> On Sun, Aug 16, 2015 at 10:04:29AM +0000, Jay K wrote: > It is like you thought:The SOLsun build used the Sun C compiler, and > probably assembler and linker.The SOLgnu build used the GNU C compiler, and > maybe assembler, and maybe linker.It isn't a big deal. Turnkey is based on > what people have installed. Are you saying now that you have consolidated the builds into the 4 you listed below they will use what the user has installed? If so that sounds ideal, except for possible search order issues. The gcc that "comes with" Solaris is always installed in the same location (/usr/sfw) but the path to Studio depends on the version and I believe you could create non-standard paths using the tarball installers vs. the package installers (Studio offers both options IIRC). And then there is what the user has in their PATH which might be all, one, or none of the above. And there are some users who install new(er) versions of gcc, usually in addition to the stock version and there is no convention for where this gcc should go. So...there should probably be a documented way to use the toolchain of your choice, if this isn't already explained somewhere. > I don't know how to acquire/install the Sun compilers. They are a free download from Oracle (free to use and free runtime). If they are not already installed on your opencsw box(es) and you would like to have them on there I can do that. It is just a matter of downloading and installing the packages. It should take 20 to 30 minutes. > Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, > AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need > configurations..just a collection of factors. I think this is the right thing to do, I just don't understand how to select which toolchain you want to use, and further, I thought the downloads contained a prebuilt cm3, so I don't understand why a C toolchain would be required at all, unless you are saying in the future the user will compile cm3 rather than get a prebuilt binary. > - Jay > > > > Date: Sun, 16 Aug 2015 07:16:08 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > > I already mostly solved this years ago. > > > > Oh, that is good news. > > > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > > jmpbuf, except the config files picked a different C compiler and > > > differentoutput directory name. > > > > I had misunderstood there was a choice because of the download names, > > suggesting you actually got a compiler built with Studio or gcc depending on > > which file you downloaded. > > > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > > compiler.I invented a separate variable with better granularity. > > > > That sounds much better, thanks. But for the downloadable stuff where > > presumably you get a working setup already built and just have to untar the > > tarball, which compiler(s) are you going forward with now? In other words, > > since you have consolidate the various Solaris versions as you listed above > > and it is no longer necessarily indicative of Studio v. gcc, are you going > > to make turnkey builds available as in the past and if so what will they be > > built with? And I would vote for using Studio if possible... > > > > Thanks. > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From microcode at zoho.com Sun Aug 16 15:43:06 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:43:06 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> Message-ID: <20150816134306.GB5008@zoho.com> On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > Autoconf output has little/none dependency.Sun's SPARC optimization are > mostly irrelevant as we have little C code,unless you use the C backend. Ok, well, it sounds like it will be increasingly important as you go to your C backend though. > > somewhat of a test of C portability > This I appreciate. We have some C code and the C backend.More compilers > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > now clang -- having found and worked around a bug in its assembler.Some > time soon I'll expand to Metrowerks and Digital Mars.. I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have the latest versions. C/C++ are not my thing but if I can help by just building stuff to see what messages we get or if it breaks etc. let me know and I will try to participate. I didn't know Metrowerks was still around. I thought they got swallowed up by Motorola and then removed from all the non-embedded space. > - Jay > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > The output of autoconf/automake should have lightweight dependencies.They > > > might stress make, might require GNU make.They might stress the sh, but I > > > think there are adequate shells out there. > > > > That is typically one issue with autoconf, requiring gnu tools in the > > path. On Solaris this can be annoying since most Solaris people don't use > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > are up to date or even current by any stretch of the imagination. > > > > > They are meant to be easy for people building stuff to use.They aren't > > > meant to be easy for people developing stuff to use. > > > > Not sure what you meant here... > > > > > Look at this way..while people complain and there are widelyused > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > Aix, Irix, etc. > > > > They often don't "work" for Solaris as-installed but they can most often be > > made to work. Increasingly, as automake and its prereqs get version bumps > > there are problems building apps on Solaris because Solaris installs with a > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > ran into a problem with the last year where Solaris awk was not good enough > > to install(!) an app that compiled on Solaris as part of autotools so I had > > to download a newish copy of gawk and install it. So really, it is much > > better not to go there if you can avoid it. > > > > I am not suggesting an alternative to autotools but just pointing out it is > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > Linux. I also run BSD on several platforms and because of the same issues as > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > shells, etc. not being good enough for autotools) it is sometimes > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > no good/easy solution to this, just to point out it is not the slam-dunk as > > might be thought. > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > buildingthat with autoconf/automake. > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > is in a non-standard location and is often not used. Ideally, apps to be run > > on Solaris should be able to be built with native (non-gnu) tools. The > > Studio compilers are very good and have optimizations for SPARC that should > > be better than what gcc can provide and it is also somewhat of a test of C > > portability since Studio doesn't necessarily provide all non-standard gcc > > extensions. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From jay.krell at cornell.edu Sun Aug 16 20:28:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 18:28:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: Message-ID: Actually, I suggest that endianness might be removable as a factor. I know of the following endian-dependentness: 1) bitfield layout This matters "only" so that bitfields can be declared that line up with C to interoperate with. Or, sort of, to interoperate with hardware -- but hardware probably has no endianness, i.e. much hardware can be used with both little and big endian CPUs, and this variability actually makes portabililiy more difficult. I suggest this might not matter and we can pick an arbitrary endianness, at least for the C backend. 2) To pick code that picks apart floating point numbers. I suggest this code can be rewritten in C and/or do a runtime detection of endianness, in C or Modula-3. I don't think endianess matters in initialization non-bitfields. Or at least integers. I have to check. Potentially leaving us with just 4: 32bit posix 64bit posix 32bit windows 64bit windows I know at least one more Target factor to bring up, later (alignment of closures). - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: posix/nt/32/64/endian platforms for C backend? Date: Fri, 14 Aug 2015 20:39:10 +0000 Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Sun Aug 16 22:59:15 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 13:59:15 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 23:40:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 21:40:45 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, Message-ID: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 02:04:03 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 17:04:03 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, > including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically > get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't > have X installed, it fails. We can make it do nothing. We can link to the > stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap > archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if > using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how > to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse > it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU > build system" > It is slow and bad to author in, but produces easy to use somewhat slow > idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would > be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: > install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. > Even gcc. > > > - Jay > > > > ------------------------------ > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus > different distributions, there are different views on how CM3 should be > setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I > want to take it further than the current installer. In particular I want to > reduce the external, internal and environmental dependencies so that you > can build anything, anywhere with just the minimal set of executables and a > standard set of tools. Others want a more elaborate setup which gives them > more flexibility; that can also be derived from that same minimal set. > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 17 03:24:07 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Aug 2015 18:24:07 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU build system" >> It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. Even gcc. >> >> >> - Jay >> >> >> >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and so >> forth. >> >> > What this system spits out is everything you need to compile anything you >> > want - a complete and working compiler. Not everyone wants or needs to use >> > all the packages. If you do, you can. Here are the instructions for doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I do >> I think anybody new is going to have similar questions/misunderstandings of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 05:14:27 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link > it all. > > > The compiler has to come with a matching m3core/libm3, binary and/or > source. The cm3cg binary can be built from source, but again it has to be > matching -- or use C backend. > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can > be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, > minimal toolset (mostly just gcc), installed in a particular place. I'm not > trying to support or test every configuration, just one. It's for people > who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know > there are no external source dependencies in the frontend and the backend > depends only on the gcc source which is in the repository. The build server > only builds the frontend and backend executables, nothing else, so there > are no other dependencies. > > If you're talking about building an X Windows application, I still don't > get it. Why do I need to support someone wanting to write an X program? I'm > not supporting people who want to do anything other than run the compiler > on their platform. If they want to make X programs they can download and > configure whatever is needed themselves. The CM3 compiler can compile (but > not link) M3 interfaces and modules for X without having X installed. > That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, >> including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically >> get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and >> don't have X installed, it fails. We can make it do nothing. We can link to >> the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap >> archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if >> using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to >> how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse >> it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU >> build system" >> It is slow and bad to author in, but produces easy to use somewhat slow >> idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would >> be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: >> install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. >> Even gcc. >> >> >> - Jay >> >> >> >> ------------------------------ >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and >> thus different distributions, there are different views on how CM3 should >> be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I >> want to take it further than the current installer. In particular I want to >> reduce the external, internal and environmental dependencies so that you >> can build anything, anywhere with just the minimal set of executables and a >> standard set of tools. Others want a more elaborate setup which gives them >> more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend >> just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to >> me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from >> the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and >> so >> forth. >> >> > What this system spits out is everything you need to compile anything >> you >> > want - a complete and working compiler. Not everyone wants or needs to >> use >> > all the packages. If you do, you can. Here are the instructions for >> doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been >> following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I >> do >> I think anybody new is going to have similar questions/misunderstandings >> of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> >> _______________________________________________ M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:42:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:42:03 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , Message-ID: I think you want is roughly: boot1.py over each architecture package those up along with matching m3core/libm3/m3cc source boot2.sh or such on the target system -- but this currently assumes an entire matching source tree, so this needs work But fair enough, you just want compilers, to get around the "crossing". Crossing is easy as far as we do it -- just see boot1.py. It only gets as far as assembly files. Then you move them to the target machine and run native assembler and linker. ARM_LINUX is a sore point for me. Debian has been through at least three different arm ports. For this all to work reasonably cleanly we have to track something like this: https://wiki.debian.org/Multiarch/Tuples Meanwhile, m3front barely cares about platform at all. Almost all platforms generate identical code until they hit the backend. And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. Our "porting" and "supported platforms" is much ado about little. We support "everything and nothing", by leveraging underlying gcc or C compiler and pthreads or Win32, and X or Win32. - Jay Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com; microcode at zoho.com If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:43:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:43:37 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> Message-ID: But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else.It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 18 03:39:09 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 17 Aug 2015 21:39:09 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150818013909.GA16240@topoi.pooq.com> On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > I think you want is roughly: > boot1.py over each architecture > package those up > along with matching m3core/libm3/m3cc source > boot2.sh or such on the target system -- but this currently > assumes an entire matching source tree, so this needs work > > But fair enough, you just want compilers, to get around the "crossing". > Crossing is easy as far as we do it -- just see boot1.py. > It only gets as far as assembly files. Then you move them to the target > machine and run native assembler and linker. Suppose you used C instead of assembler for this. Would that get you more machine types for less effort? I've had a machine on which there was a dispute which assembler to use! > > ARM_LINUX is a sore point for me. > Debian has been through at least three different arm ports. > For this all to work reasonably cleanly we have to track > something like this: > https://wiki.debian.org/Multiarch/Tuples > > Meanwhile, m3front barely cares about platform at all. > Almost all platforms generate identical code until they hit the backend. > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > Our "porting" and "supported platforms" is much ado about little. > We support "everything and nothing", by leveraging underlying gcc > or C compiler and pthreads or Win32, and X or Win32. > > - Jay > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > Subject: Re: [M3devel] Build Server - Plan > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; microcode at zoho.com > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link it all. > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > Python is convenient for cross-platform scripting, but optional. > - Jay > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. Even gcc. > > > - Jay > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > The solution for SPARC might be to create a cross-compiler. That's not > > > ideal but would be useful for verifying that things compile for the > > > platform at least, short of having the right hardware. > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > machine time but I can't keep the machine running all the time. Hopefully > > the current setup (opencsw) is still viable. > > > > > I disagree with your "only for experts" assessment. The point of this > > > server is that you don't have to compiler the compiler and a backend just > > > to get the latest (or even a properly working) compiler. That's the very > > > "expert" work I'm trying to automate. > > > > That is good to hear because from your list of deliverables it seemed to me > > there would still be a lot of tinkering to get a complete install. If you > > are scripting that and it can be built effortlessly and mindlessly from the > > deliverables then good. I think it would be ideal and very nice if people > > could download a turnkey tarball ready to run on their platform, or source > > code ready to build and an installer script that installs the final > > product. Jay mentioned autotools and they are usually far from optimal on > > Solaris because of autotools reliance on so much gnu stuff that is not > > always necessarily present on Solaris but that is the general idea- to be > > able to configure && make && make install with all the standard stuff > > happening like everything being installed in /usr/local/bin|lib|share and so > > forth. > > > > > What this system spits out is everything you need to compile anything you > > > want - a complete and working compiler. Not everyone wants or needs to use > > > all the packages. If you do, you can. Here are the instructions for doing > > > that: "cd ; cm3 " > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > sorry if I wasted anybody's time with this diversion. I have been following > > the mailing list and it looks like Jay and Rodney are mostly discussing > > things with a very small number of regulars. Coming from the outside as I do > > I think anybody new is going to have similar questions/misunderstandings of > > how cm3 is built and installed, etc. > > > > > I'll be opening up the server to anyone who wants to do something > > > different. If someone wants to produce tarballs they can setup their own > > > VMs and they will get built too. > > > > Hopefully this can be integrated with what you already have running on > > opencsw. > > > > Thank you. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Aug 18 04:31:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:31:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu>, , Message-ID: Is bitfield layout really portable enough and predictable enough that interop is achievable?I don't know the layout algorithms myself.I don't really know what packed does.Change the alignment rules for the record?Remove all padding for alignment? And then access fields piecemeal in case anything is no longer aligned?I should experiment. - Jay Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? From: hosking at purdue.edu Date: Tue, 18 Aug 2015 11:57:52 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 18, 2015, at 9:43 AM, Jay K wrote:But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else. And doesn?t interrupt make sense wherever possible. I would hope that there is a reasonable correspondence between C types and M3 types, particularly for RECORD and PACKED elements. It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 04:37:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:37:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150818013909.GA16240@topoi.pooq.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , , , <20150818013909.GA16240@topoi.pooq.com> Message-ID: Yes, but use of C is tied to the C backend.If you want to use cm3cg or LLVM it isn't an option.cm3cg was implied, but the C backend is quite good and workingand I recommend it for all targets. The reason for assembly is that is cm3cg output and probablyoptional LLVM output. I'm not sure people are ready/willing to give up cm3cg yet. ? In fact, with the C backend, we are close to having just a small number, like 6-8,service all targets, as discussed in another thread.The only factors become word size, possibly endian, NT vs. posix.(I have to send a mail yet on "aligned procedure types" -- which correlateswith processor, i.e. the same for all x86 targets). "exact" target doesn't matter.esp. with the jmpbuf thing removed. > I've had a machine on which there was a dispute which assembler to use! Which assembler to use is even a problem on modern Mac OSX 10.10.4 Yosemite.The default LLVM assembler, invoked by the default "gcc" (which is actually clang),is not compatible with gcc/cm3cg. It produces the wrong code.The default assembler works. I put a workaround into cm3cg though. It is really just a mess that any compiler these days doesn't default to outputing.o files directly, but that is gcc. - Jay > Date: Mon, 17 Aug 2015 21:39:09 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > > I think you want is roughly: > > boot1.py over each architecture > > package those up > > along with matching m3core/libm3/m3cc source > > boot2.sh or such on the target system -- but this currently > > assumes an entire matching source tree, so this needs work > > > > But fair enough, you just want compilers, to get around the "crossing". > > Crossing is easy as far as we do it -- just see boot1.py. > > It only gets as far as assembly files. Then you move them to the target > > machine and run native assembler and linker. > > Suppose you used C instead of assembler for this. Would that get > you more machine types for less effort? I've had a machine on which > there was a dispute which assembler to use! > > > > > ARM_LINUX is a sore point for me. > > Debian has been through at least three different arm ports. > > For this all to work reasonably cleanly we have to track > > something like this: > > https://wiki.debian.org/Multiarch/Tuples > > > > Meanwhile, m3front barely cares about platform at all. > > Almost all platforms generate identical code until they hit the backend. > > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > > > Our "porting" and "supported platforms" is much ado about little. > > We support "everything and nothing", by leveraging underlying gcc > > or C compiler and pthreads or Win32, and X or Win32. > > > > - Jay > > > > > > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > > Subject: Re: [M3devel] Build Server - Plan > > From: lists at darko.org > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com; microcode at zoho.com > > > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > > I meant like "the entire system" not "just the compiler". Right, to link it all. > > > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > > On many systems, even X is just an "apt get whatever" away. > > And, if we form our .deb correctly, it is fairly automatic. > > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > > > > How to ease that apart? A sort of dependency on X? > > Likely separate packages, like cm3 and cm3-gui? > > > > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > > Imagine if, you know, our config files dropped away entirely. > > Imagine if the config file was generated at install time by autoconf. > > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > > Install from binaries is faster. > > Install from source is problematic for compilers written in themselves. Even gcc. > > > > > > - Jay > > > > > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > > From: lists at darko.org > > To: m3devel at elegosoft.com; microcode at zoho.com > > Subject: Re: [M3devel] Build Server - Plan > > > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > > > The solution for SPARC might be to create a cross-compiler. That's not > > > > > ideal but would be useful for verifying that things compile for the > > > > > platform at least, short of having the right hardware. > > > > > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > > > machine time but I can't keep the machine running all the time. Hopefully > > > > the current setup (opencsw) is still viable. > > > > > > > > > I disagree with your "only for experts" assessment. The point of this > > > > > server is that you don't have to compiler the compiler and a backend just > > > > > to get the latest (or even a properly working) compiler. That's the very > > > > > "expert" work I'm trying to automate. > > > > > > > > That is good to hear because from your list of deliverables it seemed to me > > > > there would still be a lot of tinkering to get a complete install. If you > > > > are scripting that and it can be built effortlessly and mindlessly from the > > > > deliverables then good. I think it would be ideal and very nice if people > > > > could download a turnkey tarball ready to run on their platform, or source > > > > code ready to build and an installer script that installs the final > > > > product. Jay mentioned autotools and they are usually far from optimal on > > > > Solaris because of autotools reliance on so much gnu stuff that is not > > > > always necessarily present on Solaris but that is the general idea- to be > > > > able to configure && make && make install with all the standard stuff > > > > happening like everything being installed in /usr/local/bin|lib|share and so > > > > forth. > > > > > > > > > What this system spits out is everything you need to compile anything you > > > > > want - a complete and working compiler. Not everyone wants or needs to use > > > > > all the packages. If you do, you can. Here are the instructions for doing > > > > > that: "cd ; cm3 " > > > > > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > > > sorry if I wasted anybody's time with this diversion. I have been following > > > > the mailing list and it looks like Jay and Rodney are mostly discussing > > > > things with a very small number of regulars. Coming from the outside as I do > > > > I think anybody new is going to have similar questions/misunderstandings of > > > > how cm3 is built and installed, etc. > > > > > > > > > I'll be opening up the server to anyone who wants to do something > > > > > different. If someone wants to produce tarballs they can setup their own > > > > > VMs and they will get built too. > > > > > > > > Hopefully this can be integrated with what you already have running on > > > > opencsw. > > > > > > > > Thank you. > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 00:20:25 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 19 Aug 2015 17:20:25 -0500 Subject: [M3devel] cm3 is broken Message-ID: <55D50129.6030600@lcwb.coop> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into runaway recursion trying to report a fault. This happens immediately upon running it, before it produces any output. Here is one cycle of the backtrace: #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 #1748 0x00007fd9b0b0ccfe in Raise (act= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) at ../src/runtime/common/RTHooks.m3:79 #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 So far, I haven't had the patience to keep hitting return until I get to the bottom of it. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 01:57:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 19 Aug 2015 23:57:54 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: Did you upgrade.py? (or possibly upgrade.sh?) I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. I run upgrade.py frequently and recommend it. - Jay > Date: Wed, 19 Aug 2015 17:20:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: [M3devel] cm3 is broken > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 20 11:38:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 20 Aug 2015 09:38:37 +0000 Subject: [M3devel] First_readable_addr? Message-ID: First_readable_addr looks off by a factor of 8.But I haven't gotten it to trigger at all yet. I'll debug further. Proposals: - remove First_readable_addr as wrong or non functional? - fix First_readable_addr - Most importantly, First_readable_addr is I believe usually 4k, sometimes 8k.It is 8k for sparc. It "should" also be 8k for ia64 and alpha. However, the proposal is to make it always 4k, for more commonality among targets.Lower is safe, but a possible very slight loss in efficiency when it is too low. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 17:44:39 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 10:44:39 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> Message-ID: <55D5F5E7.5000109@lcwb.coop> On 08/19/2015 06:57 PM, Jay K wrote: > Did you upgrade.py? (or possibly upgrade.sh?) > upgrade.py does this: rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py Traceback (most recent call last): File "upgrade.py", line 4, in import pylib File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in if Target.startswith("NT386"): AttributeError: 'NoneType' object has no attribute 'startswith' I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', with front removed from the m3cc line in pkginfo.txt, since there were no changes to it, and it takes longer to compile than everything else combined. upgrade.sh now segfaults after cleaning things out, but that is no doubt just the now-broken cm3. I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing a compiler, but restoring what I had before does not fix the problem. I suppose it is in m3core, which I guess I need to add to my backup habit. > I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > I run upgrade.py frequently and recommend it. > > - Jay > > > > Date: Wed, 19 Aug 2015 17:20:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: [M3devel] cm3 is broken > > > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > runaway recursion trying to report a fault. This happens immediately upon > > running it, before it produces any output. > > > > Here is one cycle of the backtrace: > > > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > #1748 0x00007fd9b0b0ccfe in Raise (act= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > at ../src/runtime/common/RTHooks.m3:79 > > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 19:19:01 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 20 Aug 2015 10:19:01 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D5F5E7.5000109@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: Upgrade is failing because cm3 is already broken. I can see about making the error clearer. I often upgrade.py nocleangcc to speed it up. Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. Do you have recourse? Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) - Jay On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > > On 08/19/2015 06:57 PM, Jay K wrote: >> Did you upgrade.py? (or possibly upgrade.sh?) > > upgrade.py does this: > > rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > Traceback (most recent call last): > File "upgrade.py", line 4, in > import pylib > File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > if Target.startswith("NT386"): > AttributeError: 'NoneType' object has no attribute 'startswith' > > I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > with front removed from the m3cc line in pkginfo.txt, since there were no > changes to it, and it takes longer to compile than everything else combined. > > upgrade.sh now segfaults after cleaning things out, but that is no doubt just > the now-broken cm3. > > I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > a compiler, but restoring what I had before does not fix the problem. I suppose > it is in m3core, which I guess I need to add to my backup habit. > >> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> >> I run upgrade.py frequently and recommend it. >> >> - Jay >> >> >> > Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] cm3 is broken >> > >> > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > runaway recursion trying to report a fault. This happens immediately upon >> > running it, before it produces any output. >> > >> > Here is one cycle of the backtrace: >> > >> > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > #1748 0x00007fd9b0b0ccfe in Raise (act= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > at ../src/runtime/common/RTHooks.m3:79 >> > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 20 20:39:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 13:39:58 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: <55D61EFE.1040302@lcwb.coop> I have some old complete installations. One of them will probably be enough to build a compiler. No time just yet. On 08/20/2015 12:19 PM, Jay wrote: > Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > > I often upgrade.py nocleangcc to speed it up. > > > Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > > m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > > Do you have recourse? > Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > - Jay > > On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/19/2015 06:57 PM, Jay K wrote: >>> Did you upgrade.py? (or possibly upgrade.sh?) >> >> upgrade.py does this: >> >> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> Traceback (most recent call last): >> File "upgrade.py", line 4, in >> import pylib >> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> if Target.startswith("NT386"): >> AttributeError: 'NoneType' object has no attribute 'startswith' >> >> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> with front removed from the m3cc line in pkginfo.txt, since there were no >> changes to it, and it takes longer to compile than everything else combined. >> >> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> the now-broken cm3. >> >> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> a compiler, but restoring what I had before does not fix the problem. I suppose >> it is in m3core, which I guess I need to add to my backup habit. >> >>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>> >>> I run upgrade.py frequently and recommend it. >>> >>> - Jay >>> >>> >>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com >>>> Subject: [M3devel] cm3 is broken >>>> >>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>> runaway recursion trying to report a fault. This happens immediately upon >>>> running it, before it produces any output. >>>> >>>> Here is one cycle of the backtrace: >>>> >>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>> at ../src/runtime/common/RTHooks.m3:79 >>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> >>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From wagner at elego.de Fri Aug 21 14:46:44 2015 From: wagner at elego.de (Olaf Wagner) Date: Fri, 21 Aug 2015 14:46:44 +0200 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: References: Message-ID: <20150821144644.345ac1a25b429e6c35854d32@elego.de> On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rodney_bates at lcwb.coop Fri Aug 21 18:38:32 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 21 Aug 2015 11:38:32 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> Message-ID: <55D75408.3010900@lcwb.coop> On 08/20/2015 06:32 PM, Antony Hosking wrote: > cm3 should build with static link to m3core so as to allow backup as you describe. > Did this change recently? Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting a breakpoint at one of these places gives: (m3gdb) b RTExFrame.m3:175 No source file named RTExFrame.m3. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (RTExFrame.m3:175) pending. (m3gdb) run ........ Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. Pending breakpoint "RTExFrame.m3:175" resolved > >> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> >> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> >> >> I often upgrade.py nocleangcc to speed it up. >> >> >> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> >> >> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> >> >> Do you have recourse? >> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> >> - Jay >> >> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> >>> >>> >>> On 08/19/2015 06:57 PM, Jay K wrote: >>>> Did you upgrade.py? (or possibly upgrade.sh?) >>> >>> upgrade.py does this: >>> >>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >>> Traceback (most recent call last): >>> File "upgrade.py", line 4, in >>> import pylib >>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >>> if Target.startswith("NT386"): >>> AttributeError: 'NoneType' object has no attribute 'startswith' >>> >>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >>> with front removed from the m3cc line in pkginfo.txt, since there were no >>> changes to it, and it takes longer to compile than everything else combined. >>> >>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >>> the now-broken cm3. >>> >>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >>> a compiler, but restoring what I had before does not fix the problem. I suppose >>> it is in m3core, which I guess I need to add to my backup habit. >>> >>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>>> >>>> I run upgrade.py frequently and recommend it. >>>> >>>> - Jay >>>> >>>> >>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>>> From: rodney_bates at lcwb.coop >>>>> To: m3devel at elegosoft.com >>>>> Subject: [M3devel] cm3 is broken >>>>> >>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>> running it, before it produces any output. >>>>> >>>>> Here is one cycle of the backtrace: >>>>> >>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> >>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>> >>>>> -- >>>>> Rodney Bates >>>>> rodney.m.bates at acm.org >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 11:28:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 09:28:01 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 12:00:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 10:00:18 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: It works for me. Can you go back to a working install, maybe 5.8.6 with that slight config patch,run upgrade.py, add see what you get?Send the build log maybe? jair:python jay$ otool -L /cm3/bin/cm3/cm3/bin/cm3: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) jair:python jay$ /cm3/bin/cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-22 09:57:19 configuration: /cm3/bin/cm3.cfg host: I386_DARWIN target: I386_DARWIN On most other systems, use ldd.On Windows use link /dump /imports or link /dump /dependents. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu; rodney.m.bates at acm.org Date: Sat, 22 Aug 2015 09:28:01 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3 is broken This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Aug 22 17:44:04 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 10:44:04 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: <55D898C4.90305@lcwb.coop> Going back several cm3 backups until I see a change, I have: root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 linux-vdso.so.1 => (0x00007fff211fe000) libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-07-31 18:44:16 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 linux-vdso.so.1 => (0x00007fffa47a7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-06-04 14:24:48 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX So I compiled the change to dynamic between June 4 and July 31. On 08/22/2015 05:00 AM, Jay K wrote: > It works for me. > > Can you go back to a working install, maybe 5.8.6 with that slight config patch, > run upgrade.py, add see what you get? > Send the build log maybe? > > > jair:python jay$ otool -L /cm3/bin/cm3 > /cm3/bin/cm3: > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) > /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) > > jair:python jay$ /cm3/bin/cm3 -version > > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-22 09:57:19 > configuration: /cm3/bin/cm3.cfg > host: I386_DARWIN > target: I386_DARWIN > > > On most other systems, use ldd. > On Windows use link /dump /imports or link /dump /dependents. > > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: hosking at purdue.edu; rodney.m.bates at acm.org > Date: Sat, 22 Aug 2015 09:28:01 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken > > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:07:01 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:07:01 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: <55D89E25.9000002@lcwb.coop> On 08/22/2015 04:28 AM, Jay K wrote: > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". I think this is a very good idea. The in-place rebuilds always make me very nervous. I never feel like I quite know what is going on, and I never quite trust any of the processes we have (and rightly so, from experience). When it works, or appears to, it always is more a matter of experiment than understanding. Whenever I am the least bit suspicious, I feel the need to copy two whole subtrees (source & installed) to update in place, keeping the originals to make another copy from, if problems arise. But that's a pretty tedious process to go through every time, so I have to evaluate the risks and decide when to take them and when to play safe. And when I touch the compiler or runtime, I feel the need to rebuild all of front twice, by whatever process I am using, then look for at least identical sized executables and libraries. > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:33:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:33:12 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: <55D8A448.30703@lcwb.coop> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = VAR p := LOOPHOLE (f, PF1); BEGIN IF DEBUG THEN PutExcept ("INVOKE HANDLER", a); RTIO.PutText (" frame="); RTIO.PutAddr (f); RTIO.PutText (" class="); RTIO.PutInt (f.class); RTIO.PutText ("\n"); RTIO.Flush (); END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *) <* ASSERT p.jmpbuf # NIL *> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; (m3gdb) frame 0 #0 InvokeHandler (f=16_00007fff509f4af0, a= RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 175 <* ASSERT p.jmpbuf # NIL *> (m3gdb) p p.jmpbuf $31 = NIL --------^^^ (m3gdb) The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = VAR status: File.Status; fname := M3toC.SharedTtoS(pn); BEGIN IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; ---------^^^^^^^^^^^^^^^^^^^^^^ M3toC.FreeSharedS(pn, fname); RETURN status END Status; (m3gdb) #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p pn $30 = (*16_00000000013e4ac0*) "./cm3.cfg" --------------------------------^^^^^^^^^^^ (m3gdb) I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? Also, I notice one other thing, that probably isn't part of this problem, but maybe needs to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, after the failing CStatus above. Obviously not thread safe. Does it matter here? As I recall from many years ago, there was a change in C library that provided a thread safe alternative to this, and I even think I had to change something in pm3 or SRC to adapt to it. Do we care? PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = VAR err := Cerrno.GetErrno(); ---------------^^^^^^^^^^^^^^^^^ BEGIN M3toC.FreeSharedS(p, f); OSErrorPosix.Raise0(err); END Fail; On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:41:14 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:41:14 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A448.30703@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> Message-ID: <55D8A62A.8030205@lcwb.coop> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > VAR p := LOOPHOLE (f, PF1); > BEGIN > IF DEBUG THEN > PutExcept ("INVOKE HANDLER", a); > RTIO.PutText (" frame="); RTIO.PutAddr (f); > RTIO.PutText (" class="); RTIO.PutInt (f.class); > RTIO.PutText ("\n"); > RTIO.Flush (); > END; > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > p.info := a; (* copy the exception to the new frame *) > <* ASSERT p.jmpbuf # NIL *> > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > RAISE OUCH; > END InvokeHandler; > > (m3gdb) frame 0 > #0 InvokeHandler (f=16_00007fff509f4af0, a= > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > 175 <* ASSERT p.jmpbuf # NIL *> > (m3gdb) p p.jmpbuf > $31 = NIL > --------^^^ > (m3gdb) > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > BEGIN > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > ---------^^^^^^^^^^^^^^^^^^^^^^ > M3toC.FreeSharedS(pn, fname); > RETURN status > END Status; > > (m3gdb) > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p pn > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > --------------------------------^^^^^^^^^^^ > (m3gdb) > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? A bit more info: (m3gdb) down #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p Process.GetWorkingDirectory() $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (m3gdb) This is where I ran cm3 from. > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > after the failing CStatus above. Obviously not thread safe. Does it matter here? > As I recall from many years ago, there was a change in C library that provided a thread > safe alternative to this, and I even think I had to change something in pm3 or SRC to > adapt to it. Do we care? > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > VAR err := Cerrno.GetErrno(); > ---------------^^^^^^^^^^^^^^^^^ > BEGIN > M3toC.FreeSharedS(p, f); > OSErrorPosix.Raise0(err); > END Fail; > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> runaway recursion trying to report a fault. This happens immediately upon >> running it, before it produces any output. >> >> Here is one cycle of the backtrace: >> >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> at ../src/runtime/common/RTHooks.m3:79 >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 19:26:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 17:26:33 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> Message-ID: It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) The interface here between cm3's generated code/data and m3core has changed. You cannot mix them. It will fail. - Jay > Date: Sat, 22 Aug 2015 11:41:14 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] cm3 is broken: More info > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > VAR p := LOOPHOLE (f, PF1); > > BEGIN > > IF DEBUG THEN > > PutExcept ("INVOKE HANDLER", a); > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > RTIO.PutText ("\n"); > > RTIO.Flush (); > > END; > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > p.info := a; (* copy the exception to the new frame *) > > <* ASSERT p.jmpbuf # NIL *> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > RAISE OUCH; > > END InvokeHandler; > > > > (m3gdb) frame 0 > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > 175 <* ASSERT p.jmpbuf # NIL *> > > (m3gdb) p p.jmpbuf > > $31 = NIL > > --------^^^ > > (m3gdb) > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > BEGIN > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > M3toC.FreeSharedS(pn, fname); > > RETURN status > > END Status; > > > > (m3gdb) > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p pn > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > --------------------------------^^^^^^^^^^^ > > (m3gdb) > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > As I recall from many years ago, there was a change in C library that provided a thread > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > adapt to it. Do we care? > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > VAR err := Cerrno.GetErrno(); > > ---------------^^^^^^^^^^^^^^^^^ > > BEGIN > > M3toC.FreeSharedS(p, f); > > OSErrorPosix.Raise0(err); > > END Fail; > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >> runaway recursion trying to report a fault. This happens immediately upon > >> running it, before it produces any output. > >> > >> Here is one cycle of the backtrace: > >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >> at ../src/runtime/common/RTHooks.m3:79 > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >> > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 20:47:00 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 11:47:00 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> Message-ID: On the other matter -- Cerrno.GetErrno() is thread safe. It ends up as a function that accesses a thread local. At least on Windows and surely that is the only logical approach. It is still a bad design. Thread locals don't make things all better. They are still "fragile" (easy to change by accident), slow, and don't provide for reentrance. - Jay On Aug 22, 2015, at 9:41 AM, "Rodney M. Bates" wrote: > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> >> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> VAR p := LOOPHOLE (f, PF1); >> BEGIN >> IF DEBUG THEN >> PutExcept ("INVOKE HANDLER", a); >> RTIO.PutText (" frame="); RTIO.PutAddr (f); >> RTIO.PutText (" class="); RTIO.PutInt (f.class); >> RTIO.PutText ("\n"); >> RTIO.Flush (); >> END; >> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> p.info := a; (* copy the exception to the new frame *) >> <* ASSERT p.jmpbuf # NIL *> >> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> RAISE OUCH; >> END InvokeHandler; >> >> (m3gdb) frame 0 >> #0 InvokeHandler (f=16_00007fff509f4af0, a= >> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> 175 <* ASSERT p.jmpbuf # NIL *> >> (m3gdb) p p.jmpbuf >> $31 = NIL >> --------^^^ >> (m3gdb) >> >> >> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> >> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> BEGIN >> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> ---------^^^^^^^^^^^^^^^^^^^^^^ >> M3toC.FreeSharedS(pn, fname); >> RETURN status >> END Status; >> >> (m3gdb) >> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> at ../src/os/POSIX/FSPosix.m3:328 >> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> (m3gdb) p pn >> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> --------------------------------^^^^^^^^^^^ >> (m3gdb) >> >> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > >> >> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> after the failing CStatus above. Obviously not thread safe. Does it matter here? >> As I recall from many years ago, there was a change in C library that provided a thread >> safe alternative to this, and I even think I had to change something in pm3 or SRC to >> adapt to it. Do we care? >> >> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> VAR err := Cerrno.GetErrno(); >> ---------------^^^^^^^^^^^^^^^^^ >> BEGIN >> M3toC.FreeSharedS(p, f); >> OSErrorPosix.Raise0(err); >> END Fail; >> >> >> >> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>> runaway recursion trying to report a fault. This happens immediately upon >>> running it, before it produces any output. >>> >>> Here is one cycle of the backtrace: >>> >>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>> at ../src/runtime/common/RTHooks.m3:79 >>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> >>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 23:50:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 14:50:42 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D898C4.90305@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> <55D75408.3010900@lcwb.coop> <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> <55D898C4.90305@lcwb.coop> Message-ID: <2A7A1326-A801-4BFF-8C1D-4E50DB646272@gmail.com> Go back to one that is statically linked and upgrade from that? upgrade.py nocleangcc doesn't take too long. The backup must have, in general, cm3 cm3cg config libm3 m3core. We can relax the libm3 requirement I believe, but let's deal with that later separately. - Jay On Aug 22, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > Going back several cm3 backups until I see a change, I have: > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 > linux-vdso.so.1 => (0x00007fff211fe000) > libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) > libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) > libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) > /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-07-31 18:44:16 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 > linux-vdso.so.1 => (0x00007fffa47a7000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) > /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-06-04 14:24:48 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > So I compiled the change to dynamic between June 4 and July 31. > > > On 08/22/2015 05:00 AM, Jay K wrote: >> It works for me. >> >> Can you go back to a working install, maybe 5.8.6 with that slight config patch, >> run upgrade.py, add see what you get? >> Send the build log maybe? >> >> >> jair:python jay$ otool -L /cm3/bin/cm3 >> /cm3/bin/cm3: >> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) >> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) >> >> jair:python jay$ /cm3/bin/cm3 -version >> >> Critical Mass Modula-3 version d5.10.0 >> last updated: 2015-05-21 >> compiled: 2015-08-22 09:57:19 >> configuration: /cm3/bin/cm3.cfg >> host: I386_DARWIN >> target: I386_DARWIN >> >> >> On most other systems, use ldd. >> On Windows use link /dump /imports or link /dump /dependents. >> >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: hosking at purdue.edu; rodney.m.bates at acm.org >> Date: Sat, 22 Aug 2015 09:28:01 +0000 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 is broken >> >> This was not meant to change and I haven't seen that it did change. >> Maybe my cleaning up of the config files. I'll check. >> >> >> I understand the risks well. I have a potential plan to change it. But that is another matter. >> >> You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. >> >> The wrong uprade sequence still breaks cm3. That is at it should be. >> >> >> Tangentially, here is my plan: >> The upgrade process must be "to the side and switch all at once", not "in place". >> It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all >> into the install in tight sequence (not atomically). The trick on NT would be to unmap the >> executable, and avoid returning to it -- some sort of "continuation passing style", >> possibly off into a little bit of C++, whose main task would be to move a few files >> and then exit. >> >> >> That'd still leave all the platforms to test on to see if a running executable can be replaced. >> esp. including AIX. >> >> >> Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this >> and just leave it as an idea/plan that I don't get to >> >> - Jay >> >> >> >> >> > Subject: Re: [M3devel] cm3 is broken >> > From: hosking at purdue.edu >> > Date: Sat, 22 Aug 2015 06:48:15 +1000 >> > CC: jay.krell at cornell.edu; m3devel at elegosoft.com >> > To: rodney.m.bates at acm.org >> > >> > That is a regression. >> > >> > Sent from my iPad >> > >> > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: >> > > >> > > >> > > >> > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: >> > >> cm3 should build with static link to m3core so as to allow backup as you describe. >> > >> Did this change recently? >> > > >> > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting >> > > a breakpoint at one of these places gives: >> > > >> > > >> > > (m3gdb) b RTExFrame.m3:175 >> > > No source file named RTExFrame.m3. >> > > Make breakpoint pending on future shared library load? (y or [n]) y >> > > Breakpoint 1 (RTExFrame.m3:175) pending. >> > > (m3gdb) run >> > > >> > > ........ >> > > >> > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. >> > > Pending breakpoint "RTExFrame.m3:175" resolved >> > > >> > > >> > > >> > >> >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> > >>> >> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> > >>> >> > >>> >> > >>> I often upgrade.py nocleangcc to speed it up. >> > >>> >> > >>> >> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> > >>> >> > >>> >> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> > >>> >> > >>> >> > >>> Do you have recourse? >> > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> > >>> >> > >>> - Jay >> > >>> >> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> > >>>> >> > >>>> >> > >>>> >> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: >> > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) >> > >>>> >> > >>>> upgrade.py does this: >> > >>>> >> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> > >>>> Traceback (most recent call last): >> > >>>> File "upgrade.py", line 4, in >> > >>>> import pylib >> > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> > >>>> if Target.startswith("NT386"): >> > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' >> > >>>> >> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no >> > >>>> changes to it, and it takes longer to compile than everything else combined. >> > >>>> >> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> > >>>> the now-broken cm3. >> > >>>> >> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose >> > >>>> it is in m3core, which I guess I need to add to my backup habit. >> > >>>> >> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> > >>>>> >> > >>>>> I run upgrade.py frequently and recommend it. >> > >>>>> >> > >>>>> - Jay >> > >>>>> >> > >>>>> >> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > >>>>>> From: rodney_bates at lcwb.coop >> > >>>>>> To: m3devel at elegosoft.com >> > >>>>>> Subject: [M3devel] cm3 is broken >> > >>>>>> >> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >>>>>> runaway recursion trying to report a fault. This happens immediately upon >> > >>>>>> running it, before it produces any output. >> > >>>>>> >> > >>>>>> Here is one cycle of the backtrace: >> > >>>>>> >> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >>>>>> at ../src/runtime/common/RTHooks.m3:79 >> > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> >> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Rodney Bates >> > >>>>>> rodney.m.bates at acm.org >> > >>>>>> _______________________________________________ >> > >>>>>> M3devel mailing list >> > >>>>>> M3devel at elegosoft.com >> > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >>>> >> > >>>> -- >> > >>>> Rodney Bates >> > >>>> rodney.m.bates at acm.org >> > >>> _______________________________________________ >> > >>> M3devel mailing list >> > >>> M3devel at elegosoft.com >> > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >> > >> >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 23:54:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 16:54:26 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop> Message-ID: <55D8EF92.20403@lcwb.coop> On 08/22/2015 12:26 PM, Jay K wrote: > It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > > > Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run independently. After a good lot of successful-looking output, I get: new source -> compiling LLGen.i3 new source -> compiling LLGen.m3 new source -> compiling Version.i3 new source -> compiling M3Backend.i3 new source -> compiling Arg.i3 new source -> compiling Utils.i3 new source -> compiling Msg.i3 new source -> compiling M3Backend.m3 new source -> compiling UtilsPosix.m3 new source -> compiling Arg.m3 new source -> compiling M3Loc.i3 new source -> compiling M3Unit.i3 new source -> compiling Builder.i3 new source -> compiling M3Options.i3 new source -> compiling WebFile.i3 new source -> compiling Dirs.i3 new source -> compiling Builder.m3 new source -> compiling Dirs.m3 new source -> compiling M3Build.i3 new source -> compiling M3Build.m3 new source -> compiling M3Loc.m3 new source -> compiling M3Options.m3 new source -> compiling M3Unit.m3 new source -> compiling Makefile.i3 new source -> compiling Makefile.m3 new source -> compiling Msg.m3 new source -> compiling Utils.m3 new source -> compiling WebFile.m3 new source -> compiling Main.m3 "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure 1 warning encountered new source -> compiling cm3unix.c new exporters -> recompiling Utils.i3 -> linking cm3 --- shipping from AMD64_LINUX --- . => /usr/local/cm3/pkg/cm3/AMD64_LINUX .M3EXPORTS .M3WEB ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX cm3unix.c Version.i3 ../src => /usr/local/cm3/pkg/cm3/src M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 Arg.m3 Builder.m3 Builder.i3 Dirs.i3 Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 Msg.m3 Utils.m3 Utils.i3 WebFile.i3 WebFile.m3 Main.m3 ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy LLGen.m3 LLGen.i3 . => /usr/local/cm3/bin cm3 . => /usr/local/cm3/man/man1 cm3.1 . => /usr/local/cm3/man/man7 m3makefile.7 Segmentation fault (core dumped) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done ------------------------------------------------------------------------------------- several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: ------------------------------------------------------------------------------------- /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ *** execution of [, ] failed *** > The interface here between cm3's generated code/data and m3core has changed. > You cannot mix them. It will fail. > > > - Jay > > > > > > Date: Sat, 22 Aug 2015 11:41:14 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] cm3 is broken: More info > > > > > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > > VAR p := LOOPHOLE (f, PF1); > > > BEGIN > > > IF DEBUG THEN > > > PutExcept ("INVOKE HANDLER", a); > > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > > RTIO.PutText ("\n"); > > > RTIO.Flush (); > > > END; > > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > > p.info := a; (* copy the exception to the new frame *) > > > <* ASSERT p.jmpbuf # NIL *> > > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > > RAISE OUCH; > > > END InvokeHandler; > > > > > > (m3gdb) frame 0 > > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > > 175 <* ASSERT p.jmpbuf # NIL *> > > > (m3gdb) p p.jmpbuf > > > $31 = NIL > > > --------^^^ > > > (m3gdb) > > > > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > > BEGIN > > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > > M3toC.FreeSharedS(pn, fname); > > > RETURN status > > > END Status; > > > > > > (m3gdb) > > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > > at ../src/os/POSIX/FSPosix.m3:328 > > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > (m3gdb) p pn > > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > > --------------------------------^^^^^^^^^^^ > > > (m3gdb) > > > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > > > A bit more info: > > > > (m3gdb) down > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p Process.GetWorkingDirectory() > > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > (m3gdb) > > > > This is where I ran cm3 from. > > > > > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > > As I recall from many years ago, there was a change in C library that provided a thread > > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > > adapt to it. Do we care? > > > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > > VAR err := Cerrno.GetErrno(); > > > ---------------^^^^^^^^^^^^^^^^^ > > > BEGIN > > > M3toC.FreeSharedS(p, f); > > > OSErrorPosix.Raise0(err); > > > END Fail; > > > > > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >> runaway recursion trying to report a fault. This happens immediately upon > > >> running it, before it produces any output. > > >> > > >> Here is one cycle of the backtrace: > > >> > > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >> at ../src/runtime/common/RTHooks.m3:79 > > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> > > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >> > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 23 01:37:19 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 16:37:19 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8EF92.20403@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> Message-ID: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. I upgraded multiple installations on multiple targets. I'll try again from 5.8.6. I suggest upgrade also check for the bad dynamic linking. Anyone else able/unable to upgrade? - Jay On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > On 08/22/2015 12:26 PM, Jay K wrote: >> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >> >> >> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install > directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > independently. > > After a good lot of successful-looking output, I get: > > new source -> compiling LLGen.i3 > new source -> compiling LLGen.m3 > new source -> compiling Version.i3 > new source -> compiling M3Backend.i3 > new source -> compiling Arg.i3 > new source -> compiling Utils.i3 > new source -> compiling Msg.i3 > new source -> compiling M3Backend.m3 > new source -> compiling UtilsPosix.m3 > new source -> compiling Arg.m3 > new source -> compiling M3Loc.i3 > new source -> compiling M3Unit.i3 > new source -> compiling Builder.i3 > new source -> compiling M3Options.i3 > new source -> compiling WebFile.i3 > new source -> compiling Dirs.i3 > new source -> compiling Builder.m3 > new source -> compiling Dirs.m3 > new source -> compiling M3Build.i3 > new source -> compiling M3Build.m3 > new source -> compiling M3Loc.m3 > new source -> compiling M3Options.m3 > new source -> compiling M3Unit.m3 > new source -> compiling Makefile.i3 > new source -> compiling Makefile.m3 > new source -> compiling Msg.m3 > new source -> compiling Utils.m3 > new source -> compiling WebFile.m3 > new source -> compiling Main.m3 > "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > 1 warning encountered > new source -> compiling cm3unix.c > new exporters -> recompiling Utils.i3 > -> linking cm3 > --- shipping from AMD64_LINUX --- > > . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > .M3EXPORTS .M3WEB > ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > cm3unix.c Version.i3 > ../src => /usr/local/cm3/pkg/cm3/src > M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > WebFile.m3 Main.m3 > ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > LLGen.m3 LLGen.i3 > . => /usr/local/cm3/bin > cm3 > . => /usr/local/cm3/man/man1 > cm3.1 > . => /usr/local/cm3/man/man7 > m3makefile.7 > Segmentation fault (core dumped) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > > > ------------------------------------------------------------------------------------- > several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > ------------------------------------------------------------------------------------- > > > /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > *** execution of [, ] failed *** > > > > > > > > > >> The interface here between cm3's generated code/data and m3core has changed. >> You cannot mix them. It will fail. >> >> >> - Jay >> >> >> >> >> > Date: Sat, 22 Aug 2015 11:41:14 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: Re: [M3devel] cm3 is broken: More info >> > >> > >> > >> > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> > > >> > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> > > VAR p := LOOPHOLE (f, PF1); >> > > BEGIN >> > > IF DEBUG THEN >> > > PutExcept ("INVOKE HANDLER", a); >> > > RTIO.PutText (" frame="); RTIO.PutAddr (f); >> > > RTIO.PutText (" class="); RTIO.PutInt (f.class); >> > > RTIO.PutText ("\n"); >> > > RTIO.Flush (); >> > > END; >> > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> > > p.info := a; (* copy the exception to the new frame *) >> > > <* ASSERT p.jmpbuf # NIL *> >> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> > > RAISE OUCH; >> > > END InvokeHandler; >> > > >> > > (m3gdb) frame 0 >> > > #0 InvokeHandler (f=16_00007fff509f4af0, a= >> > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > > 175 <* ASSERT p.jmpbuf # NIL *> >> > > (m3gdb) p p.jmpbuf >> > > $31 = NIL >> > > --------^^^ >> > > (m3gdb) >> > > >> > > >> > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> > > >> > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> > > BEGIN >> > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > ---------^^^^^^^^^^^^^^^^^^^^^^ >> > > M3toC.FreeSharedS(pn, fname); >> > > RETURN status >> > > END Status; >> > > >> > > (m3gdb) >> > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > > at ../src/os/POSIX/FSPosix.m3:328 >> > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > (m3gdb) p pn >> > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> > > --------------------------------^^^^^^^^^^^ >> > > (m3gdb) >> > > >> > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >> > >> > A bit more info: >> > >> > (m3gdb) down >> > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > at ../src/os/POSIX/FSPosix.m3:328 >> > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > (m3gdb) p Process.GetWorkingDirectory() >> > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >> > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > (m3gdb) >> > >> > This is where I ran cm3 from. >> > >> > >> > > >> > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> > > after the failing CStatus above. Obviously not thread safe. Does it matter here? >> > > As I recall from many years ago, there was a change in C library that provided a thread >> > > safe alternative to this, and I even think I had to change something in pm3 or SRC to >> > > adapt to it. Do we care? >> > > >> > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> > > VAR err := Cerrno.GetErrno(); >> > > ---------------^^^^^^^^^^^^^^^^^ >> > > BEGIN >> > > M3toC.FreeSharedS(p, f); >> > > OSErrorPosix.Raise0(err); >> > > END Fail; >> > > >> > > >> > > >> > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >> runaway recursion trying to report a fault. This happens immediately upon >> > >> running it, before it produces any output. >> > >> >> > >> Here is one cycle of the backtrace: >> > >> >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >> at ../src/runtime/common/RTHooks.m3:79 >> > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 24 19:41:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 24 Aug 2015 17:41:51 +0000 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: <20150821144644.345ac1a25b429e6c35854d32@elego.de> References: , <20150821144644.345ac1a25b429e6c35854d32@elego.de> Message-ID: It is a bug and there is a workaround. See: https://connect.microsoft.com/VisualStudio/feedback/details/1651822/incorrect-mspdb140-dll-version-picked-in-x86-x64-cross-tools-environment I can probably workaround it in pylib.py also. - Jay Date: Fri, 21 Aug 2015 14:46:44 +0200 From: wagner at elego.de To: m3devel at elegosoft.com CC: sevettel at hotmail.de Subject: Re: [M3devel] critical mass 3 auf window10 On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 04:08:09 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 24 Aug 2015 21:08:09 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Message-ID: <55DBCE09.3020005@lcwb.coop> Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field jmpbuf of an exception frame being NIL. The only place I find that this field could be set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, in procedure CaptureState. From a version without this rework, it will require compiling the compiler twice, once to get a compiler that would generate this code, and again to get a compiler that executes it when run. You are saying the middle compiler legitimately raises and catches an exception, in order to run, so we can't get past this step. And it could do that in other places too. I am guessing you have re-bootstrapped your compiler through more than one intermediate state of source code, so you don't get this problem. But it has to work somehow from the original version, for everybody else. Will it work with the new compiler-generated code but old runtime? Seems like a long shot. On 08/22/2015 06:37 PM, Jay wrote: > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > I upgraded multiple installations on multiple targets. > I'll try again from 5.8.6. > > > I suggest upgrade also check for the bad dynamic linking. > > > Anyone else able/unable to upgrade? > > - Jay > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > >> >> >> On 08/22/2015 12:26 PM, Jay K wrote: >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >>> >>> >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) >> >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run >> independently. >> >> After a good lot of successful-looking output, I get: >> >> new source -> compiling LLGen.i3 >> new source -> compiling LLGen.m3 >> new source -> compiling Version.i3 >> new source -> compiling M3Backend.i3 >> new source -> compiling Arg.i3 >> new source -> compiling Utils.i3 >> new source -> compiling Msg.i3 >> new source -> compiling M3Backend.m3 >> new source -> compiling UtilsPosix.m3 >> new source -> compiling Arg.m3 >> new source -> compiling M3Loc.i3 >> new source -> compiling M3Unit.i3 >> new source -> compiling Builder.i3 >> new source -> compiling M3Options.i3 >> new source -> compiling WebFile.i3 >> new source -> compiling Dirs.i3 >> new source -> compiling Builder.m3 >> new source -> compiling Dirs.m3 >> new source -> compiling M3Build.i3 >> new source -> compiling M3Build.m3 >> new source -> compiling M3Loc.m3 >> new source -> compiling M3Options.m3 >> new source -> compiling M3Unit.m3 >> new source -> compiling Makefile.i3 >> new source -> compiling Makefile.m3 >> new source -> compiling Msg.m3 >> new source -> compiling Utils.m3 >> new source -> compiling WebFile.m3 >> new source -> compiling Main.m3 >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure >> 1 warning encountered >> new source -> compiling cm3unix.c >> new exporters -> recompiling Utils.i3 >> -> linking cm3 >> --- shipping from AMD64_LINUX --- >> >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> .M3EXPORTS .M3WEB >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> cm3unix.c Version.i3 >> ../src => /usr/local/cm3/pkg/cm3/src >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 >> WebFile.m3 Main.m3 >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy >> LLGen.m3 LLGen.i3 >> . => /usr/local/cm3/bin >> cm3 >> . => /usr/local/cm3/man/man1 >> cm3.1 >> . => /usr/local/cm3/man/man7 >> m3makefile.7 >> Segmentation fault (core dumped) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done >> >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done >> >> >> ------------------------------------------------------------------------------------- >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: >> ------------------------------------------------------------------------------------- >> >> >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> *** execution of [, ] failed *** >> >> >> >> >> >> >> >> >> >>> The interface here between cm3's generated code/data and m3core has changed. >>> You cannot mix them. It will fail. >>> >>> >>> - Jay >>> >>> >>> >>> >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu >>>> Subject: Re: [M3devel] cm3 is broken: More info >>>> >>>> >>>> >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >>>>> >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >>>>> VAR p := LOOPHOLE (f, PF1); >>>>> BEGIN >>>>> IF DEBUG THEN >>>>> PutExcept ("INVOKE HANDLER", a); >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); >>>>> RTIO.PutText ("\n"); >>>>> RTIO.Flush (); >>>>> END; >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >>>>> p.info := a; (* copy the exception to the new frame *) >>>>> <* ASSERT p.jmpbuf # NIL *> >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >>>>> RAISE OUCH; >>>>> END InvokeHandler; >>>>> >>>>> (m3gdb) frame 0 >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> 175 <* ASSERT p.jmpbuf # NIL *> >>>>> (m3gdb) p p.jmpbuf >>>>> $31 = NIL >>>>> --------^^^ >>>>> (m3gdb) >>>>> >>>>> >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >>>>> >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >>>>> BEGIN >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ >>>>> M3toC.FreeSharedS(pn, fname); >>>>> RETURN status >>>>> END Status; >>>>> >>>>> (m3gdb) >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>>> at ../src/os/POSIX/FSPosix.m3:328 >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> (m3gdb) p pn >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >>>>> --------------------------------^^^^^^^^^^^ >>>>> (m3gdb) >>>>> >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >>>> >>>> A bit more info: >>>> >>>> (m3gdb) down >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>> at ../src/os/POSIX/FSPosix.m3:328 >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>> (m3gdb) p Process.GetWorkingDirectory() >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> (m3gdb) >>>> >>>> This is where I ran cm3 from. >>>> >>>> >>>>> >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? >>>>> As I recall from many years ago, there was a change in C library that provided a thread >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to >>>>> adapt to it. Do we care? >>>>> >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >>>>> VAR err := Cerrno.GetErrno(); >>>>> ---------------^^^^^^^^^^^^^^^^^ >>>>> BEGIN >>>>> M3toC.FreeSharedS(p, f); >>>>> OSErrorPosix.Raise0(err); >>>>> END Fail; >>>>> >>>>> >>>>> >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>>> running it, before it produces any output. >>>>>> >>>>>> Here is one cycle of the backtrace: >>>>>> >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>>> >>>>> >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 25 06:12:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55DBCE09.3020005@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> ,<55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop> Message-ID: Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 25 11:56:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 09:56:36 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: I should have said you are close, it makes mostly sense, but not entirely. Anyway.. upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: % cm3 -versionCritical Mass Modula-3 version 5.8.6 last updated: 2010-04-11 compiled: 2010-07-12 20:10:34 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX to % cm3 -versionCritical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 09:33:26 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX in "one" step (automating many steps) Here is from scripts/python/capture-boot.py: https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz Do you have any local edits? Removed stuff or reordered in pkginfo.txt? (I edited one line in 5.8.6's config, to stop looking for m3cc). - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: Re: [M3devel] cm3 is broken: More info Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 22:08:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 25 Aug 2015 15:08:41 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: <55DCCB49.80103@lcwb.coop> OK, I got a working compiler from 5.8.6, and it is now statically linked to all modula-3 libs. rodney at allegheny:~/proj/m3/exp/trytemp/src$ cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 19:02:01 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX rodney at allegheny:~/proj/m3/exp/trytemp/src$ ldd /usr/local/cm3/bin/cm3 linux-vdso.so.1 => (0x00007fff54bfe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8cca5b8000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8cca39a000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8cc9fd2000) /lib64/ld-linux-x86-64.so.2 (0x00007f8cca8de000) I did a fresh pull of modula3-cm3, and removed and re-checked out a couple of locally edited git-tracked files, so my working is as github. git diff HEAD shows only a prompt. Then I make a full backup of my 5.8.6 installation directory, switched to it (the original), went to scripts/python, and did ./upgrade-full.sh 2>&1 | tee upgrade-full.sh.log. I did not do anything to disable rebuilding of m3cc. I had some changes in the last pull, and lots since 5.8.6. Eventually the rebuild of everything failed in cvsup, which is probably entirely irrelevant. There are many clean steps that look like: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ rm -rf AMD64_LINUX +++ ==> [] returned 0 Are they cleaning or failing to clean? In selected cases, they are followed by build and ship steps that do nothing: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done +++ /usr/local/cm3/bin/cm3 -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done So I must have had some non-bootstrappable intermediate version of something? > On 08/25/2015 04:56 AM, Jay K wrote: > I should have said you are close, it makes mostly sense, but not entirely. Anyway.. > > > upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: > > > > % cm3 -version > Critical Mass Modula-3 version 5.8.6 > last updated: 2010-04-11 > compiled: 2010-07-12 20:10:34 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > to > > > % cm3 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-25 09:33:26 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > in "one" step (automating many steps) > > Here is from scripts/python/capture-boot.py: > https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz > > > Do you have any local edits? > > > Removed stuff or reordered in pkginfo.txt? > > > (I edited one line in 5.8.6's config, to stop looking for m3cc). > > - Jay > > > >-- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 02:28:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 00:28:48 +0000 Subject: [M3devel] Target.First_readable_addr Message-ID: Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 26 07:54:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 05:54:25 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> References: , <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> Message-ID: At the very least, can we use the same number for all targets, 4K? I don't like this inconsistency: IF RTParams.IsPresent("checked") THEN EVAL F4096x8p2.F1(NIL); (* large -- gets null checked *) END; IF RTParams.IsPresent("unchecked") THEN EVAL F0.F1(NIL); (* small -- does not get null check *) END; jair:p263 jay$ ./I386_DARWIN/pgm @M3checked ****** runtime error:*** Attempt to reference an illegal memory location.*** file "../F4096x8p2.m3", line 8*** Abort trap: 6jair:p263 jay$ ./I386_DARWIN/pgm @M3unchecked ****** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0xb8dc5 = F1 + 0x6 in ../F0.m3*** I wonder if we have even implemented this for Windows, the unchecked form.The inefficient checked form is easy.Notice how "large" gives the file/line of the null deref, "small" does not. What do Java, gcj, C#, mono, .NET Native, go, rust do? (Extra credit to anyone who can show meboth their compiler outputting the checksand their generated code in a debugger..For Modula-3/cm3cg, grep for fault in the *ms files for *.c files) - Jay Subject: Re: [M3devel] Target.First_readable_addr From: hosking at purdue.edu Date: Wed, 26 Aug 2015 14:44:30 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu This optimization has some performance benefit.Surely we can simply fix the problems rather than removing the capability. On Aug 26, 2015, at 10:28 AM, Jay K wrote:Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 26 17:37:54 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 10:37:54 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: References: Message-ID: <55DDDD52.5090008@lcwb.coop> I have always thought "segfault, could it be a NIL pointer deref?", or whatever words to that effect, was a rather lame excuse of an error message, and not much help. Since this scheme doesn't fault at the point of the real deref, but afterwards, when a memory reference is actually made, possibly computed from that address, could the location given for the fault (whether line no or just code displacement) be wrong? Very wrong? TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; PROCEDURE Q ( ) = VAR Ptr : REF A := NIL; BEGIN P(Ptr^); <---deref occurs here END Q; .... lots of other code, different module, etc. ... PROCEDURE P ( VAR x : A ) = BEGIN x[119] := 17 <---memory ref & segfault don't happen until here. END P; Or is this scheme only used in selective places, such as implicit deref when putting a subscript onto a pointer? e.g.: Ptr[117] := 43; On 08/25/2015 07:28 PM, Jay K wrote: > Of course, my agenda is to remove target-specificity. > > > > Target.First_readable_addr is a target-specific optimization > to omit null checks. Or maybe range checks.. > > > The assumption is that dereferencing "actual" null is checked > by the hardware, so code doesn't have to be generated to check for null. > > > I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > > > The assumption is further that accesses to the first page, at least, > are subject to the same hardware checks, and can also be optimized away. > > > At first, glance, I thought this was based on offset + size of a static access. > > > For example: > > a: ARRAY [0...large] OF CHAR > > > a[0..4095] no check > a[4096 and up] would check > > > Target.First_deadable_addr is 4k on many targets; 8k on others. > Setting it too low would yield extra checks. > Setting it too high would yield missing checks and a loss of safety. > > > Here is what I actually found though. > > > - The check is based on the size of the type being accessed. > > - It is off by a factor of 8 -- there is confusing between m3middle and m3front > as to if it is bit number of a byte number. > > > small: ARRAY[0..100] OF CHAR > large:ARRAY[0..100000] OF CHAR > > no access to small gets checked and every access to larger gets checked. > > Should we do anything about this? > > In m3-sys/m3tests/src/p2/p263: > cm3 -realclean > cm3 -keep > grep fault /*ms > > > All the accesses are to offset 0. > So, by some expectation, no null checks are needed. > Null checks are output when the size of the > containing type is, for x86, larger than 4096*8. > > > The checks have been largely but not completely wrong/missing. > Safety behooves us to check though. > > - fix the factor of 8? > - make it 0?? too slow? > - make it 4k on all target? until such time as a target manifests with a smaller page size? > - base the checks on access offset + size, not containing size? > Containing size is conservative. It checks more than i think is meant. > > > I couldn't actually figure out the code here, I added various: > > IF RTParams.IsPresent("qual") THEN > RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > RTIO.Flush(); > END; > > and such to m3front to figure it out. > > > Thanks, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 18:10:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 26 Aug 2015 09:10:36 -0700 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DDDD52.5090008@lcwb.coop> References: <55DDDD52.5090008@lcwb.coop> Message-ID: <08316510-3124-4372-A514-3D4881D26966@gmail.com> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. - Jay On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > I have always thought "segfault, could it be a NIL pointer deref?", or whatever > words to that effect, was a rather lame excuse of an error message, and not > much help. > > Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > memory reference is actually made, possibly computed from that address, could the location > given for the fault (whether line no or just code displacement) be wrong? Very wrong? > > TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > > PROCEDURE Q ( ) > = VAR Ptr : REF A := NIL; > BEGIN > P(Ptr^); <---deref occurs here > END Q; > > .... lots of other code, different module, etc. ... > > PROCEDURE P ( VAR x : A ) > = BEGIN > x[119] := 17 <---memory ref & segfault don't happen until here. > END P; > > Or is this scheme only used in selective places, such as implicit deref > when putting a subscript onto a pointer? e.g.: > > Ptr[117] := 43; > > On 08/25/2015 07:28 PM, Jay K wrote: >> Of course, my agenda is to remove target-specificity. >> >> >> >> Target.First_readable_addr is a target-specific optimization >> to omit null checks. Or maybe range checks.. >> >> >> The assumption is that dereferencing "actual" null is checked >> by the hardware, so code doesn't have to be generated to check for null. >> >> >> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >> >> >> The assumption is further that accesses to the first page, at least, >> are subject to the same hardware checks, and can also be optimized away. >> >> >> At first, glance, I thought this was based on offset + size of a static access. >> >> >> For example: >> >> a: ARRAY [0...large] OF CHAR >> >> >> a[0..4095] no check >> a[4096 and up] would check >> >> >> Target.First_deadable_addr is 4k on many targets; 8k on others. >> Setting it too low would yield extra checks. >> Setting it too high would yield missing checks and a loss of safety. >> >> >> Here is what I actually found though. >> >> >> - The check is based on the size of the type being accessed. >> >> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >> as to if it is bit number of a byte number. >> >> >> small: ARRAY[0..100] OF CHAR >> large:ARRAY[0..100000] OF CHAR >> >> no access to small gets checked and every access to larger gets checked. >> >> Should we do anything about this? >> >> In m3-sys/m3tests/src/p2/p263: >> cm3 -realclean >> cm3 -keep >> grep fault /*ms >> >> >> All the accesses are to offset 0. >> So, by some expectation, no null checks are needed. >> Null checks are output when the size of the >> containing type is, for x86, larger than 4096*8. >> >> >> The checks have been largely but not completely wrong/missing. >> Safety behooves us to check though. >> >> - fix the factor of 8? >> - make it 0?? too slow? >> - make it 4k on all target? until such time as a target manifests with a smaller page size? >> - base the checks on access offset + size, not containing size? >> Containing size is conservative. It checks more than i think is meant. >> >> >> I couldn't actually figure out the code here, I added various: >> >> IF RTParams.IsPresent("qual") THEN >> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >> RTIO.Flush(); >> END; >> >> and such to m3front to figure it out. >> >> >> Thanks, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 27 04:23:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 21:23:05 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <08316510-3124-4372-A514-3D4881D26966@gmail.com> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> Message-ID: <55DE7489.9000609@lcwb.coop> On 08/26/2015 11:10 AM, Jay wrote: > Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > No, I disagree adamantly. The whole idea of linguistic support of parameters passed by reference, instead of the programmer manually concocting it by taking addresses and contents of pointers, is that the language ensures to the called procedure that the hidden pointer will be neither NIL, uninitialized, nor point to something of the wrong type. In fact, pass by ref is not even defined in terms of the compiler lowering into pointer twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the type system that should only happen in UNSAFE code. NIL deref, as a runtime error, should in concept happen when and where ^ is applied to a pointer, including the implied dereference when a subscript or field selection is applied directly without the ^ operator. It's purely an implementation question how to do this. If a segfault message will point to the line of code where the dereference happens, and before execution proceeds beyond that code, that would be marginally OK. (Only marginally, because the error message should be able to say with certainty that it's a NIL deref. Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his obligation, its effects can escape into safe code. > > - Jay > > On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >> words to that effect, was a rather lame excuse of an error message, and not >> much help. >> >> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >> memory reference is actually made, possibly computed from that address, could the location >> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >> >> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >> >> PROCEDURE Q ( ) >> = VAR Ptr : REF A := NIL; >> BEGIN >> P(Ptr^); <---deref occurs here >> END Q; >> >> .... lots of other code, different module, etc. ... >> >> PROCEDURE P ( VAR x : A ) >> = BEGIN >> x[119] := 17 <---memory ref & segfault don't happen until here. >> END P; >> >> Or is this scheme only used in selective places, such as implicit deref >> when putting a subscript onto a pointer? e.g.: >> >> Ptr[117] := 43; >> >> On 08/25/2015 07:28 PM, Jay K wrote: >>> Of course, my agenda is to remove target-specificity. >>> >>> >>> >>> Target.First_readable_addr is a target-specific optimization >>> to omit null checks. Or maybe range checks.. >>> >>> >>> The assumption is that dereferencing "actual" null is checked >>> by the hardware, so code doesn't have to be generated to check for null. >>> >>> >>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>> >>> >>> The assumption is further that accesses to the first page, at least, >>> are subject to the same hardware checks, and can also be optimized away. >>> >>> >>> At first, glance, I thought this was based on offset + size of a static access. >>> >>> >>> For example: >>> >>> a: ARRAY [0...large] OF CHAR >>> >>> >>> a[0..4095] no check >>> a[4096 and up] would check >>> >>> >>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>> Setting it too low would yield extra checks. >>> Setting it too high would yield missing checks and a loss of safety. >>> >>> >>> Here is what I actually found though. >>> >>> >>> - The check is based on the size of the type being accessed. >>> >>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>> as to if it is bit number of a byte number. >>> >>> >>> small: ARRAY[0..100] OF CHAR >>> large:ARRAY[0..100000] OF CHAR >>> >>> no access to small gets checked and every access to larger gets checked. >>> >>> Should we do anything about this? >>> >>> In m3-sys/m3tests/src/p2/p263: >>> cm3 -realclean >>> cm3 -keep >>> grep fault /*ms >>> >>> >>> All the accesses are to offset 0. >>> So, by some expectation, no null checks are needed. >>> Null checks are output when the size of the >>> containing type is, for x86, larger than 4096*8. >>> >>> >>> The checks have been largely but not completely wrong/missing. >>> Safety behooves us to check though. >>> >>> - fix the factor of 8? >>> - make it 0?? too slow? >>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>> - base the checks on access offset + size, not containing size? >>> Containing size is conservative. It checks more than i think is meant. >>> >>> >>> I couldn't actually figure out the code here, I added various: >>> >>> IF RTParams.IsPresent("qual") THEN >>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>> RTIO.Flush(); >>> END; >>> >>> and such to m3front to figure it out. >>> >>> >>> Thanks, >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Aug 28 00:20:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 27 Aug 2015 17:20:58 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> <55DE7489.9000609@lcwb.coop> <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> Message-ID: <55DF8D4A.5050707@lcwb.coop> In any case, I would shed no tears if First_readable_addr were set target-indepenently to the lowest value on any target, since that would only increase the number of cases that give the proper error at the proper place & time. On 08/26/2015 10:07 PM, Antony Hosking wrote: > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: >> >> >> >> On 08/26/2015 11:10 AM, Jay wrote: >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. >>> >> >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by >> reference, instead of the programmer manually concocting it by taking addresses and >> contents of pointers, is that the language ensures to the called procedure that the >> hidden pointer will be neither NIL, uninitialized, nor point to something of the >> wrong type. >> >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the >> type system that should only happen in UNSAFE code. >> >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied >> to a pointer, including the implied dereference when a subscript or field selection is >> applied directly without the ^ operator. It's purely an implementation question how to >> do this. If a segfault message will point to the line of code where the dereference >> happens, and before execution proceeds beyond that code, that would be marginally OK. >> (Only marginally, because the error message should be able to say with certainty that >> it's a NIL deref. >> >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his >> obligation, its effects can escape into safe code. >> >>> >>> - Jay >>> >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: >>> >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >>>> words to that effect, was a rather lame excuse of an error message, and not >>>> much help. >>>> >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >>>> memory reference is actually made, possibly computed from that address, could the location >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >>>> >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >>>> >>>> PROCEDURE Q ( ) >>>> = VAR Ptr : REF A := NIL; >>>> BEGIN >>>> P(Ptr^); <---deref occurs here >>>> END Q; >>>> >>>> .... lots of other code, different module, etc. ... >>>> >>>> PROCEDURE P ( VAR x : A ) >>>> = BEGIN >>>> x[119] := 17 <---memory ref & segfault don't happen until here. >>>> END P; >>>> >>>> Or is this scheme only used in selective places, such as implicit deref >>>> when putting a subscript onto a pointer? e.g.: >>>> >>>> Ptr[117] := 43; >>>> >>>> On 08/25/2015 07:28 PM, Jay K wrote: >>>>> Of course, my agenda is to remove target-specificity. >>>>> >>>>> >>>>> >>>>> Target.First_readable_addr is a target-specific optimization >>>>> to omit null checks. Or maybe range checks.. >>>>> >>>>> >>>>> The assumption is that dereferencing "actual" null is checked >>>>> by the hardware, so code doesn't have to be generated to check for null. >>>>> >>>>> >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>>>> >>>>> >>>>> The assumption is further that accesses to the first page, at least, >>>>> are subject to the same hardware checks, and can also be optimized away. >>>>> >>>>> >>>>> At first, glance, I thought this was based on offset + size of a static access. >>>>> >>>>> >>>>> For example: >>>>> >>>>> a: ARRAY [0...large] OF CHAR >>>>> >>>>> >>>>> a[0..4095] no check >>>>> a[4096 and up] would check >>>>> >>>>> >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>>>> Setting it too low would yield extra checks. >>>>> Setting it too high would yield missing checks and a loss of safety. >>>>> >>>>> >>>>> Here is what I actually found though. >>>>> >>>>> >>>>> - The check is based on the size of the type being accessed. >>>>> >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>>>> as to if it is bit number of a byte number. >>>>> >>>>> >>>>> small: ARRAY[0..100] OF CHAR >>>>> large:ARRAY[0..100000] OF CHAR >>>>> >>>>> no access to small gets checked and every access to larger gets checked. >>>>> >>>>> Should we do anything about this? >>>>> >>>>> In m3-sys/m3tests/src/p2/p263: >>>>> cm3 -realclean >>>>> cm3 -keep >>>>> grep fault /*ms >>>>> >>>>> >>>>> All the accesses are to offset 0. >>>>> So, by some expectation, no null checks are needed. >>>>> Null checks are output when the size of the >>>>> containing type is, for x86, larger than 4096*8. >>>>> >>>>> >>>>> The checks have been largely but not completely wrong/missing. >>>>> Safety behooves us to check though. >>>>> >>>>> - fix the factor of 8? >>>>> - make it 0?? too slow? >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>>>> - base the checks on access offset + size, not containing size? >>>>> Containing size is conservative. It checks more than i think is meant. >>>>> >>>>> >>>>> I couldn't actually figure out the code here, I added various: >>>>> >>>>> IF RTParams.IsPresent("qual") THEN >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>>>> RTIO.Flush(); >>>>> END; >>>>> >>>>> and such to m3front to figure it out. >>>>> >>>>> >>>>> Thanks, >>>>> - Jay >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Aug 28 02:32:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:32:22 +0000 Subject: [M3devel] github history In-Reply-To: References: <73FE1A29-7F80-46DB-8B3C-1CA168C6FC92@purdue.edu>, , Message-ID: I don't really know what I'm doing with git. And it seems dangerous. I do know what I'm doing with Perforce, but oh well. I am a big fan of preserving history, so if I lost any, you can assume it was an accident. (That said, "local history" on my machine maybe isn't so valuable, noisy.) I really don't understand git merging. I tried using stash when there was a conflict, and that didn't go well. I don't know how to have multiple "checkouts" w/o multiple repositories. Wasteful. The web suggests some ways, but they didn't work for me -- e.g. cloning a local; I couldn't push from that. I gave up and created another large local repository. At least it was easy and fast enough. I don't want little temporary branches for this -- I don't want my one working directory to bounce between branches, I want multiple independent working directories against the same branch. - Jay Subject: Re: github history From: hosking at purdue.edu Date: Thu, 27 Aug 2015 14:54:57 +1000 CC: m3devel at elegosoft.com; jay.krell at cornell.edu; dragisha at m3w.org To: hosking at purdue.edu Update: I have successfully brought back history by: 1. setting the default branch to our old trunk,2. merging from master,3. deleting master,4. branching a new master from the (up-to-date) trunk, and5. setting the default branch back to master. On Aug 27, 2015, at 1:46 PM, Antony Hosking wrote:PS Jay, in poking around on github it seems that the source of this weirdness is a merge you did earlier this year. I suspect you were not using proper github workflow and it messed up the history (perhaps you inverted the merge?). Does anyone out there have better github fu than I do, who can dig into this and try to repair? Dragisha? On Aug 27, 2015, at 1:14 PM, Antony Hosking wrote:I am dismayed to find that the full commit histories on github do not appear to have made it over from the transition from svn to github. For example, I find the following very abbreviated history in the master tree for NilChkExpr.m3 in package m3front (see below). Did something strange happen in a merge somewhere along the way from the initial import into github? It looks like it occurred at the time of the commit "jaykrell on Jan 4, 2008 initial diff from ../config/PPC_LINUX" when it looks like the master got clobbered somehow. Anyone have any ideas if we can recover? Commits on May 4, 2008<1635728.png>add more information printed for some assertion failures .jaykrell authored on May 4, 200866139d4 Commits on Jan 4, 2008<1635728.png>initial diff from ../config/PPC_LINUXjaykrell authored on Jan 4, 20082412437 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 28 02:40:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:40:32 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DF8D4A.5050707@lcwb.coop> References: , <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com>, <55DE7489.9000609@lcwb.coop>, <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu>, <55DF8D4A.5050707@lcwb.coop> Message-ID: That is done -- it is a constant 4k now. Hypothetically Sparc/IA64/Alpha could be 8k. Sparc used to be. I'm still leary of the omitted checks and suggest we compare against C# (JIT and native, Microsoft and Mono), Java, Go, Rust, Gnat, etc. If most/all of them check aggressively, we should to imho. If they actually omit more aggressively, which I believe is possible, we should consider that. In particular, I think the omission should be based on the offset+size of the access, not of the containing type. i.e. the field and not the record. However this might be a fallacy also. e.g. TYPE BigRecord = RECORD ...lots.... INTEGER smallFieldPast4K END; PROCEDURE F1(VAR a:INTEGER); PROCEDURE F2(b:REF BigRecord) BEGIN F1(b.a); END F2; Hm. I see. Maybe you have to check at b.a and a (not shown here)? The recipient of a "small" type doesn't know its offset, if any, within a larger type, and it can vary. I can have: TYPE SmallRecord = RECORD .. nothing .. INTEGER smallFieldAtStart END; F1(smallRecord.a); So maybe you are right anyway -- check needed at the "address generation" not just the deref. I need to study this more. Another idea, perhaps, is that if there is any dereference of a parameter within a function, do all the checks up front, and then none in the function. You know -- in case there are loops. But I also don't know the guarantees around "dynamic paths" -- turning conditional dereferences into unconditional dereferences. I need to study this more. :) - Jay > Date: Thu, 27 Aug 2015 17:20:58 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] Target.First_readable_addr > > In any case, I would shed no tears if First_readable_addr were set target-indepenently > to the lowest value on any target, since that would only increase the number of cases > that give the proper error at the proper place & time. > > On 08/26/2015 10:07 PM, Antony Hosking wrote: > > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > > > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: > >> > >> > >> > >> On 08/26/2015 11:10 AM, Jay wrote: > >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > >>> > >> > >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by > >> reference, instead of the programmer manually concocting it by taking addresses and > >> contents of pointers, is that the language ensures to the called procedure that the > >> hidden pointer will be neither NIL, uninitialized, nor point to something of the > >> wrong type. > >> > >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer > >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual > >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. > >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the > >> type system that should only happen in UNSAFE code. > >> > >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied > >> to a pointer, including the implied dereference when a subscript or field selection is > >> applied directly without the ^ operator. It's purely an implementation question how to > >> do this. If a segfault message will point to the line of code where the dereference > >> happens, and before execution proceeds beyond that code, that would be marginally OK. > >> (Only marginally, because the error message should be able to say with certainty that > >> it's a NIL deref. > >> > >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault > >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his > >> obligation, its effects can escape into safe code. > >> > >>> > >>> - Jay > >>> > >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >>> > >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever > >>>> words to that effect, was a rather lame excuse of an error message, and not > >>>> much help. > >>>> > >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > >>>> memory reference is actually made, possibly computed from that address, could the location > >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? > >>>> > >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > >>>> > >>>> PROCEDURE Q ( ) > >>>> = VAR Ptr : REF A := NIL; > >>>> BEGIN > >>>> P(Ptr^); <---deref occurs here > >>>> END Q; > >>>> > >>>> .... lots of other code, different module, etc. ... > >>>> > >>>> PROCEDURE P ( VAR x : A ) > >>>> = BEGIN > >>>> x[119] := 17 <---memory ref & segfault don't happen until here. > >>>> END P; > >>>> > >>>> Or is this scheme only used in selective places, such as implicit deref > >>>> when putting a subscript onto a pointer? e.g.: > >>>> > >>>> Ptr[117] := 43; > >>>> > >>>> On 08/25/2015 07:28 PM, Jay K wrote: > >>>>> Of course, my agenda is to remove target-specificity. > >>>>> > >>>>> > >>>>> > >>>>> Target.First_readable_addr is a target-specific optimization > >>>>> to omit null checks. Or maybe range checks.. > >>>>> > >>>>> > >>>>> The assumption is that dereferencing "actual" null is checked > >>>>> by the hardware, so code doesn't have to be generated to check for null. > >>>>> > >>>>> > >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > >>>>> > >>>>> > >>>>> The assumption is further that accesses to the first page, at least, > >>>>> are subject to the same hardware checks, and can also be optimized away. > >>>>> > >>>>> > >>>>> At first, glance, I thought this was based on offset + size of a static access. > >>>>> > >>>>> > >>>>> For example: > >>>>> > >>>>> a: ARRAY [0...large] OF CHAR > >>>>> > >>>>> > >>>>> a[0..4095] no check > >>>>> a[4096 and up] would check > >>>>> > >>>>> > >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. > >>>>> Setting it too low would yield extra checks. > >>>>> Setting it too high would yield missing checks and a loss of safety. > >>>>> > >>>>> > >>>>> Here is what I actually found though. > >>>>> > >>>>> > >>>>> - The check is based on the size of the type being accessed. > >>>>> > >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front > >>>>> as to if it is bit number of a byte number. > >>>>> > >>>>> > >>>>> small: ARRAY[0..100] OF CHAR > >>>>> large:ARRAY[0..100000] OF CHAR > >>>>> > >>>>> no access to small gets checked and every access to larger gets checked. > >>>>> > >>>>> Should we do anything about this? > >>>>> > >>>>> In m3-sys/m3tests/src/p2/p263: > >>>>> cm3 -realclean > >>>>> cm3 -keep > >>>>> grep fault /*ms > >>>>> > >>>>> > >>>>> All the accesses are to offset 0. > >>>>> So, by some expectation, no null checks are needed. > >>>>> Null checks are output when the size of the > >>>>> containing type is, for x86, larger than 4096*8. > >>>>> > >>>>> > >>>>> The checks have been largely but not completely wrong/missing. > >>>>> Safety behooves us to check though. > >>>>> > >>>>> - fix the factor of 8? > >>>>> - make it 0?? too slow? > >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? > >>>>> - base the checks on access offset + size, not containing size? > >>>>> Containing size is conservative. It checks more than i think is meant. > >>>>> > >>>>> > >>>>> I couldn't actually figure out the code here, I added various: > >>>>> > >>>>> IF RTParams.IsPresent("qual") THEN > >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > >>>>> RTIO.Flush(); > >>>>> END; > >>>>> > >>>>> and such to m3front to figure it out. > >>>>> > >>>>> > >>>>> Thanks, > >>>>> - Jay > >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> M3devel mailing list > >>>>> M3devel at elegosoft.com > >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Aug 28 21:45:16 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 28 Aug 2015 14:45:16 -0500 Subject: [M3devel] llvm backend integration Message-ID: <55E0BA4C.7060802@lcwb.coop> The llvm back end now has an initial degree of integration into cm3. To get it, add M3_BACKEND_MODE="StAloneLlvmAsm" near the front of the m3makefile, or elsewhere in quake code that will get interpreted. It does the following: Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm bitcode IR, in file *.mb or *.ib. Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. Runs 'as' to assemble to object code, in file *.mo or *.io. Prerequisites: llc on the path. m3llvm on the path. as on the path. In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, see the m3makefile of llvm, which must give their locations. 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR and llvm IR are created, but you can get human-readable versions of them with 'm3cgcat' and 'llvm-dis'. Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line parameters to m3llvm. (Other targets are not supported). -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:17:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:17:05 -0500 Subject: [M3devel] llvm backend integration In-Reply-To: <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> References: <55E0BA4C.7060802@lcwb.coop> <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> Message-ID: <55E1F721.1090106@lcwb.coop> On 08/28/2015 07:08 PM, Antony Hosking wrote: > Sounds great! What's missing? > No provision for starting from an intermediate file format, as there is for m3cc. Also, it might be nice to get the human-readable forms of the two IRs as side outputs, for development work. Just a convenience, though. And, it just feels like Builder.m3 could use a good code review in general. > Sent from my iPad > >> On 29 Aug 2015, at 5:45 am, Rodney M. Bates wrote: >> >> The llvm back end now has an initial degree of integration into cm3. To get it, add >> >> M3_BACKEND_MODE="StAloneLlvmAsm" >> >> near the front of the m3makefile, or elsewhere in quake code that will get interpreted. >> >> It does the following: >> >> Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. >> >> Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm >> bitcode IR, in file *.mb or *.ib. >> >> Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. >> >> Runs 'as' to assemble to object code, in file *.mo or *.io. >> >> Prerequisites: >> llc on the path. >> m3llvm on the path. >> as on the path. >> >> In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, >> see the m3makefile of llvm, which must give their locations. >> >> 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR >> and llvm IR are created, but you can get human-readable versions of them with >> 'm3cgcat' and 'llvm-dis'. >> >> Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to >> /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line >> parameters to m3llvm. (Other targets are not supported). >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:54:46 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:54:46 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop> Message-ID: <55E1FFF6.1070102@lcwb.coop> On 08/29/2015 06:02 AM, Peter McKinna wrote: > Hi Rodney, > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, instead of the 3.5.0. > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > upgrade.py worked OK for me after I went back to the release compiler and did it from there, which is the case we really care about. Llvm changes themselves should not affect the upgrade process, since no llvm mode is used in building the distribution. > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm changed the debug info data structure design to make it not derived from Value. Not only did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, though systematic and simple. These are in separate M3 package llvmbindings. In the case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather than just what is currently used. I still am uncomfortable about how the complex C++ type hierarchies should be reflected in the bindings. I doesn't look feasible to reflect these into true M3 hierarcharies. But clients need some kind of conversions. Some functions return results that need to be passed back in later, but to a parameter having a static subtype or supertype thereof. The unwrap functions used in llvm appear to sometimes just silently convert (the equivalent of) failing NARROWs into NIL pointers, rather than runtime errors. This will inevitably make some bugs much harder to track down. I'm not even sure dynamic type checks alway happen. Maybe some undetected type errors can propagate into C++ and give inscrutable segfaults. I am hoping future llvm releases will not affect this so much. They do change things a lot. I also like to avoid chasing the new releases too often. Just as I was getting llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. This all compiled and linked, as last I checked, but I have done very little execution checking. These do require getting some llvm stuff in the right places and states. I have had similar frustrations as Jay has with trying to use git branches, so for now, I just made this a separate package. They generate alternative executables with comparable function, so should be easy to switch between. Eventually, we will have to decide what to do about getting needed llvm stuff in, in a standard place and also, how much work will be done outside of the cm3 build system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc would be cleanest for users, but it is very big, takes hours to compile, and maybe not everybody will want to use it badly enough to pay that price. Right now, you have to do building and installing of llvm on the side, then edit a few paths in m3makefiles. Keeping m3llvm as a completely separate executable means those who don't want to use it can be unbothered. With its still being in major development, this seems best for now. I originally set out on the path of having the functions of m3llvm linked in to cm3, as with the C backend, but am skeptical about all the llvm libraries that have to be built and linked in. > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > Those things will take some work, but should not be conceptually difficult. > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > Regards Peter > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > Peter, > > How actively are you working on the llvm back end? I don't see a lot of updates to > the github repository. I have checked in some minor things and think I am about > ready to do more. But I think it best we keep things merged as much as reasonable. > > One change is adding specialized support for calls on alloca. These are very > recently being generated as library calls by the front end, and come through > the whole process as link failures. I think it best to detect these and convert > to llvm alloca instructions. m3cc does something like this now. > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From lemming at henning-thielemann.de Sat Aug 29 21:56:33 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Sat, 29 Aug 2015 21:56:33 +0200 (CEST) Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> Message-ID: On Sat, 29 Aug 2015, Rodney M. Bates wrote: > I still am uncomfortable about how the complex C++ type hierarchies > should be reflected in the bindings. In Haskell I am calling the C interface to LLVM, like certainly many other high-level language bindings do. Would this be an option for Modula-3 bindings to LLVM, too? However, the C interface of LLVM is not well documented and maybe not complete. From jay.krell at cornell.edu Sun Aug 30 00:12:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:12:37 +0000 Subject: [M3devel] paths too truncated in assertion failures Message-ID: These messages: ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** 1) It should really be a full path. I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. Yes, full paths could "leak" across machines but I think that is ok. I did work on this long ago but people disagreed with at the time. 2) We could/should trim the root of the git checkout, so it sayse.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. Some C compilers have that feature, when people are conceredwith cross machine consistency for the sake of caching and similarbuild optimizations. I still prefer #1. 3) Or at least m3front/src/convert/Convert.m33b) or ../../m3front/src/convert/Convert.m3 if we really want it to be a correct relative path from the target directory. A big part of the problem is the build system is geared toward packages,not multiple packages. I'd like to solve this problem too somehow someday. Really -- #1 -- source paths should be full paths. C compilers vary here btw, in terms of what __FILE__ is and what isin debugging information.Sometimes whatever is passed on the command line is used.Sometimes full paths are computed by the compiler. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 00:20:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:20:16 +0000 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: I understand your dilemas.Static linking vs. dynamic.Cloning the source or not.Significantly growing our repository or not.Forking or not -- do we need any patches? Hopefully not but ok if we do. Clearly the C backend cheats -- the C compiler appears on the systemas if by magic, or various external means. I am somewhat keen on seeing this LLVM stuff work.And somewhat interested in helping. First, you know, there are many sayings and their redundancies and opposites: 1a too many cooks spoil the broth 1b 9 women cannot have a baby in one month (but they can have 9 in 9) 2 more/many hands make less/little work Which is it here, 1 or 2?i.e. do you need/want help? I have built LLVM now a few times.Yes it is large and takes time.Do we have any required patches for it?I'm still on the fence as to if we should "import" it, vs. depend on its presence"somehow". If there really is version sensitivity, that argues for importing.While "apt-get install llvm" might be in widespread availability,the exact "apt-get install llvm-3.5.0" I imagine isn't. The current setup obviously needs work.I'll likely commit something at least a little better. Do we really need 3.5.0, or can I use 3.5.2? > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > problems other people have had, guess I will have to trust upgrade.py. Really, I run this stuff all the time. On various hosts. What you/we/I do need though is a safe way to rebuild the compiler when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. In particular, if you have an ABI change, you need two different m3cores. The one for the working compiler and the one for the new compiler. Maybe I just need to use "buildlocal" more. It doesn't help that the C backend isn't ABI compatible with cm3cg, due to nested functions and static links. cm3cg uses a target-dependent extra register not accessible to C as far as I know. C backend uses an extra last parameter. - Jay > Date: Sat, 29 Aug 2015 13:54:46 -0500 > From: rodney_bates at lcwb.coop > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > Subject: Re: [M3devel] Modula-3 llvm backend > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > Hi Rodney, > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > instead of the 3.5.0. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > from there, which is the case we really care about. Llvm changes themselves should > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > changed the debug info data structure design to make it not derived from Value. Not only > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > though systematic and simple. These are in separate M3 package llvmbindings. In the > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > than just what is currently used. > > I still am uncomfortable about how the complex C++ type hierarchies should be > reflected in the bindings. I doesn't look feasible to reflect these into true > M3 hierarcharies. But clients need some kind of conversions. Some functions > return results that need to be passed back in later, but to a parameter having > a static subtype or supertype thereof. The unwrap functions used in llvm appear > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > pointers, rather than runtime errors. This will inevitably make some bugs much > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > I am hoping future llvm releases will not affect this so much. They do change things > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > This all compiled and linked, as last I checked, but I have done very little execution > checking. These do require getting some llvm stuff in the right places and states. > I have had similar frustrations as Jay has with trying to use git branches, so > for now, I just made this a separate package. They generate alternative executables > with comparable function, so should be easy to switch between. > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > in a standard place and also, how much work will be done outside of the cm3 build > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > would be cleanest for users, but it is very big, takes hours to compile, and maybe > not everybody will want to use it badly enough to pay that price. Right now, you > have to do building and installing of llvm on the side, then edit a few paths in > m3makefiles. > > Keeping m3llvm as a completely separate executable means those who don't want to use > it can be unbothered. With its still being in major development, this seems best > for now. I originally set out on the path of having the functions of m3llvm linked > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > have to be built and linked in. > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > Those things will take some work, but should not be conceptually difficult. > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > Regards Peter > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > Peter, > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > the github repository. I have checked in some minor things and think I am about > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > One change is adding specialized support for calls on alloca. These are very > > recently being generated as library calls by the front end, and come through > > the whole process as link failures. I think it best to detect these and convert > > to llvm alloca instructions. m3cc does something like this now. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:45:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? Message-ID: The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:57:58 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:57:58 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: On the IA64 matter, it appears the bundle is the lower 5 bits.And the bundles 6, 7, 14, 15, 1A, 1B, 1E, 1F are invalid.It appears IA64 can be little or big endian, but the instructionsare always little endian.Therefore a 64bit value of -1 would seem invalid and ok for a marker,but 0 is less clear -- might still be invalid. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 30 19:26:21 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:26:21 -0500 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <55E33CBD.307@lcwb.coop> I too have long wished for a longer path in these messages. Although all source files in a package are required to have unique simple names, I doubt this is true between packages, except for visible interfaces. Even when unique, unless you have it all memorized, it is just an extra step to find a source file from only a simple name, when you are trying concentrate on the bug. "../src/" usually provides no information. I would be happy if the paths only went back as far as, say, cm3, for things in the repo, but how would we define where to stop for outside code? In a way, I think a full path as on the machine where the program was compiled might be more helpful anyway. Otherwise, that is really lost information, if you have any reason to care about it. A partial path that translates in an apparent way from the compiling to the executing machine could even be misleading, as the files could be quite different. OTOH, this can happen even when it's all the same machine, though probably less likely. On 08/29/2015 05:12 PM, Jay K wrote: > These messages: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/convert/Convert.m3", line 47 > *** > > > 1) It should really be a full path. > > I know people will disagree with me. > You want more commonality across machines. > I'm not sure that is worth it. > In particular, debuggers always work more easily with full paths, for local private builds. > Hopefully for debugging someone else's, some search path > with "prefix replacement" is viable. > But debugging your own build is more common and ideally > no special setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > > 2) We could/should trim the root of the git checkout, so it says > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > Some C compilers have that feature, when people are concered > with cross machine consistency for the sake of caching and similar > build optimizations. > > > I still prefer #1. > > > 3) Or at least m3front/src/convert/Convert.m3 > 3b) or ../../m3front/src/convert/Convert.m3 > > > if we really want it to be a correct relative path from the target directory. > > > A big part of the problem is the build system is geared toward packages, > not multiple packages. I'd like to solve this problem too somehow someday. > > > Really -- #1 -- source paths should be full paths. > > C compilers vary here btw, in terms of what __FILE__ is and what is > in debugging information. > Sometimes whatever is passed on the command line is used. > Sometimes full paths are computed by the compiler. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 19:52:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:52:59 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: <55E342FB.7060503@lcwb.coop> On 08/29/2015 05:20 PM, Jay K wrote: > I understand your dilemas. > Static linking vs. dynamic. > Cloning the source or not. > Significantly growing our repository or not. > Forking or not -- do we need any patches? Hopefully not but ok if we do. > > > Clearly the C backend cheats -- the C compiler appears on the system > as if by magic, or various external means. > > > I am somewhat keen on seeing this LLVM stuff work. > And somewhat interested in helping. > > > First, you know, there are many sayings and their redundancies and opposites: > > 1a too many cooks spoil the broth > 1b 9 women cannot have a baby in one month (but they can have 9 in 9) > > 2 more/many hands make less/little work > Which is it here, 1 or 2? > i.e. do you need/want help? > Help is surely needed/wanted, as long as we don't start too much tripping over each other. Just exactly where needs some thought. > I have built LLVM now a few times. > Yes it is large and takes time. > Do we have any required patches for it? I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain it is to retrofit patches to new versions. But that may be unrealistic. The last I checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be fixed in the llvm source tree. Maybe we can get them to do that for us. > I'm still on the fence as to if we should "import" it, vs. depend on its presence > "somehow". > > If there really is version sensitivity, that argues for importing. There is definitely version sensitivity. There were big changes in the data structure for debug info and the calls to construct it, somewhere between 3.5.0 and 3.6.1, require big rework of bindings (which they do not provide.) They do not hesitate to change interface things. There has been a roaring discussion about whether/how to start keeping the "Core" interface stable--something apparently has not be a goal so far. Core is a C interface they provide to certain things, notably not debug info. > While "apt-get install llvm" might be in widespread availability, > the exact "apt-get install llvm-3.5.0" I imagine isn't. > > > The current setup obviously needs work. > I'll likely commit something at least a little better. > > Do we really need 3.5.0, or can I use 3.5.2? > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything pretending to be a binding for. I prefer to at least start with 3.6.1, since it is later and significantly different, so will require just that much less retrofitting. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > > problems other people have had, guess I will have to trust upgrade.py. > > Really, I run this stuff all the time. > On various hosts. > What you/we/I do need though is a safe way to rebuild the compiler > when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. > In particular, if you have an ABI change, you need two different m3cores. > The one for the working compiler and the one for the new compiler. > Maybe I just need to use "buildlocal" more. > It doesn't help that the C backend isn't ABI compatible with cm3cg, due to > nested functions and static links. cm3cg uses a target-dependent extra > register not accessible to C as far as I know. C backend uses an extra > last parameter. > > - Jay > > > > > > Date: Sat, 29 Aug 2015 13:54:46 -0500 > > From: rodney_bates at lcwb.coop > > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > > Subject: Re: [M3devel] Modula-3 llvm backend > > > > > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > > Hi Rodney, > > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > > instead of the 3.5.0. > > > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > > from there, which is the case we really care about. Llvm changes themselves should > > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > > changed the debug info data structure design to make it not derived from Value. Not only > > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > > though systematic and simple. These are in separate M3 package llvmbindings. In the > > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > > than just what is currently used. > > > > I still am uncomfortable about how the complex C++ type hierarchies should be > > reflected in the bindings. I doesn't look feasible to reflect these into true > > M3 hierarcharies. But clients need some kind of conversions. Some functions > > return results that need to be passed back in later, but to a parameter having > > a static subtype or supertype thereof. The unwrap functions used in llvm appear > > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > > pointers, rather than runtime errors. This will inevitably make some bugs much > > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > > > I am hoping future llvm releases will not affect this so much. They do change things > > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > > This all compiled and linked, as last I checked, but I have done very little execution > > checking. These do require getting some llvm stuff in the right places and states. > > I have had similar frustrations as Jay has with trying to use git branches, so > > for now, I just made this a separate package. They generate alternative executables > > with comparable function, so should be easy to switch between. > > > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > > in a standard place and also, how much work will be done outside of the cm3 build > > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > > would be cleanest for users, but it is very big, takes hours to compile, and maybe > > not everybody will want to use it badly enough to pay that price. Right now, you > > have to do building and installing of llvm on the side, then edit a few paths in > > m3makefiles. > > > > Keeping m3llvm as a completely separate executable means those who don't want to use > > it can be unbothered. With its still being in major development, this seems best > > for now. I originally set out on the path of having the functions of m3llvm linked > > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > > have to be built and linked in. > > > > > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > > > > Those things will take some work, but should not be conceptually difficult. > > > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > > > Regards Peter > > > > > > > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > > > Peter, > > > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > > the github repository. I have checked in some minor things and think I am about > > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > > > One change is adding specialized support for calls on alloca. These are very > > > recently being generated as library calls by the front end, and come through > > > the whole process as link failures. I think it best to detect these and convert > > > to llvm alloca instructions. m3cc does something like this now. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 20:24:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 13:24:55 -0500 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: <55E34A77.1050001@lcwb.coop> On 08/30/2015 02:45 AM, Jay K wrote: > The agenda remains seeing if Target variables can be made constants. > > The discussion in this case is more complicated and some facts are unclear. > > > Background: > > > Nested functions are a problem. > In particular, if you can take their address. > Taking the address of a nested function presents a problem. > You are presented with two or three solutions. > > > - runtime code gen > - either on the stack > > - or somewhere more expensive, possibly with garbage collection > > - possibly "templated" instead of "arbitrary"; the meaning > of this is a lot to explain -- related to libffi and mremap, which > isn't entirely portable, but is e.g. portable to Windows > > > - or instead of runtime codegen, altering how function pointers are > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > The sequence is roughly: > Check if it is a "regular" function pointer or a "closure". > If it is a "regular" function pointer, just call it. > If it is a "closure", it contains a function pointer and a "static link". > Call the function pointer, passing the static link. > > > To tell if it is "regular" function pointer or a "closure", roughly > what is done is the data at the function pointer is read and compared > against a marker value. If it equals the marker value, it is a closure. > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > The value of the marker checked for is 0 or -1, I'd have to check. > The alignment of the pointer might be a factor. In particular, on most > architectures, all instructions have a certain alignment. If the pointer has > less alignment, it can't be an instruction. Or maybe on those architectures, > the bytes are read one at a time to avoid alignment faults. > > > In particular, as far as I know, the following: > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > 4 bytes and 4 byte aligned, so functions are at least also as much > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > pointer is actually odd as well, and the low bit is removed to really find the instructions > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > I could use confirmation on much of this. > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > value that works on all. > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > then checking for two in a row reduces the odds of a false positive. > > > However, given that the closure is a marker and two pointers, it isn't like you are going > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a valid instruction or sequence thereof. So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity here. Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for certain (especially if we want to be able to call procedures produced by other compilers) but much less likely to result in a problem. Just a wild guess, but I would not be surprised if ELF and other object formats would require the machine code of a function/procedure to begin on a native word boundary, even if the hardware instruction set does not. Where so, this would obviate checking the alignment before checking for a closure, though probably target-dependently. > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > However, I want less target-variation not more. > > > Here are some my lingering questions: > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > - Is a 64bit marker value actually sufficient on IA64? > The way to help here, I think, is to ensure that a 64bit marker, > not a 128bit marker, contains the "template", and an invalid "template". > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > i believe all of these function pointers are rare. > I hope/believe the object method calls do not check for closures -- though actually > that is related to a useful language construct, that I doubt we have. > Method body procedures are required to be top-level, ensured statically, so there is no need for method call code to consider the possibility of the pointer in the object type to be a closure. > > The simplest solution is likely: > - ignore IA64, or research it further > - keep marker size at integer Pointer would be target-independent in getting the following two pointers aligned. > - for the C backend, assume no alignment of function pointers -- give up > any of the optimization, esp. x86/amd64. I think this optimization both applies to a low-frequency situation and has a very small benefit, so I would not worry about giving up on it. > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > Thoughts? > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 30 22:45:18 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 30 Aug 2015 13:45:18 -0700 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E342FB.7060503@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> <55E342FB.7060503@lcwb.coop> Message-ID: <6B3E0C66-4A5A-44FF-9910-42DD01413214@gmail.com> I'm kind of impatient, sorry, that will be reflected here: - if you have a known working version, stick with it for now? - maintain multiple bindings? - patches: good & bad: it can be fun to fork, but also wasteful; maintaining diff/patch files may be an option. - Proceed without dwarf? If much else to do? You certain the problem is in LLVM? Your patches totally fix the problem. - Aomewhat interesting, m3x86 & m3cc for *some* platforms, e.g. Darwin, produce a minimum of debugging info, kinda surprising, & people don't seem to notice; I have largely fixed this with the C backend though & intend to fix it more. - Presumably I can/should expand beyond AMD_LINUX. I was going to use I386_DARWIN, but until I convince LLVM to be that, probably AMD64_DARWIN. - Jay On Aug 30, 2015, at 10:52 AM, "Rodney M. Bates" wrote: > > > On 08/29/2015 05:20 PM, Jay K wrote: >> I understand your dilemas. >> Static linking vs. dynamic. >> Cloning the source or not. >> Significantly growing our repository or not. >> Forking or not -- do we need any patches? Hopefully not but ok if we do. >> >> >> Clearly the C backend cheats -- the C compiler appears on the system >> as if by magic, or various external means. >> >> >> I am somewhat keen on seeing this LLVM stuff work. >> And somewhat interested in helping. >> >> >> First, you know, there are many sayings and their redundancies and opposites: >> >> 1a too many cooks spoil the broth >> 1b 9 women cannot have a baby in one month (but they can have 9 in 9) >> >> 2 more/many hands make less/little work >> Which is it here, 1 or 2? >> i.e. do you need/want help? > > Help is surely needed/wanted, as long as we don't start too much tripping > over each other. Just exactly where needs some thought. > >> I have built LLVM now a few times. >> Yes it is large and takes time. >> Do we have any required patches for it? > > I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain > it is to retrofit patches to new versions. But that may be unrealistic. The last I > checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be > fixed in the llvm source tree. Maybe we can get them to do that for us. > >> I'm still on the fence as to if we should "import" it, vs. depend on its presence >> "somehow". >> >> If there really is version sensitivity, that argues for importing. > > There is definitely version sensitivity. There were big changes in the > data structure for debug info and the calls to construct it, somewhere > between 3.5.0 and 3.6.1, require big rework of bindings (which they do > not provide.) They do not hesitate to change interface things. There > has been a roaring discussion about whether/how to start keeping the > "Core" interface stable--something apparently has not be a goal so far. > Core is a C interface they provide to certain things, notably not > debug info. > >> While "apt-get install llvm" might be in widespread availability, >> the exact "apt-get install llvm-3.5.0" I imagine isn't. >> >> >> The current setup obviously needs work. >> I'll likely commit something at least a little better. >> >> Do we really need 3.5.0, or can I use 3.5.2? > > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything > pretending to be a binding for. I prefer to at least start with 3.6.1, since it is > later and significantly different, so will require just that much less retrofitting. > >> >> > So I will be upgrading asap. I am always a bit wary of upgrading seeing the >> > problems other people have had, guess I will have to trust upgrade.py. >> >> Really, I run this stuff all the time. >> On various hosts. >> What you/we/I do need though is a safe way to rebuild the compiler >> when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. >> In particular, if you have an ABI change, you need two different m3cores. >> The one for the working compiler and the one for the new compiler. >> Maybe I just need to use "buildlocal" more. >> It doesn't help that the C backend isn't ABI compatible with cm3cg, due to >> nested functions and static links. cm3cg uses a target-dependent extra >> register not accessible to C as far as I know. C backend uses an extra >> last parameter. >> >> - Jay >> >> >> >> >> > Date: Sat, 29 Aug 2015 13:54:46 -0500 >> > From: rodney_bates at lcwb.coop >> > To: peter.mckinna at gmail.com; m3devel at elegosoft.com >> > Subject: Re: [M3devel] Modula-3 llvm backend >> > >> > >> > >> > On 08/29/2015 06:02 AM, Peter McKinna wrote: >> > > Hi Rodney, >> > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. >> > > >> > >> > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, >> > instead of the 3.5.0. >> > >> > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. >> > > >> > >> > upgrade.py worked OK for me after I went back to the release compiler and did it >> > from there, which is the case we really care about. Llvm changes themselves should >> > not affect the upgrade process, since no llvm mode is used in building the distribution. >> > >> > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. >> > > >> > >> > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm >> > changed the debug info data structure design to make it not derived from Value. Not only >> > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, >> > though systematic and simple. These are in separate M3 package llvmbindings. In the >> > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather >> > than just what is currently used. >> > >> > I still am uncomfortable about how the complex C++ type hierarchies should be >> > reflected in the bindings. I doesn't look feasible to reflect these into true >> > M3 hierarcharies. But clients need some kind of conversions. Some functions >> > return results that need to be passed back in later, but to a parameter having >> > a static subtype or supertype thereof. The unwrap functions used in llvm appear >> > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL >> > pointers, rather than runtime errors. This will inevitably make some bugs much >> > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe >> > some undetected type errors can propagate into C++ and give inscrutable segfaults. >> > >> > I am hoping future llvm releases will not affect this so much. They do change things >> > a lot. I also like to avoid chasing the new releases too often. Just as I was getting >> > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. >> > >> > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. >> > This all compiled and linked, as last I checked, but I have done very little execution >> > checking. These do require getting some llvm stuff in the right places and states. >> > I have had similar frustrations as Jay has with trying to use git branches, so >> > for now, I just made this a separate package. They generate alternative executables >> > with comparable function, so should be easy to switch between. >> > >> > Eventually, we will have to decide what to do about getting needed llvm stuff in, >> > in a standard place and also, how much work will be done outside of the cm3 build >> > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc >> > would be cleanest for users, but it is very big, takes hours to compile, and maybe >> > not everybody will want to use it badly enough to pay that price. Right now, you >> > have to do building and installing of llvm on the side, then edit a few paths in >> > m3makefiles. >> > >> > Keeping m3llvm as a completely separate executable means those who don't want to use >> > it can be unbothered. With its still being in major development, this seems best >> > for now. I originally set out on the path of having the functions of m3llvm linked >> > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that >> > have to be built and linked in. >> > >> > >> > >> > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. >> > > >> > >> > Those things will take some work, but should not be conceptually difficult. >> > >> > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. >> > > >> > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. >> > > >> > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. >> > > >> > > Regards Peter >> > > >> > >> > >> > >> > > >> > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: >> > > >> > > Peter, >> > > >> > > How actively are you working on the llvm back end? I don't see a lot of updates to >> > > the github repository. I have checked in some minor things and think I am about >> > > ready to do more. But I think it best we keep things merged as much as reasonable. >> > > >> > > One change is adding specialized support for calls on alloca. These are very >> > > recently being generated as library calls by the front end, and come through >> > > the whole process as link failures. I think it best to detect these and convert >> > > to llvm alloca instructions. m3cc does something like this now. >> > > >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> > > >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 08:26:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 06:26:32 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <55E33CBD.307@lcwb.coop> References: , <55E33CBD.307@lcwb.coop> Message-ID: It isn't perfectly scientific, as even full paths on a machinecan have variable meaning through time. For example, I can delete some source, put other source in its place,similar or different. And on Windows there is the "net use" feature where drive lettersmap to the network. Then you delete the use, and make another one,same letter, different place. Similar to mounting on Unixy systems. Here is another story in this area.Hopefully I have my facts correct.Perhaps it is just apocryphal and I filled in details to suit my agenda.We can double check it all with a bit of time. Visual C++, has, like any other system, some value for __FILE__and some values in the debugging information. A long time ago, they were both whatever you passed on the command line. At some point, a switch was added /FC that meant "full path the valuesin the debugging information". I believe this was added around Visual C++ 5.0.Later however, it was realized, that should always be how it works. The switchno longer does anything. Or maybe /FC still serves to full path __FILE__ but no longer has anyaffect on debugging information.I'd have to go back and try every compiler I have to verify. So, I think this actually motivate my other point from years ago,which is to full path the file names in the debug info, but not necessarily here.But I want both. :) I debug a lot of stuff, and it is unavoidably difficult, but some thingsjust make it unnecessarily more difficult and tedious. We should actually be recording crcs of source files and the assertcan go off and find the source and report if it matches.Or at least the assert can report the crc and user can verify it himself.i.e. once an assert fails, you can't do much reliably. And yes, when there is no CM3_ROOT or such, unclear what partof the path to truncate.Which motivates the other option -- don't truncate anything. I believe the names have to be unique "across the link".Visible or not, I'm not sure it matters.I don't believe we use the "static" feature of C/.obj files,so even non public functions, and the globals, are in a shared globalnamespace. And with dynamic linking, you can't be sure they don't surviveinto the dynamic link search space at least on Unixy systems.. - Jay > Date: Sun, 30 Aug 2015 12:26:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] paths too truncated in assertion failures > > I too have long wished for a longer path in these messages. Although > all source files in a package are required to have unique simple names, > I doubt this is true between packages, except for visible interfaces. > Even when unique, unless you have it all memorized, it is just an extra > step to find a source file from only a simple name, when you are trying > concentrate on the bug. "../src/" usually provides no information. > > I would be happy if the paths only went back as far as, say, cm3, for > things in the repo, but how would we define where to stop for outside > code? In a way, I think a full path as on the machine where the program > was compiled might be more helpful anyway. Otherwise, that is really lost > information, if you have any reason to care about it. A partial path > that translates in an apparent way from the compiling to the executing > machine could even be misleading, as the files could be quite different. > > OTOH, this can happen even when it's all the same machine, though probably > less likely. > > On 08/29/2015 05:12 PM, Jay K wrote: > > These messages: > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file "../src/convert/Convert.m3", line 47 > > *** > > > > > > 1) It should really be a full path. > > > > I know people will disagree with me. > > You want more commonality across machines. > > I'm not sure that is worth it. > > In particular, debuggers always work more easily with full paths, for local private builds. > > Hopefully for debugging someone else's, some search path > > with "prefix replacement" is viable. > > But debugging your own build is more common and ideally > > no special setting is needed to make that work. > > > > Yes, full paths could "leak" across machines but I think that is ok. > > > > I did work on this long ago but people disagreed with at the time. > > > > > > 2) We could/should trim the root of the git checkout, so it says > > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > > > > Some C compilers have that feature, when people are concered > > with cross machine consistency for the sake of caching and similar > > build optimizations. > > > > > > I still prefer #1. > > > > > > 3) Or at least m3front/src/convert/Convert.m3 > > 3b) or ../../m3front/src/convert/Convert.m3 > > > > > > if we really want it to be a correct relative path from the target directory. > > > > > > A big part of the problem is the build system is geared toward packages, > > not multiple packages. I'd like to solve this problem too somehow someday. > > > > > > Really -- #1 -- source paths should be full paths. > > > > C compilers vary here btw, in terms of what __FILE__ is and what is > > in debugging information. > > Sometimes whatever is passed on the command line is used. > > Sometimes full paths are computed by the compiler. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 15:31:29 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 15:31:29 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Mon Aug 31 16:26:59 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: > On Sat, 29 Aug 2015 22:12:37 +0000 > Jay K wrote: > > > These messages: > > ****** runtime error:*** <*ASSERT*> failed.*** file > "../src/convert/Convert.m3", line 47*** > > > > 1) It should really be a full path. > > I know people will disagree with me.You want more commonality across > machines.I'm not sure that is worth it.In particular, debuggers always work > more easily with full paths, for local private builds.Hopefully for > debugging someone else's, some search pathwith "prefix replacement" is > viable.But debugging your own build is more common and ideallyno special > setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > I'd vote for full absolute path. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 31 17:06:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 31 Aug 2015 10:06:12 -0500 Subject: [M3devel] Oops, belated commit of llvm3.6.1 Message-ID: <55E46D64.1020806@lcwb.coop> I apparently forgot, a while back, to commit new package llvm3.6.1, which I have been writing about. It is now in the Github repo. It compiles, but segfaults, most likely because of type problems in llvmbindings. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 20:02:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 18:02:07 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: , <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com>, Message-ID: And in compiler/warnings errors? i.e. do we need to preserve the "short" path in some cases? I vote for full path always. We can also provide options to trim specified prefixes, so builds at different paths get identical output. - Jay Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: Re: [M3devel] paths too truncated in assertion failures From: lists at darko.org To: wagner at elegosoft.com CC: jay.krell at cornell.edu; m3devel at elegosoft.com Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 20:08:44 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 20:08:44 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: <20150831200844.5b6c4c09e86238534fd1f659@elegosoft.com> On Mon, 31 Aug 2015 18:02:07 +0000 Jay K wrote: > And in compiler/warnings errors? > > i.e. do we need to preserve the "short" path in some cases? > > I vote for full path always. I second that. > > We can also provide options to trim specified prefixes, so > builds at different paths get identical output. > > - Jay -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Sun Aug 2 02:43:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 01 Aug 2015 19:43:43 -0500 Subject: [M3devel] better typing on SUBARRAY temporaries? In-Reply-To: References: <55B03594.20800@lcwb.coop> <55B25261.5010106@lcwb.coop> <55B3AA70.5050906@lcwb.coop> <55B3B8CF.1060208@lcwb.coop> Message-ID: <55BD67BF.4040606@lcwb.coop> On 07/26/2015 02:23 AM, Peter McKinna wrote: > So far not doing anything with free_temp, declare_temp just does an alloca but does it at the end of the first basic block to avoid dominate all uses problems. ( I have since discovered that there is a flag -alloca-hoisting which does this for you). The language ref hardly mentions temps except that they are referred to as unnamed values ie %1 %2 etc whether the optimiser does anything with these I dont know. I think I'm giving the temps names like %tmp.1 etc > The workaround I have at the moment is that if in the store, (which I think is the first place one of these temps is referenced) the var is a temp ie having NoID and is not declared in the current procedure then declare it ie alloca it. Seems to work but seems pretty kludgy. > My first thought when I struck this problem was that I should put all temps on the static link (since there is no parameter in declare_temp to say its uplevel and hence say which could be nested proc material) but that didnt work and really it was a pretty dumb idea. I cant see why the front end couldnt just declare a new temp in the finally proc. > Commit 496e9be1dcdcf87bda8e72239fc90132591b4cf4 fixes this, for this test case. > Regards Peter > > On Sun, Jul 26, 2015 at 2:26 AM, Rodney M. Bates > wrote: > > More on this: > > I appears that, in the CM3 IR, "variables" declared by declare_local and one > declared by declare_temp (and a few others, too), while declared with different > operators, are accessible interchangeably. In this example, the FINALLY procedure > accesses both s and the temporary nonlocally, in the same way. The difference > is, the temp, unlike a local variable, has the possibility that its space in > the activation record is freed prior to the end of the corresponding code. > > What llvm IR are you translating declare_temp and free_temp into? Llvm might > have different rules for temps. > > > > > On 07/25/2015 10:25 AM, Rodney M. Bates wrote: > > I compiled this and looked at the cm3 IR for it. At first glance, it looks like > a bug in the front end. The FINALLY code is translated as a nested procedure, > inside the one containing the entire TRY..FINALLY statement. The temp is created by > a declare_temp in the outer procedure, freed by a free_temp in the inner one, > and used in both, the 2 uses being unrelated. > > In m3-sys/m3middle/src/M3CG_Ops.i3, I see: > ----------------------------------------------------------------------------- > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > free_temp (v: Var); > (* releases the space occupied by temp 'v' so that it may be reused by > other new temporaries. *) > ----------------------------------------------------------------------------- > > And it also seems strange for the temp to be freed inside the nested procedure, > although this does not violate what the comment says. The fact that every > temp has a unique variable number would take care of matching the free_temp to the > right declare_temp, and maybe the code generators can handle freeing a nonlocal > temp. > > Apparently, this has caused no problems with preexisting code generators. > But it certainly looks wrong, and clearly violates the comments. > > I recall that the gcc-derived code generator and the integrated x86 code generator > both unnest nested procedures, in opposite ways (one pulls them out in front, the > other in back), which might have something to do with how they handle this. > > What happens in the llvm back end for a programmer-declared nested procedure > making a nonlocal reference to a programmer-declared local variable of the > containing procedure? If you can handle this latter case, can you handle the > failing one the same way? Maybe this is what is happening in the other code > generators, and the comment is just too strict. > > > > > On 07/24/2015 08:13 PM, Peter McKinna wrote: > > On the subject of temporaries can I get your thoughts on the following? > > TRY > s := s & "a"; > FINALLY > s := s & "b"; > END; > > The front end declares a temp in the try block as part of its concat and then refers to the same temp in the finally block. The trouble is that the finally code is generated in a separate procedure and references a temp declared in the proc of the try. In llvm the first you know of the problem is a store to the temp which has not been declared. > Just wondering whether the front end should redeclare this temp? Also is the front end generating similar temps for other runtime operations? > > Regards Peter > > On Sat, Jul 25, 2015 at 12:57 AM, Rodney M. Bates >> wrote: > > > > On 07/24/2015 03:57 AM, Jay K wrote: > > I model this in C like: > > > M3C.m3: > print(self, "/*declare_open_array*/typedef struct {"); > > > Presumably, this comment means you have some way of knowing, in M3C, that this is an open array? > > print(self, element_type.text); > print(self, "* _elts; CARDINAL _size"); > IF bit_size > Target.Integer.size * 2 THEN > print(self, "s["); > print(self, IntToDec((bit_size - Target.Integer.size) DIV Target.Integer.size)); > print(self, "]"); > END; > > > that is..and this i stinky that I get the struct "size", > > > size == 2 * sizeof(integer): > > > struct { > T* elements > size_t size; > } > > > else: > N = size - sizeof(INTEGER) / sizeof(INTEGER) > T ** elements; // where the number of star is N > > > I would not do pointer to pointer to pointer ... here. Just "T* elements", regardless > of the number of open dimensions. As I understand them, C's language-supported > multidimensional arrays are like Modula3 multidimensional _fixed_ arrays, i.e., > the language does the multi-subscript address arithmetic, which means the type > system needs to provide a static element size of each dimension. And that, > except for the innermost, depends on the element count of the next inner > dimension, which is not static here. > > So, the multiple dimensions are essentially flattened into one, and access to A[I,J,K] > is lowered by the front end into explicit address arithmetic into the flattened > array, using the values in the shape. Something like A[I*Shape[0]+J*Shape[1]+K] > > size_t sizes[N] > > > It is kind of lame that the frontend just gives the overall size > and the backend is just left to assume the layout like that. > > > If you know in M3C that it's an open array, you can infer what the layout is. > But yes, it's kind of lame. This is just another of the several places we have > seen that the front end has lowered things too far, and you have to > You know it's generated by code in OpenArray.m3, and you know the general dope layout, > and open dimension count, so you can generate an appropriate type. > > > > Really, the frontend should declare a type "pointer to pointer to pointer" with > the right "depth", and then a record with that pointer and a size or fixed size array of sizes. > > > Again, really ugly how it works now where backend is just given a size and can only > assume the layout. > > I don't know what a "dope vector" is. A value used as an initializer? > > > This is a very old and rather uninformative term for any block of stuff stored at runtime > that describes some source programmer's real data and how to access it. The only alternative > term I can think of would be "metadata", although that is rather overgeneral, and is usually > used with quite different specific meanings. But it is data describing data. > > > It is even worse for subarray. In this case we aren't even told it is an open array, just > some random struct with a size. That is what I first want to fix. It should declare an open array, > assuming they do have the same layout, which I think they do. > > > What you really need to know is that it's an open array, of which subarray is a subcategory. > In our implementation, all open array values have the same dope. E.g., look for the case where > a fixed array actual parameter is passed to an open array formal. > > > subarray temporaries and jmpbufs are I believe the only place the frontend passes so little > type information. > > > For jmpbufs I'm hoping to notice their name, and, unfortunately expensive, replace them > with #include and jmpbuf, instead of just a struct with an array of bytes. > > > > > - Jay > > > > Date: Wed, 22 Jul 2015 19:30:12 -0500 > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] better typing on SUBARRAY temporaries? > > > > I'm not exactly sure what you are asking, but here is some light on what > > you are seeing. These temporaries are exactly the dope the compiler uses > > to represent all open array values. First a pointer to the zeroth > > array element, then the "shape", as defined in M3 definition, 2.2.3, i.e. > > an array of element counts for each open subscript. For an open array > > parameter, this would be the machine representation of the parameter > > itself, authored in M3. (but passed by reference.) For a heap object, > > it is stored right before the elements themselves. For a SUBARRAY > > expression, it has to be a temporary. It also has to be constructed > > at the call site, as an anonymous temporary, when passing an entire fixed > > array to an open array parameter > > > > So, a good type for it might look like: > > > > > > RECORD > > Elements : REF ARRAY [0..Max, ... ,0..Max] OF ElementType > > ; Shape : ARRAY [0..OpenDepth-1] of CARDINAL > > END > > > > Max will be like the notorious TextLiteral.MaxBytes, i.e., we don't want any > > static limit here in the type of Elements, as it will be enforced dynamically, > > using Shape. But we don't want to just say REF ARRAY OF ElementType either, > > as this would mean another open array inside the dope, resulting in infinite > > recursion. > > > > On 07/22/2015 12:42 AM, Jay K wrote: > > > In the C backend I have a notion of "weak struct types" and "strong struct types". > > > > > > > > > "Strong" types have fields with types and names corresponding to the original Modula-3. i.e. they debug well. > > > > > > > > > "Weak" types have just arrays of characters (in a struct), sized/aligned to what the front end asked for. i.e. they debug poorly. > > > > > > > > > > > > Originally I had only weak types. > > > Ideally I have no weak types. > > > I'm down to very few weak types now. > > > I'd like to finish eliminating weak types. > > > > > > > > > > > > A quick investigation shows weak types come from open arrays and jmpbufs. > > > Open array temporaries from SUBARRAY specifically. > > > > > > > > > > > > Can we fix this? > > > > > > > > > > > > We have: > > > m3front/src/types/OpenArrayType.m3: > > > > > > PROCEDURE DeclareTemp (t: Type.T): CG.Var = > > > VAR > > > p := Reduce (t); > > > size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > RETURN CG.Declare_temp (size, Target.Address.align, > > > CG.Type.Struct, in_memory := TRUE); > > > END DeclareTemp; > > > > > > > > > PROCEDURE Compiler (p: P) = > > > VAR size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > Type.Compile (p.element); > > > CG.Declare_open_array (Type.GlobalUID(p), Type.GlobalUID(p.element), size); > > > END Compiler; > > > > > > > > > DeclareTemp is used in SUBARRAY expressions -- truly temporaries, > > > not variables authored by anyone in Modula-3. > > > > > > > > > Can this be easily fixed? > > > > > > > > > Thanks, > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > Rodney Bates > rodney.m.bates at acm.org > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 2 05:36:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 03:36:05 +0000 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? Message-ID: I propose the following: - A flag to m3front indicating if the backend "knows" what a jmpbuf is. - extend M3CG.Type = { int8, uint8, ... addr ... struct } by adding a jmpbuf element - if the backend "knows" what a jmpbuf is, then, vs. the current code, the frontend shall just declare a jmpbuf per try, leaving the backend to size it, and w/o using the upcoming alloca approach. - if the backend does not "know" what a jmpbuf is, then, well, there is the current approach of making a sized struct, or my approach that is almost working where we use alloca. The frontend will not output cgtype.jmpbuf unless the backend "knows" what it is. ok? The information as to the backend's knowledge of a jmpbuf shall beyet another internal m3front flag, like what guides nested functionoutput order. I was going to propose something slightly different, likem3front always use this new jmpbuf type, and existing backendscan just change it "back" to struct. The problem with that approach..it works easily with the current code, but does not addressmy upcoming alloca(sizeof_jmpbuf) change. In this way, what the C backend will do is: #include jmp_buf buf1; jmp_buf buf2; etc. leaving the size of the jmpbuf completely absent in the generated code,and allocating in the slightly more efficient way than alloca. Exception handling is special, so accomodating this seems reasonable. Yes, it might be temporary, replaced by libunwind stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 2 10:44:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 08:44:22 +0000 Subject: [M3devel] m3core.h broken In-Reply-To: <55BBDD0F.4080206@lcwb.coop> References: <55BBDD0F.4080206@lcwb.coop> Message-ID: Sorry, yes, my mistake.I should have used the UINT64/INT64 typedefs from nearby earlier.and my change wasn't well motivated anyway. - Jay > Date: Fri, 31 Jul 2015 15:39:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: [M3devel] m3core.h broken > > On AMD64_LINUX, with gcc 4.8.1, m3-libs/src/m3core.h suffers many compile errors, > when compiling on m3core and cm3. reverting line 363 of m3core.h makes them go away. > This is probably not the right way to fix this. > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 2 22:18:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 02 Aug 2015 15:18:58 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail>, Message-ID: <55BE7B32.7000502@lcwb.coop> On 08/02/2015 04:23 AM, Jay K wrote: > Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. > > so I have to move shared code to something called...ProcedureOrFinallyProcedure? > that otherwise was just in Procedure. I'm not sure what the breakage is, but for finally, the custom procedure generation consists only of: CG.Begin_procedure (p.handler.cg_proc); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc); in two places in TryFinStmt.m3. I would expect what you need to do to happen somewhere inside Stmt.Compile. Only there would it be known whether any TRYs are inside the FINALLY procedure. I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no doubt be compiler internally generated procedures. I would expect only those emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements inside. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3commit at elegosoft.com > Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > Date: Sun, 2 Aug 2015 02:20:51 +0000 > > Rodney, Do you have a test case that was broken before and fixed afterward? > > If possible, though not required, check something into m3-sys/m3tests? > > I don't mean to imply the code was correct before. I don't know, at all. > > Thank you, > - Jay > > > > Date: Sat, 1 Aug 2015 17:36:54 -0700 > From: rodney.m.bates at acm.org > To: m3commit at elegosoft.com > Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > > Branch: refs/heads/master > Home:https://github.com/modula3/cm3 > Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 > https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 > Author: Rodney Bates > Date: 2015-08-01 (Sat, 01 Aug 2015) > > Changed paths: > M m3-sys/m3front/src/misc/CG.m3 > > Log Message: > ----------- > Fix compiler-generated FINALLY procedure using parent procedure's temporary. > > Changes to be committed: > > modified: m3-sys/m3front/src/misc/CG.m3 > > CG was not stacking global variables used for keeping track of temporaries > when entering nested procedures. In the case of a compiler-generated > nested procedure for the finally part of a TRY--FINALLY block, this > meant the finally code could use a temporary allocated by the parent, > though not in use at the time. This violated the comment in M3CG_Ops.i3: > > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > > In cases of nested procedures explicitly declared in source code, this > didn't matter because they are located ahead of any of the parent > procedure's executable code, when it has no allocated temps, and they > free their own temps at their end, restoring the previous state. > > > > > _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > _______________________________________________ > M3commit mailing list > M3commit at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 3 09:05:17 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 00:05:17 -0700 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: <55BE7B32.7000502@lcwb.coop> References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail> <55BE7B32.7000502@lcwb.coop> Message-ID: <13B01241-502D-415A-A98E-F818DD8C77CA@gmail.com> Thank you for the obvious reminder I needed -- look at all begin_procedure calls. I briefly mulled over that my changes belong in Stmt or elsewhere, but, the thing is..maybe my understanding isn't right, Stmt occurs in many places, you can say "begin" anywhere. My changes are very related to procedures -- counting TRY per procedure, calling alloca once per procedure, near the start -- before any branches or TRYs. The breakage is that I was considering finally procs to be part of the enclosing function. Very wrong. I put the alloca & setting a local pointer per TRY circa Procedure.GenBody, which isn't for the "synthetic" procedures.. I'll poke around more. Thanks, - Jay On Aug 2, 2015, at 1:18 PM, "Rodney M. Bates" wrote: > > > On 08/02/2015 04:23 AM, Jay K wrote: >> Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. >> >> so I have to move shared code to something called...ProcedureOrFinallyProcedure? >> that otherwise was just in Procedure. > > I'm not sure what the breakage is, but for finally, the custom procedure generation consists > only of: > > CG.Begin_procedure (p.handler.cg_proc); > xc := Stmt.Compile (p.finally); > CG.Exit_proc (CG.Type.Void); > CG.End_procedure (p.handler.cg_proc); > > in two places in TryFinStmt.m3. I would expect what you need to do to happen > somewhere inside Stmt.Compile. Only there would it be known whether any TRYs > are inside the FINALLY procedure. > > I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), > RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no > doubt be compiler internally generated procedures. I would expect only those > emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements > inside. > > > >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3commit at elegosoft.com >> Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> Date: Sun, 2 Aug 2015 02:20:51 +0000 >> >> Rodney, Do you have a test case that was broken before and fixed afterward? >> >> If possible, though not required, check something into m3-sys/m3tests? >> >> I don't mean to imply the code was correct before. I don't know, at all. >> >> Thank you, >> - Jay >> >> >> >> Date: Sat, 1 Aug 2015 17:36:54 -0700 >> From: rodney.m.bates at acm.org >> To: m3commit at elegosoft.com >> Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> >> Branch: refs/heads/master >> Home:https://github.com/modula3/cm3 >> Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> Author: Rodney Bates >> Date: 2015-08-01 (Sat, 01 Aug 2015) >> >> Changed paths: >> M m3-sys/m3front/src/misc/CG.m3 >> >> Log Message: >> ----------- >> Fix compiler-generated FINALLY procedure using parent procedure's temporary. >> >> Changes to be committed: >> >> modified: m3-sys/m3front/src/misc/CG.m3 >> >> CG was not stacking global variables used for keeping track of temporaries >> when entering nested procedures. In the case of a compiler-generated >> nested procedure for the finally part of a TRY--FINALLY block, this >> meant the finally code could use a temporary allocated by the parent, >> though not in use at the time. This violated the comment in M3CG_Ops.i3: >> >> declare_temp (s: ByteSize; a: Alignment; t: Type; >> in_memory: BOOLEAN): Var; >> (* declares an anonymous local variable. Temps are declared >> and freed between their containing procedure's begin_procedure and >> end_procedure calls. Temps are never referenced by nested procedures. *) >> >> In cases of nested procedures explicitly declared in source code, this >> didn't matter because they are located ahead of any of the parent >> procedure's executable code, when it has no allocated temps, and they >> free their own temps at their end, restoring the previous state. >> >> >> >> >> _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit >> >> >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 04:35:10 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 19:35:10 -0700 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? In-Reply-To: References: Message-ID: <542D0438-1C2D-463A-96BC-E7CA6D0F2122@gmail.com> The alloca approach is fairly sound, w/o this change. However in the "cake and eat it too" dept..the C backend could instead say: #include setjmp.h . . . jmp_buf x; thereby still not knowing the size of a jmp_buf but generating correct portable slightly more efficient code. Also doing a better job of telling the C compiler that setjmp is being used -- rather than e.g. declaring setjmp ourselves, which I think is somewhat dubious, though faster to compile... I admit that the efficiency still stinks & we have much greater efficiency goals here. - Jay On Aug 3, 2015, at 7:05 PM, Antony Hosking wrote: > Remind me again what the issue is with the alloca approach? > >> On Aug 2, 2015, at 1:36 PM, Jay K wrote: >> >> I propose the following: >> >> >> - A flag to m3front indicating if the backend "knows" what a jmpbuf is. >> >> - extend M3CG.Type = { int8, uint8, ... addr ... struct } >> by adding a jmpbuf element >> >> >> - if the backend "knows" what a jmpbuf is, then, vs. the current code, >> the frontend shall just declare a jmpbuf per try, leaving the backend >> to size it, and w/o using the upcoming alloca approach. >> >> >> - if the backend does not "know" what a jmpbuf is, then, well, there >> is the current approach of making a sized struct, or my approach >> that is almost working where we use alloca. The frontend >> will not output cgtype.jmpbuf unless the backend "knows" what it is. >> >> >> ok? >> >> >> The information as to the backend's knowledge of a jmpbuf shall be >> yet another internal m3front flag, like what guides nested function >> output order. >> >> >> I was going to propose something slightly different, like >> m3front always use this new jmpbuf type, and existing backends >> can just change it "back" to struct. The problem with that approach.. >> it works easily with the current code, but does not address >> my upcoming alloca(sizeof_jmpbuf) change. >> >> >> In this way, what the C backend will do is: >> #include >> jmp_buf buf1; >> jmp_buf buf2; >> etc. >> >> >> leaving the size of the jmpbuf completely absent in the generated code, >> and allocating in the slightly more efficient way than alloca. >> >> >> Exception handling is special, so accomodating this seems reasonable. >> >> >> Yes, it might be temporary, replaced by libunwind stuff. >> >> >> Thank you, >> - Jay >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 09:36:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 07:36:21 +0000 Subject: [M3devel] Modula-2 parsing Message-ID: Procedure.m3: Do we need this?I know it isn't much, but remove it? ELSIF (cur.token = TK.tSEMI) THEN t.body := NEW (Body, self := t); ProcBody.Push (t.body); (* try accepting the Modula-2 syntax *) Error.ID (id, "expecting \'=\' before procedure body"); GetToken (); (* ; *) t.syms := Scope.PushNew (TRUE, id); t.block := BlockStmt.Parse (FALSE); t.fails := BlockStmt.ExtractFails (t.block); t.end_origin := Scanner.offset; final_id := MatchID (); IF (final_id # id) THEN Error.ID (id, "Initial name doesn\'t match final name"); END; Scope.PopNew (); ProcBody.Pop (); ELSE It always errors? But I guess it errors and checks nicer than it might otherwise? Why am I looking here? Tangential: I'm looking for where/how to model counting TRYs per "procedure"and doing the alloca at the start of a "procedure". "procedure"'s definition for my context is being worked on. It appears to be three things: 1) Things called "PROCEDURE". 2) FINALLY blocks, sometimes 3) "Module main" I was looking for what they have in common already, and it looks like maybe "ProcBody.Push". So I was looking all of them. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 12:02:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 10:02:03 +0000 Subject: [M3devel] declaring/initializing variables "anywhere" Message-ID: I had: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count;BEGIN which is fairly nice and elegant.I like the static type inference. but p could be NIL so I need like: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module: Module.T; alloca: CG.Proc; size: CG.Var; try_count: INTEGER;BEGIN IF p = NIL THEN RETURN END; module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count; double the lines, and a need to state types that I didn't have to state before, OR I can use WITH to get the "best" of both, but implied extra indentation. Pretty much all other mainstream languages in use today,except C89, are "better" than this, they all let youboth delay local variable declaration until arbitrarilyinto a function, but with implying that you should indent further,or close an extra block. Most mainstream languages also share Modula-3's type inferenceand let you omit the type, except C. Can we update Modula-3 here somehow? Or just use "with"? Proposals for a syntax that works well with the existing language and compiler? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 4 18:42:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 11:42:23 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: References: Message-ID: <55C0EB6F.7050704@lcwb.coop> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually involves finding one's way around in hundreds of thousands of lines of unfamiliar code. If you're exceptionally lucky, an eventual one-line change can be figured out after vetting a mere several tens of lines. Very often, it is more like a few hundreds, and not infrequently, even more. The average line of actively maintained code is written once and read something like seven times. (I don't remember where that number came from.) Speaking for myself, even most brand new code gets read at least seven times before it gets past my own testing, before it gets passed to anybody else to either use or further test. Moreover, the initial writer already has lots of context in his head that a maintainer has to ferret out. What I am leading to is that things should be *locally explicit*, far more than we usually see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches who have to come along and figure it out later to constantly locate, open, scan another source file, every fourth token they read, is penny wise and pound foolish. It's classic case of instant-gratification immaturity. Ensuring a ton of aggravation later, to save an ounce of effort now. It's just plain laziness. (more below:) On 08/04/2015 05:02 AM, Jay K wrote: > I had: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); So, coded like this, the poor schmuck who has to come along and figure this out will have to find Module.i3 and search it for declarations of Current, GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly more steps, as with alloca, whose type, as we will find out, requires going to yet another source file CG.i3. It can be even worse, e.g.: VAR V := Ptr^.Field[I].method(X); You have to track down the type of Ptr, to find Field, to find the type of its elements, to find method, to find its result type. Sometimes it's obvious from identifier names, but more often, it is not, or at least only too vaguely for serious software work. Is a module denoted by an abstract type? An integer numbering of modules? something else? Very often, there are two or three different types that denote a module or whatever. (An aside: In C++, things often get dramatically worse when coders, overly enamored with cleverness for its own sake, define implicit type conversions among these, so the poor maintainer has even less idea what is going on. Moreover, the conversions could be declared anywhere in the tens of transitively #included files, and there is no path leading to them, only exhaustive search through all the files. And this possibility has to be considered for every single assignment and actual parameter.) Instead, I always code cases like this by putting both the type and the initializer in the declaration: VAR module : Module.T := Module.Current (); I even often do it this way when the initializer is a literal. For one thing, I may want a subrange for the type of the variable. The exception is when the initializer itself also names the type. It's shorter, but at no loss of explicitness. VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > try_count := p.try_count; > BEGIN > > > which is fairly nice and elegant. > I like the static type inference. > > > but p could be NIL so I need like: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module: Module.T; > alloca: CG.Proc; > size: CG.Var; > try_count: INTEGER; > BEGIN > IF p = NIL THEN RETURN END; > module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); > try_count := p.try_count; > > > double the lines, and a need to state types that I didn't have to state before, > This will save wasted computation calling the functions, if it turns out p=NIL. Depending on the purpose of the code. In some code, I care about efficiency at this level. > > OR I can use WITH to get the "best" of both, but implied extra indentation. > If I were to propose any language change at all, it would be to allow a WITH binding to optionally specify the type too, for exactly the same reasons. WITH w : T = Var.method(x) DO ... Many a WITH statement has been far too hard to read, requiring the side trips of checking other source files to see what this value really is. > > > Pretty much all other mainstream languages in use today, > except C89, are "better" than this, they all let you > both delay local variable declaration until arbitrarily > into a function, but with implying that you should indent further, > or close an extra block. > To be clear, I definitely do not advocate allowing declarations and statements to be arbitrarily mixed in a block, BUT... if we did it, at least the decls would be syntactically explicit with a VAR, and we would not have to "define" our language with words like "if it looks like a declaration...". > > Most mainstream languages also share Modula-3's type inference > and let you omit the type, except C. > > > Can we update Modula-3 here somehow? > > > Or just use "with"? > > > Proposals for a syntax that works well with the existing language and compiler? > > > Thank you, > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 20:07:54 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 11:07:54 -0700 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <55C0EB6F.7050704@lcwb.coop> References: <55C0EB6F.7050704@lcwb.coop> Message-ID: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> But...Modula-3 already has this in VAR and WITH. Every language has this for temporaries: F1(F2() + F3()) Are these bad? Too much explicit detail can actually get in the way of reading the code. People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. - Jay On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually > involves finding one's way around in hundreds of thousands of lines of unfamiliar code. > If you're exceptionally lucky, an eventual one-line change can be figured out after vetting > a mere several tens of lines. Very often, it is more like a few hundreds, and not > infrequently, even more. > > The average line of actively maintained code is written once and read something like > seven times. (I don't remember where that number came from.) Speaking for myself, > even most brand new code gets read at least seven times before it gets past my own testing, > before it gets passed to anybody else to either use or further test. Moreover, the initial > writer already has lots of context in his head that a maintainer has to ferret out. > > What I am leading to is that things should be *locally explicit*, far more than we usually > see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches > who have to come along and figure it out later to constantly locate, open, scan another > source file, every fourth token they read, is penny wise and pound foolish. It's classic > case of instant-gratification immaturity. Ensuring a ton of aggravation later, to > save an ounce of effort now. It's just plain laziness. > > (more below:) > > On 08/04/2015 05:02 AM, Jay K wrote: >> I had: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); > > So, coded like this, the poor schmuck who has to come along and figure this > out will have to find Module.i3 and search it for declarations of Current, > GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly > more steps, as with alloca, whose type, as we will find out, requires > going to yet another source file CG.i3. It can be even worse, e.g.: > > VAR V := Ptr^.Field[I].method(X); > > You have to track down the type of Ptr, to find Field, to find the type of > its elements, to find method, to find its result type. > > Sometimes it's obvious from identifier names, but more often, it is not, or at > least only too vaguely for serious software work. Is a module denoted by an > abstract type? An integer numbering of modules? something else? Very often, > there are two or three different types that denote a module or whatever. > > (An aside: In C++, things often get dramatically worse when coders, overly > enamored with cleverness for its own sake, define implicit type conversions > among these, so the poor maintainer has even less idea what is going on. > Moreover, the conversions could be declared anywhere in the tens of transitively > #included files, and there is no path leading to them, only exhaustive search > through all the files. And this possibility has to be considered for every > single assignment and actual parameter.) > > Instead, I always code cases like this by putting both the type and the initializer > in the declaration: > VAR module : Module.T := Module.Current (); > > I even often do it this way when the initializer is a literal. For one thing, > I may want a subrange for the type of the variable. > > The exception is when the initializer itself also names the type. It's shorter, > but at no loss of explicitness. > > VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > >> try_count := p.try_count; >> BEGIN >> >> >> which is fairly nice and elegant. >> I like the static type inference. >> >> >> but p could be NIL so I need like: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module: Module.T; >> alloca: CG.Proc; >> size: CG.Var; >> try_count: INTEGER; >> BEGIN >> IF p = NIL THEN RETURN END; >> module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); >> try_count := p.try_count; >> >> >> double the lines, and a need to state types that I didn't have to state before, > > This will save wasted computation calling the functions, if it turns out p=NIL. > Depending on the purpose of the code. In some code, I care about efficiency > at this level. > >> >> OR I can use WITH to get the "best" of both, but implied extra indentation. > > If I were to propose any language change at all, it would be to allow a WITH > binding to optionally specify the type too, for exactly the same reasons. > > WITH w : T = Var.method(x) DO ... > > Many a WITH statement has been far too hard to read, requiring the side trips of > checking other source files to see what this value really is. > > > >> >> >> Pretty much all other mainstream languages in use today, >> except C89, are "better" than this, they all let you >> both delay local variable declaration until arbitrarily >> into a function, but with implying that you should indent further, >> or close an extra block. > > To be clear, I definitely do not advocate allowing declarations and statements > to be arbitrarily mixed in a block, BUT... if we did it, at least the decls > would be syntactically explicit with a VAR, and we would not have to "define" > our language with words like "if it looks like a declaration...". > >> >> Most mainstream languages also share Modula-3's type inference >> and let you omit the type, except C. >> >> >> Can we update Modula-3 here somehow? >> >> >> Or just use "with"? >> >> >> Proposals for a syntax that works well with the existing language and compiler? >> >> >> Thank you, >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:18:15 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C101E7.60106@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:30:47 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:30:47 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C104D7.4040601@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > It looks to me like the comment "accepting" is misleading, since it produces an error message. My guess is that this is a specialized syntax error recovery technique, designed to do a better job of continuing when this particular error occurs, probably based on experience. Modula-3 evolved from Modula-2+, which evolved from Modula-2, and I'll bet there was a lot of code converted from one language to another. This would have been a common leftover from that process. I'd change the comment, but otherwise leave it. > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > Are you counting TRYs in the front end, or on the back side of the cm3 IR? If the former, try looking at the way "fails" is collected from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, to a field of Procedure.T. But it has to come out of TryStmt.Parse, to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not a list? If the latter, aren't you already doing two passes over the IR? > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 21:02:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 14:02:53 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> References: <55C0EB6F.7050704@lcwb.coop> <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> Message-ID: <55C10C5D.40402@lcwb.coop> On 08/04/2015 01:07 PM, Jay wrote: > But...Modula-3 already has this in VAR and WITH. Huh? WithSt = WITH Binding {"," Binding} DO S END. Binding = Id "=" Expr. ^ no explicit type allowed here. > > Every language has this for temporaries: > > F1(F2() + F3()) > > Are these bad? > > Too much explicit detail can actually get in the way of reading the code. Yes, it is a balance. But much of the world is waaaay to far on the implicit side. Explicitly typing proper subexpressions would be far too tedious. But that is exactly why the rules for inferring should not be highly complex, multi-pronged, or depend on a huge amount of context. > > > People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. > Yes, if such tools were ubiquitous and dependable, it would go a long way to help. > > "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. > Yes, there is a tension in making things self-adapt to one-place changes. But if it's as complicated as is often the case, the time saved editing will be overwhelmed by the time figuring out what to change. And trusting a language/compiler to make the deduction that works right while the human can't understand it, or just doesn't have the time and gumption, is insane. > > - Jay > > On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > >> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually >> involves finding one's way around in hundreds of thousands of lines of unfamiliar code. >> If you're exceptionally lucky, an eventual one-line change can be figured out after vetting >> a mere several tens of lines. Very often, it is more like a few hundreds, and not >> infrequently, even more. >> >> The average line of actively maintained code is written once and read something like >> seven times. (I don't remember where that number came from.) Speaking for myself, >> even most brand new code gets read at least seven times before it gets past my own testing, >> before it gets passed to anybody else to either use or further test. Moreover, the initial >> writer already has lots of context in his head that a maintainer has to ferret out. >> >> What I am leading to is that things should be *locally explicit*, far more than we usually >> see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches >> who have to come along and figure it out later to constantly locate, open, scan another >> source file, every fourth token they read, is penny wise and pound foolish. It's classic >> case of instant-gratification immaturity. Ensuring a ton of aggravation later, to >> save an ounce of effort now. It's just plain laziness. >> >> (more below:) >> >> On 08/04/2015 05:02 AM, Jay K wrote: >>> I had: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >> >> So, coded like this, the poor schmuck who has to come along and figure this >> out will have to find Module.i3 and search it for declarations of Current, >> GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly >> more steps, as with alloca, whose type, as we will find out, requires >> going to yet another source file CG.i3. It can be even worse, e.g.: >> >> VAR V := Ptr^.Field[I].method(X); >> >> You have to track down the type of Ptr, to find Field, to find the type of >> its elements, to find method, to find its result type. >> >> Sometimes it's obvious from identifier names, but more often, it is not, or at >> least only too vaguely for serious software work. Is a module denoted by an >> abstract type? An integer numbering of modules? something else? Very often, >> there are two or three different types that denote a module or whatever. >> >> (An aside: In C++, things often get dramatically worse when coders, overly >> enamored with cleverness for its own sake, define implicit type conversions >> among these, so the poor maintainer has even less idea what is going on. >> Moreover, the conversions could be declared anywhere in the tens of transitively >> #included files, and there is no path leading to them, only exhaustive search >> through all the files. And this possibility has to be considered for every >> single assignment and actual parameter.) >> >> Instead, I always code cases like this by putting both the type and the initializer >> in the declaration: >> VAR module : Module.T := Module.Current (); >> >> I even often do it this way when the initializer is a literal. For one thing, >> I may want a subrange for the type of the variable. >> >> The exception is when the initializer itself also names the type. It's shorter, >> but at no loss of explicitness. >> >> VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; >> >>> try_count := p.try_count; >>> BEGIN >>> >>> >>> which is fairly nice and elegant. >>> I like the static type inference. >>> >>> >>> but p could be NIL so I need like: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module: Module.T; >>> alloca: CG.Proc; >>> size: CG.Var; >>> try_count: INTEGER; >>> BEGIN >>> IF p = NIL THEN RETURN END; >>> module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >>> try_count := p.try_count; >>> >>> >>> double the lines, and a need to state types that I didn't have to state before, >> >> This will save wasted computation calling the functions, if it turns out p=NIL. >> Depending on the purpose of the code. In some code, I care about efficiency >> at this level. >> >>> >>> OR I can use WITH to get the "best" of both, but implied extra indentation. >> >> If I were to propose any language change at all, it would be to allow a WITH >> binding to optionally specify the type too, for exactly the same reasons. >> >> WITH w : T = Var.method(x) DO ... >> >> Many a WITH statement has been far too hard to read, requiring the side trips of >> checking other source files to see what this value really is. >> >> >> >>> >>> >>> Pretty much all other mainstream languages in use today, >>> except C89, are "better" than this, they all let you >>> both delay local variable declaration until arbitrarily >>> into a function, but with implying that you should indent further, >>> or close an extra block. >> >> To be clear, I definitely do not advocate allowing declarations and statements >> to be arbitrarily mixed in a block, BUT... if we did it, at least the decls >> would be syntactically explicit with a VAR, and we would not have to "define" >> our language with words like "if it looks like a declaration...". >> >>> >>> Most mainstream languages also share Modula-3's type inference >>> and let you omit the type, except C. >>> >>> >>> Can we update Modula-3 here somehow? >>> >>> >>> Or just use "with"? >>> >>> >>> Proposals for a syntax that works well with the existing language and compiler? >>> >>> >>> Thank you, >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:47:16 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:47:16 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: <55C104D7.4040601@lcwb.coop> References: <55C104D7.4040601@lcwb.coop> Message-ID: Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. This change is for all backends. No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. If a backend wants to know about jmp_buf, that ability will be added soon thereafter. I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? There are obvious counter points but most seem moot these days. Yes, it means a larger working set and more likely-traced allocations. Ignoring the tracing though, this is how cm3cg has always worked. And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. At least one of the transforms M3C does could be moved to m3front -- removing unused locals. And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. Given the M3C code, this is easy at this point. Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. I'll send diffs later. I think I'm quite close. Thoughts? - Jay On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > On 08/04/2015 02:36 AM, Jay K wrote: >> Procedure.m3: >> >> Do we need this? >> I know it isn't much, but remove it? > > It looks to me like the comment "accepting" is misleading, since it produces > an error message. My guess is that this is a specialized syntax error recovery > technique, designed to do a better job of continuing when this particular > error occurs, probably based on experience. Modula-3 evolved from Modula-2+, > which evolved from Modula-2, and I'll bet there was a lot of code converted > from one language to another. This would have been a common leftover from > that process. > > I'd change the comment, but otherwise leave it. > >> >> ELSIF (cur.token = TK.tSEMI) THEN >> t.body := NEW (Body, self := t); >> ProcBody.Push (t.body); >> (* try accepting the Modula-2 syntax *) >> Error.ID (id, "expecting \'=\' before procedure body"); >> GetToken (); (* ; *) >> t.syms := Scope.PushNew (TRUE, id); >> t.block := BlockStmt.Parse (FALSE); >> t.fails := BlockStmt.ExtractFails (t.block); >> t.end_origin := Scanner.offset; >> final_id := MatchID (); >> IF (final_id # id) THEN >> Error.ID (id, "Initial name doesn\'t match final name"); >> END; >> Scope.PopNew (); >> ProcBody.Pop (); >> ELSE >> >> >> It always errors? >> >> But I guess it errors and checks nicer than it might otherwise? >> >> Why am I looking here? >> >> Tangential: >> >> I'm looking for where/how to model counting TRYs per "procedure" >> and doing the alloca at the start of a "procedure". > > Are you counting TRYs in the front end, or on the back side of the > cm3 IR? If the former, try looking at the way "fails" is collected > from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, > to a field of Procedure.T. But it has to come out of TryStmt.Parse, > to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not > a list? > > If the latter, aren't you already doing two passes over the IR? > >> >> "procedure"'s definition for my context is being worked on. >> It appears to be three things: >> 1) Things called "PROCEDURE". >> 2) FINALLY blocks, sometimes >> 3) "Module main" >> >> I was looking for what they have in common already, >> and it looks like maybe "ProcBody.Push". >> >> So I was looking all of them. >> >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:56:44 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:56:44 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: <55C104D7.4040601@lcwb.coop> Message-ID: Ps: yes just a count. A list could be useful but I achieved similar end another way. Each try records the current count as its index before incrementing it. Then we allocate an array of that many vars, which are initialized (perhaps too preemptively) to point into the bulk alloca. Then each TRY uses the corresponding variable. With a list I could poke each var into each Try*.T -- introducing a common base. The difference is negligible -- in m3front an extra array per procedure and extra indirection when compiling TRY, vs. walking the list. The generated code would be the same either way. The "spreading out of the division of labor" also similar. I introduced module Jmpbufs in m3front to hold most of this. I consider Marker, even existing functions in it, but it didn't seem to fit well. - Jay On Aug 4, 2015, at 12:47 PM, Jay wrote: > Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. > > > "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. > > > This change is for all backends. > > No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. > > If a backend wants to know about jmp_buf, that ability will be added soon thereafter. > > > I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). > > > So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? > > > There are obvious counter points but most seem moot these days. > > > Yes, it means a larger working set and more likely-traced allocations. > > > Ignoring the tracing though, this is how cm3cg has always worked. > > > And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. > > > At least one of the transforms M3C does could be moved to m3front -- removing unused locals. > > > And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. > > > Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. > > > Given the M3C code, this is easy at this point. > > Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. > > I'll send diffs later. I think I'm quite close. > > Thoughts? > > - Jay > > On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > >> On 08/04/2015 02:36 AM, Jay K wrote: >>> Procedure.m3: >>> >>> Do we need this? >>> I know it isn't much, but remove it? >> >> It looks to me like the comment "accepting" is misleading, since it produces >> an error message. My guess is that this is a specialized syntax error recovery >> technique, designed to do a better job of continuing when this particular >> error occurs, probably based on experience. Modula-3 evolved from Modula-2+, >> which evolved from Modula-2, and I'll bet there was a lot of code converted >> from one language to another. This would have been a common leftover from >> that process. >> >> I'd change the comment, but otherwise leave it. >> >>> >>> ELSIF (cur.token = TK.tSEMI) THEN >>> t.body := NEW (Body, self := t); >>> ProcBody.Push (t.body); >>> (* try accepting the Modula-2 syntax *) >>> Error.ID (id, "expecting \'=\' before procedure body"); >>> GetToken (); (* ; *) >>> t.syms := Scope.PushNew (TRUE, id); >>> t.block := BlockStmt.Parse (FALSE); >>> t.fails := BlockStmt.ExtractFails (t.block); >>> t.end_origin := Scanner.offset; >>> final_id := MatchID (); >>> IF (final_id # id) THEN >>> Error.ID (id, "Initial name doesn\'t match final name"); >>> END; >>> Scope.PopNew (); >>> ProcBody.Pop (); >>> ELSE >>> >>> >>> It always errors? >>> >>> But I guess it errors and checks nicer than it might otherwise? >>> >>> Why am I looking here? >>> >>> Tangential: >>> >>> I'm looking for where/how to model counting TRYs per "procedure" >>> and doing the alloca at the start of a "procedure". >> >> Are you counting TRYs in the front end, or on the back side of the >> cm3 IR? If the former, try looking at the way "fails" is collected >> from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, >> to a field of Procedure.T. But it has to come out of TryStmt.Parse, >> to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not >> a list? >> >> If the latter, aren't you already doing two passes over the IR? >> >>> >>> "procedure"'s definition for my context is being worked on. >>> It appears to be three things: >>> 1) Things called "PROCEDURE". >>> 2) FINALLY blocks, sometimes >>> 3) "Module main" >>> >>> I was looking for what they have in common already, >>> and it looks like maybe "ProcBody.Push". >>> >>> So I was looking all of them. >>> >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 5 09:17:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 07:17:42 +0000 Subject: [M3devel] names in (hopefully) upcoming changes Message-ID: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 11:15:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:15:22 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review Message-ID: Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 22.txt URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.i3 Type: application/octet-stream Size: 1138 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.m3 Type: application/octet-stream Size: 5020 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Aug 5 11:49:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:49:15 +0000 Subject: [M3devel] setjmp/jmpbuf declaration vs. #include setjmp.h Message-ID: fyi...having updated tools, I now get: cm3.d/Main.m3.c:757:1: warning: declaration of built-in function '_setjmp' requires inclusion of the header [-Wbuiltin-requires-header] which I think helps vindicate the upcoming addition of CGType.Jmpbuf. That is, we shouldn't declare setjmp ourselves.And therefore, we also can't pretend jmpbuf is just a struct with a size, but must use whatever setjmp.h says.I hope to fix this after the first round of "alloca(jmpbuf)". Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 5 17:54:48 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 5 Aug 2015 15:54:48 +0000 (UTC) Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: Message-ID: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Hi all:long time ago, a proposal for supporting more higher level R like Aimer Diwan et al, would benefit, support C -> Fortran compilation to further parallelize in big machines (Virtual Speculative Parallel Machine), see [1]:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear Anyway, hearing of what you are trying to do especially support OPenVMS, etc, is nice Thanks in advance [1]R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of Language Transcription?, en Languages and Compilers for Parallel Computing, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. El Mi?rcoles 5 de agosto de 2015 2:17, Jay K escribi?: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called?Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed? "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks,?- Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 20:53:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 18:53:32 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 Message-ID: Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 6 01:45:02 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> References: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Message-ID: Just wondering if the OpenVMS port is being used or if indeed its been tested. I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter On Thu, Aug 6, 2015 at 1:54 AM, Daniel Alejandro Benavides D. < dabenavidesd at yahoo.es> wrote: > Hi all: > long time ago, a proposal for supporting more higher level R like Aimer > Diwan et al, would benefit, support C -> Fortran compilation to further > parallelize in big machines (Virtual Speculative Parallel Machine), see [1]: > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear > > > > > Anyway, hearing of what you are trying to do especially support OPenVMS, > etc, is nice > > Thanks in advance > > [1] > R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of > Language Transcription?, en *Languages and Compilers for Parallel > Computing*, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. > Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. > > > > El Mi?rcoles 5 de agosto de 2015 2:17, Jay K > escribi?: > > > I need some names. > > C has a type "jmp_buf" -- with no "u" and with an underscore. > > > I need a module in m3front for counting tries and managing jmp_bufs. > I call it Jmpbufs. > It could be Jumpbufs. > Or JumpBuffers. > Perhaps that is too direct. > Or sjljeh (setjmp/longjmp exception handling) > Or worked into the existing Marker. > > > A "constant variable" in C to hold sizeof(jmp_buf). > So far this was called Csetjmp__Jumpbuf_size. > However this "constant variable" should never be used from Modula-3. > Putting in a Modula-3 interface (Csetjmp), with the two level naming and a > double underscore is unnecessary and I suggest should not be done. > > > This name is an "interface" between m3front and m3core, i.e. > m3core/src/unix/Common/Uconstants.c. > It is always statically linked. > > > Potential names here are: > > > m3_jmpbuf_size my favorate at the moment > jmpbuf_size > Csetjmp_Jumpbuf_size already present by this name, but can be changed > > > > "alloca" > > > This is (will be) part of the interface between m3front and every backend. > While it is *almost* pass-through, it isn't. > The C backend targeting non-Win32, non-VMS, can just pass through the name. > But every other case in every other backend must treat the name specially. > cm3cg has to convert it to "builtin_alloca". > LLVM probably is like cm3cg. > Win32 non-C has to change it to __chkstk or perhaps _chkstk. > Win32 C has to change it to _alloca. > OpenVMS has to change it to __ALLOCA or such. > That is, the symbol "alloca" is almost but not quite portable in C. > > > Again the function is never called from Modula-3 and need not be in an > interface. > > > Potential names here are: > > "alloca" what I'm using currently and remains my favorate. > "m3_alloca" > "m3_allocate_jmpbufs" > > > So far I'm calling it "alloca". > > > Thoughts/suggestions/criticisms? > > > Just wait for the larger diff? > > > As well, some of this could be more than "just random string function > names". > We could make them CONSTs in M3CG_Ops.i3. > > > We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" > and leave the "lowering" always to the backend -- this isn't likely, as > many backends can share part of the lowering. > > > But adding alloca to M3CG_Ops.i3 might be reasonable. > > > Thanks, > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:06:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:06:02 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 In-Reply-To: References: , Message-ID: So far the problems aren't in anything you did, and likely won't be. They are all subtle minor things. I gave a detailed commit. I can repeat it if needed. - Jay Subject: Re: [M3devel] cm3 on Mac OSX 10.10.4 From: hosking at purdue.edu Date: Thu, 6 Aug 2015 09:51:54 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Keep me posted on the problems. I did the original OS X port. On Aug 6, 2015, at 4:53 AM, Jay K wrote:Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:12:18 +0000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: , <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com>, Message-ID: I doubt the OpenVMS port is being used.It may or may not be complete. I managed to construct a Mac-hosted OpenVMS-targeted gcc.I built a native gcc with it. I didn't set the environment variables on the OpenVMS machinefor it to work. The biggest problem in all that is building the "sysroot".i.e. the headers and libraries. I built a bootstrap archive with cm3cg.I went to the trouble of building a ".com" file (makefile/scripty thing).So I might have gotten a "cm3" out of it. I don't remember. If someone has an OpenVMS machine available via ssh I might give it a try.I was working on an Alpha. I can see if it is still available. Also of note -- IA64 would need a little work, for all operating systems -- OpenVMS, HP-UX, Linux, FreeBSD, NT -- to scan the upper half of the stack. Thanks, - Jay From: peter.mckinna at gmail.com Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: Re: [M3devel] names in (hopefully) upcoming changes CC: jay.krell at cornell.edu; m3devel at elegosoft.com Just wondering if the OpenVMS port is being used or if indeed its been tested.I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 10:46:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 08:46:23 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review In-Reply-To: References: Message-ID: Any objections? criticisms? suggestions?I'd like to commit this. My bootstrapping procedure (boot1.py) precludes cm3 incompatible with m3core/libm3 -- so this is holding up moving to 10.10.4, unless I do some git gyrations. Bootstrapping really needs to produce the matching m3core and libm3, so target can have older/newer source. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: alloca(sizeof(jmp_buf)) changes for review Date: Wed, 5 Aug 2015 09:15:22 +0000 Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:31:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:31:14 +0000 Subject: [M3devel] alloca(jmpbuf_size) is in In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, Message-ID: I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:33:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:33:00 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? Message-ID: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:01:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:01:30 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , Message-ID: And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:02:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:02:42 +0000 Subject: [M3devel] updating m3cg? Message-ID: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:05:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:05:27 +0000 Subject: [M3devel] multi-threaded m3front? Message-ID: Is anyone interested in updating m3front to be multi-threaded? I haven't seen a single core system in a while. Surely each module can be compiled separately, possibly with some serialization around compiling interfaces? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 18:18:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 16:18:59 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> Message-ID: https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Aug 9 18:58:41 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 09 Aug 2015 09:58:41 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150809165841.BA38C1A2062@async.async.caltech.edu> I think this is a great idea. I made the back-end parallel already (at least for the version of the compiler that calls the gcc-based backend). To enable that you have to set "M3_PARALLEL_BACK" to the number of threads you want to spawn in your cm3.cfg . Mika Jay K writes: >--===============7278135725256123965== >Content-Type: multipart/alternative; > boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Is anyone interested in updating m3front to be multi-threaded? >I haven't seen a single core system in a while. >Surely each module can be compiled separately=2C possibly with some seriali= >zation around compiling interfaces? >Thanks=2C - Jay > > = > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
Is anyone interested in updating= > m3front to be multi-threaded?

I haven't seen a single c= >ore system in a while.

Surely each module can be c= >ompiled separately=2C possibly with some serialization around compiling int= >erfaces?

Thanks=2C
 =3B- Jay

= >
>= > >--_9e876ef9-e673-486b-befa-abff52e19a99_-- > >--===============7278135725256123965== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============7278135725256123965==-- From rodney_bates at lcwb.coop Sun Aug 9 21:05:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:05:57 -0500 Subject: [M3devel] updating m3cg? In-Reply-To: References: Message-ID: <55C7A495.6090409@lcwb.coop> Not high on my priority list, but I wouldn't object. On 08/08/2015 10:02 PM, Jay K wrote: > Is there any interesting in updating the gcc backend from 4.7, like to 5.2? > I'm guessing not really. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 9 21:14:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:14:48 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150809165841.BA38C1A2062@async.async.caltech.edu> References: <20150809165841.BA38C1A2062@async.async.caltech.edu> Message-ID: <55C7A6A8.2050805@lcwb.coop> On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > I think this is a great idea. I made the back-end parallel already (at > least for the version of the compiler that calls the gcc-based backend). I believe this works for any executable(s) separate from cm3 itself. > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > you want to spawn in your cm3.cfg . > > Mika > > Jay K writes: >> --===============7278135725256123965== >> Content-Type: multipart/alternative; >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> Is anyone interested in updating m3front to be multi-threaded? >> I haven't seen a single core system in a while. >> Surely each module can be compiled separately=2C possibly with some seriali= >> zation around compiling interfaces? >> Thanks=2C - Jay >> >> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
Is anyone interested in updating= >> m3front to be multi-threaded?

I haven't seen a single c= >> ore system in a while.

Surely each module can be c= >> ompiled separately=2C possibly with some serialization around compiling int= >> erfaces?

Thanks=2C
 =3B- Jay

= >>
>> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- >> >> --===============7278135725256123965== >> Content-Type: text/plain; charset="us-ascii" >> MIME-Version: 1.0 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: inline >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> --===============7278135725256123965==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 9 21:18:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:18:12 +0000 Subject: [M3devel] updating m3cg? In-Reply-To: <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> References: , <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> Message-ID: The level of change of supported targets would likely be small either way.Maybe a little lost, maybe a little gained.The C or C++ backend to me has the most reach. - Jay Subject: Re: [M3devel] updating m3cg? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:05:02 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu If it gets us running on newer targets then yes. Sent from my iPad On 9 Aug 2015, at 1:02 pm, Jay K wrote: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 21:20:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:20:01 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <55C7A6A8.2050805@lcwb.coop> References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: Right, agreed -- this should work for LLVM, and the C compiler.LLVM should be multi-threadable in-proc too. Fixing m3front itself -- we'd have to identify and deal with all the globals.For the alloca/setjmp work I put the "globals" in Module.T. - Jay > Date: Sun, 9 Aug 2015 14:14:48 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > I think this is a great idea. I made the back-end parallel already (at > > least for the version of the compiler that calls the gcc-based backend). > > I believe this works for any executable(s) separate from cm3 itself. > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > you want to spawn in your cm3.cfg . > > > > Mika > > > > Jay K writes: > >> --===============7278135725256123965== > >> Content-Type: multipart/alternative; > >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> Is anyone interested in updating m3front to be multi-threaded? > >> I haven't seen a single core system in a while. > >> Surely each module can be compiled separately=2C possibly with some seriali= > >> zation around compiling interfaces? > >> Thanks=2C - Jay > >> > >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
Is anyone interested in updating= > >> m3front to be multi-threaded?

I haven't seen a single c= > >> ore system in a while.

Surely each module can be c= > >> ompiled separately=2C possibly with some serialization around compiling int= > >> erfaces?

Thanks=2C
 =3B- Jay

= > >>
> >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- > >> > >> --===============7278135725256123965== > >> Content-Type: text/plain; charset="us-ascii" > >> MIME-Version: 1.0 > >> Content-Transfer-Encoding: 7bit > >> Content-Disposition: inline > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> --===============7278135725256123965==-- > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:19:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:19:27 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, Message-ID: The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:23:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:23:05 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? Message-ID: I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 07:02:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 05:02:57 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, , , Message-ID: Right -- I would have just said, add a name or such parameter to load/store, but I'm aware of LLVM GetElementPointer so came up with a name like it.But, I'm not sure we want a separate call -- GetElementPointer, followed by a load/store of whatever is stacked -- or build it into the load/store. I've always found the presence of the offset parameter in load/store "surprising".I'd'a'naively'thunk that'd a be a separate add operation.But I also suspect the original authors knew what they were doing and had possiblydone it several times already, and this parameter's presence was an easy lesson they learnedfrom experience. Heck, perhaps they had my current desires in mind? Then again, I was also /initially/ surprised by both the low level nature of the IR, all the layout done already, and that I was able to get it to work generating C. But ultimately it makes a lot of sense in the current form. If you look at the "level" and sophistication of M3x86 -- it doesn't do layout. Even M3C doesn't really do layout -- it echoes/renders the information in another form, leaving the C compiler to do the layout. > something that suits LLVM and C And presumably for the time being, gcc and M3x86? :) My actual extremely long term goal that I might never get to is that once we have something very portable -- C. And I guess something everyone likes -- LLVM.Perhaps try to write more of our backends. Like starting with M3x86.m3.I know I'll never achieve the reach of C/gcc/LLVM though. Merely targeting AMD64 across NT/Linux/Mac will be difficult -- three file formats and at least two ABIs. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:31:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Ideally, we would also converge on something that suits both LLVM and C.Your notion of get_member_reference is similar to LLVM getelementptr, which allows the backend to do layout for structs.It would be very nice if the front-end used this level of abstraction.I assume that is what you are talking about. On Aug 10, 2015, at 2:19 PM, Jay K wrote:The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay_______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 08:55:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 06:55:01 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , Message-ID: Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? > BYTESIZE(INTEGER) I know this is trickier but I consider it almost the same problem. Imagine I have this C code: typedef struct T1 { size_t a,b; } T1;T1* pt1; pt1->b (*size_t*)(((char*)pt1) + sizeof(size_t)) while the second is not idiomatic, and may skirt alignment/padding rules (imagine if a was a char and b a size_t), it is just about as portable. And you can write Modula-3 that way, with LOOPHOLE, ADR, BITESIZE, etc. I think "expressions" or "constant expressions" need to be maintained in "string" or "structural" form and passed to the backend..."string" would be, like, the original Modula-3."structural" would be parse them and make an "expression tree" with nodes like plus, minus, multiplication, division. I think, perhaps, the expression tree would be optional. Backends that ignore them could say so and save that cost. I believe most/all Modula-3 constant expressions render trivially in C. Really, like,RTIO.PutInt(BYTESIZE(INTEGER)); ideally translates to C involving the string/type "INTEGER", and not INT32 or INT64. I have to go. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:37:03 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 2:23 PM, Jay K wrote:I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. A cleaner solution is to use types and request the nth field of the type, etc. That way we don?t need symbolic names for the fields. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? That is a bit trickier if not impossible. Modula-3 inherently exposes these as compile-time integer constants. Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 19:34:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 17:34:34 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , , , Message-ID: > BYTESIZE(INTEGER) I have other ideas here. 1) Give up. Be like the 3.6 release and have target-specific boot archives, and target specific intermediate C, like current. 2) kinda of compile everything twice. The generated C would be: #ifdef _LP64_ // or such ... #else ... almost but not quite the same .. #endif To cut down cost somewhat, rereading dead code, it could be: foo32.c foo64.c foo.c #ifdef _LP64_ #include "foo64.c" #else #include "foo32.c" #endif 3) Sort of compile as always/only 64bit, when using C backend. Specifically, in m3middle/Target.m3, when using C backend, setup 64bit types. And in M3C output something like: #ifdef _LP64_ /* or such -- not quite*/ #define PAD64(x) /* nothing */ #else #define PAD64(x) int x##pad; #endif RECORD T1 = a: REF INTEGER; END; => struct T1 { INTEGER* a; PAD64(a); /* after any pointer field in a record */ }; This is an efficiency loss on 32bit platforms but maybe ok. It would gain a certain compatibility, i.e. in persistance formats. Local variables not in a record..would possibly be put into such a record, depending on if we could guarantee the entire pointer doesn't get overwritten..consider unusual valid Modula-3 such as: Cstring.memset(ADR(ptr), 0, BYTESIZE(ptr)); so we would have to pad out locals also. Possibly subject to #ifdef _LP64_. Big problem here would be interfacing with actual C code. For example Win32 and X Windows. If not for that, I was thinking this is a very viable approach. Given that, I'm not sure. Maybe as I was saying really kind of preserve all constant expressions symbolically and let the C backend rerender them.including stuff like: a: ARRAY [0.. BYTESIZE(INTEGER)] OF CHAR. or a: ARRAY [0.. BYTESIZE(INTEGER) * 2] OF CHAR. or a: ARRAY [0.. (BYTESIZE(INTEGER) = 32) * 4 + (BYTESIZE(INTEGER) = 64) * 8] OF CHAR. would have to be certain basically what is constant in Modula-3 is constant in C. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 17:02:37 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 4:55 PM, Jay K wrote:Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. The names are in the type. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? Yes. I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 11 00:13:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 10 Aug 2015 17:13:00 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: <55C921EC.7070303@lcwb.coop> On 08/09/2015 02:20 PM, Jay K wrote: > Right, agreed -- this should work for LLVM, and the C compiler. > LLVM should be multi-threadable in-proc too. > > Fixing m3front itself -- we'd have to identify and deal with all the globals. > For the alloca/setjmp work I put the "globals" in Module.T. > There are also sequence constraints on order of compilation that the back ends don't have. At a minimum, a unit can't be front-ended (yes, I am told that any word can be verbed, although I try to resist unless it has a really useful conciseness benefit. ;-|) until all its imported and exported interfaces have been front-ended, particularly, their .M3EXPORTS files created. > - Jay > > > > Date: Sun, 9 Aug 2015 14:14:48 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > > > I think this is a great idea. I made the back-end parallel already (at > > > least for the version of the compiler that calls the gcc-based backend). > > > > I believe this works for any executable(s) separate from cm3 itself. > > > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > > you want to spawn in your cm3.cfg . > > > > > > Mika > > > ..... -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Tue Aug 11 06:01:50 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:50 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 06:01:51 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:51 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 07:00:17 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 22:00:17 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: A more fruitful approach might be pipelining the compiler: - files enter the pipeline in reverse dependency order - have a thread (stage) to read those files into memory - have a thread to tokenize and parse the files and form the data structure - have a thread to do intermediary processing - have a thread to generate the output representation for the back end You'll only get a maximum 4x speedup and you'll only be as fast as the slowest thread, but you can reliably tune the divisions based on simple benchmarking of each thread. You're very likely to get somewhat close to the 4x speedup. This works best when there is a 1:1 between pipeline stages and cores. If you were ambitious you could attempt an 8 stage pipeline for 8 core processors. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Aug 11 09:13:25 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 11 Aug 2015 09:13:25 +0200 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Aug 11 12:47:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , Message-ID: I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 13:44:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , , Message-ID: Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 19:20:26 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 10:20:26 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 20:46:35 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 14:46:35 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811184635.GA3909@topoi.pooq.com> On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. I'm even pleased how *fast* the compiler is. If I could make it generate portable C, it would satisy most of the uses I can imagine for it, not just the uses I actually have. -- hendrik From jay.krell at cornell.edu Tue Aug 11 20:56:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 18:56:03 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, Message-ID: The problem with cross compiling is you need: a working cross-compiler for C/C++ a working cross-assembler a working cross-linker If you have those, it is fairly easy. * Getting those is often not easy. Esp. the C/C+ compiler as you need headers/libraries/startup code. "sysroot" or "buildroot" it is often called. * fairly easy: for NT386 -- nothing extra to do, the backend is in cm3 There are endianness bugs in mklib, but it works on the majority of systems -- little endian hosts. I can fix it. For the C backend -- nothing extra to do, the backend is in cm3. For the gcc-based backend -- fairly easy -- you need to rebuild a different cm3cg and point the config at it. It kind of is already setup to work. For the LLVM backend, probably also easy. I think on Debian and Gentoo, cross C compilers at least to every Linux architecture are readily available. But I don't know about otherwise. In as much as the compiler is gcc, the assembler GNU as, and the linker GNU ld, that gets you far, but you still need the headers/libraries/startup code. If your C compiler is LLVM, that might also help for compiler/assembler/linker. But again, the headers/libraries/startup code. There are various systems out there that try to automate this but I don't have much experience with them, and they often are slightly different than what we want. Otherwise, what we have setup is we cross compile to assembly files, copy them to the target, which typically does have the compiler/assembler/linker, and finish there. What remains there is that I only have this automated to build cm3. It should also at least also build m3core and libm3 statically. And therefore reach approximately parity with the 3.6 "boot" archives and install process. Has anyone here installed 3.6? And consider there method a decent goal and stopping point? Today is slightly different since quake has been rewritten from C to Modula-3 in the 4.0 timeframe. And *possibly* boot archives should contain everything -- going beyond what 3.6 did. This is just a matter of work in scripts/python/pylib.py. Copying stuff into sub directories and generating a hierarhical recursive or not make system in there. My make skills aren't great, and I've been hung up on details like depending on GNU make or trying to be more portable. On the threading area, I believe there is a simple almost ideal design. - parse and mostly "execute" all the quake code almost unchanged, single thread - difference starts at about the last line of quake where it says library or program - possibly though queueing every file to a "prefetcher" thread, it just reads every file into a reused buffer and throws out the data, populating the operating system's file system cache; or possibly mmaping every file and keeping it mapped, and touching every page; doing this in a somewhat thread aware/safe way for the upcoming actual accesses - once quake is done, one of two choices: simple but not optimal: do a single threaded parse of every interface less simple: parsing here can be in parallel, depending on the dependency graph; you'd just start up a thread per interface, but block on parsing its dependencies; as you get away from the root (RT0.i3), the tree should get ever wider You would either manually throttle the number of threads, or rely on an underlying threadpool. NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be simplar. Maybe that wasn't good. Simpler as in, including, thread id is now just the underlying Win32 thread id. It wasn't before. Win32 has had a thread pool since circa Windows 2000. That might be profitable to use. - once parsing interfaces is done, I believe codegen of every interface, and parsing and codegen of every module can proceed with arbitrary parallelism Fetching interface/module contents might synchronize with the prefetcher depending on which of the two approaches -- if the prefetcher is just prefetch and discard, populating the file system cache, then later compilation just refetches oblibviously/unchanged. If the prefetcher thread mmaps and keeps everything, then serialize with it. Also, you might have multiple prefetcher threads. What you really want is..difficult. You want a prefetcher thread per spindle. How to count them up? If you have an SSD, then prefetching might not buy much and just forget about it. The thing is, if you have spindles, this might be the most gain, and if you have an SSD, the cost might be negligible or slightly beneficial. I/O is so often the bottleneck, at least in the days of rotating storage. In the event that the file system cache is small, or the source for a package really large, prefetching can be counterproductive -- moving stuff through the cache only to flush it before it is used and fetching it a second time. I expect on most systems this is not a concern though. One of the pieces of work here, I believe, is to move most globals in m3front to be in Module.T. This would actually make things cleaner imho. Yes, it consolidates knowledge that you might want distributed. And accessing context'ed data is slightly slower than globals. But imho globals are bad and everything should be placed in *some* context other than the process or thread. I would not advocate compiling functions within a module in parallel, only modules. Similarly, code generators must have no globals, and put all the state in the cg object. - Jay Date: Tue, 11 Aug 2015 10:20:26 -0700 From: lists at darko.org To: wagner at elegosoft.com CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] multi-threaded m3front? I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 21:04:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 19:04:23 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811184635.GA3909@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: There are clearly three speeds to the compiler in my informal observation. The NT386 backend is fastest. The m3cg backend is medium. The C backed is a little slower. But if this can all be divided by number of cores, and the I/O overlapped,they might all be fast. :) I would like to write more integrated backends, but the C backend needs a bit more work,and we want to remove setjmp/longjmp for most targets still,and the existing M3x86.m3 has a terrible debugging story -- only line numbers. - Jay > Date: Tue, 11 Aug 2015 14:46:35 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > I couldn't agree with you more. I think being able to compile or cross > > compile the system without spending hours (or days) hacking > > scripts/environments would be a huge step forward for the project. Or > > having compiler binaries for more than two platforms (four if you count > > different word sizes). Meanwhile I've never heard anyone complain that the > > compiler is too slow. > > I'm even pleased how *fast* the compiler is. > > If I could make it generate portable C, it would satisy most of the > uses I can imagine for it, not just the uses I actually have. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 21:32:37 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 12:32:37 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: Jay I can see the complexity of the task but can we have an option for users who cannot invest time in this? What would be great is if you could set up the scripts in a static environment available to all that builds various platforms and back ends that are requested or that you feel like maintaining. Having it so it could auto build based on a particular version from github would be optimal. Surely this would help with developing and debugging your scripts and backend and would solve the problem of other people getting the environment right. I have dozens of servers (x86 and ARM) with ample memory and storage plus PDUs that allow powering the servers up and down over the net. If you want to set up the environments and scripts I can setup the access and install Mac OS and whatever Unixes on some of them. You can have carte blanche as to what you do with them, I don't care as long as working compilers pop out somewhere. Maybe others are interested in helping with this too. On Tue, Aug 11, 2015 at 11:56 AM, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ > a working cross-assembler > a working cross-linker > > > If you have those, it is fairly easy. * > Getting those is often not easy. > Esp. the C/C+ compiler as you need headers/libraries/startup code. > "sysroot" or "buildroot" > it is often called. > > > * fairly easy: > for NT386 -- nothing extra to do, the backend is in cm3 > There are endianness bugs in mklib, but it works on the majority of > systems -- little endian hosts. I can fix it. > > For the C backend -- nothing extra to do, the backend is in cm3. > > For the gcc-based backend -- fairly easy -- you need to rebuild a > different cm3cg > and point the config at it. It kind of is already setup to work. > > > For the LLVM backend, probably also easy. > > > I think on Debian and Gentoo, cross C compilers at least to every Linux > architecture are readily available. > But I don't know about otherwise. > > > In as much as the compiler is gcc, the assembler GNU as, and the linker > GNU ld, that gets you far, > but you still need the headers/libraries/startup code. > > > If your C compiler is LLVM, that might also help for > compiler/assembler/linker. > But again, the headers/libraries/startup code. > > > There are various systems out there that try to automate this but I > don't have much experience > with them, and they often are slightly different than what we want. > > > Otherwise, what we have setup is we cross compile to assembly files, copy > them to the target, > which typically does have the compiler/assembler/linker, and finish there. > > > What remains there is that I only have this automated to build cm3. > It should also at least also build m3core and libm3 statically. And > therefore reach > approximately parity with the 3.6 "boot" archives and install process. > > > Has anyone here installed 3.6? And consider there method a decent goal > and stopping point? > Today is slightly different since quake has been rewritten from C to > Modula-3 in the 4.0 timeframe. > > > And *possibly* boot archives should contain everything -- going beyond > what 3.6 did. > > > > This is just a matter of work in scripts/python/pylib.py. > Copying stuff into sub directories and generating a hierarhical recursive > or not make system in there. > My make skills aren't great, and I've been hung up on details like > depending on GNU make or trying to > be more portable. > > > On the threading area, I believe there is a simple almost ideal design. > > - parse and mostly "execute" all the quake code almost unchanged, single > thread > > - difference starts at about the last line of quake where it says library > or program > > - possibly though queueing every file to a "prefetcher" thread, it just > reads > every file into a reused buffer and throws out the data, populating the > operating > system's file system cache; or possibly mmaping every file and keeping > it mapped, > and touching every page; doing this in a somewhat thread aware/safe way > for the > upcoming actual accesses > > - once quake is done, one of two choices: > > simple but not optimal: do a single threaded parse of every interface > less simple: parsing here can be in parallel, depending on the dependency > graph; > you'd just start up a thread per interface, but block on parsing its > dependencies; > as you get away from the root (RT0.i3), the tree should get ever wider > > > You would either manually throttle the number of threads, or rely on an > underlying threadpool. > > > NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be > simplar. > Maybe that wasn't good. Simpler as in, including, thread id is now just > the underlying > Win32 thread id. It wasn't before. > > Win32 has had a thread pool since circa Windows 2000. That might be > profitable to use. > > - once parsing interfaces is done, I believe codegen of every interface, > and parsing and codegen > of every module can proceed with arbitrary parallelism > > > Fetching interface/module contents might synchronize with the prefetcher > depending > on which of the two approaches -- if the prefetcher is just prefetch and > discard, > populating the file system cache, then later compilation just refetches > oblibviously/unchanged. > If the prefetcher thread mmaps and keeps everything, then serialize with > it. > > > Also, you might have multiple prefetcher threads. What you really want > is..difficult. > You want a prefetcher thread per spindle. How to count them up? > If you have an SSD, then prefetching might not buy much and just forget > about it. > The thing is, if you have spindles, this might be the most gain, and if > you have an SSD, > the cost might be negligible or slightly beneficial. > > > I/O is so often the bottleneck, at least in the days of rotating storage. > > > In the event that the file system cache is small, or the source for a > package > really large, prefetching can be counterproductive -- moving stuff > through the cache > only to flush it before it is used and fetching it a second time. > > > I expect on most systems this is not a concern though. > > > One of the pieces of work here, I believe, is to move most globals in > m3front > to be in Module.T. This would actually make things cleaner imho. > Yes, it consolidates knowledge that you might want distributed. > And accessing context'ed data is slightly slower than globals. > But imho globals are bad and everything should be placed in *some* context > other than the process or thread. > > > I would not advocate compiling functions within a module in parallel, > only modules. > > > Similarly, code generators must have no globals, and put all the state in > the cg object. > > > - Jay > > > > > ------------------------------ > Date: Tue, 11 Aug 2015 10:20:26 -0700 > From: lists at darko.org > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] multi-threaded m3front? > > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. > > But of course everyone has different priorities, different interests and > different opinions in a volunteer project so any discussion on this subject > will inevitably boil down to "he who writes the code determines the > priorities." > > > > > On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner > wrote: > > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 22:57:21 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 16:57:21 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811205721.GA9712@topoi.pooq.com> On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ Slightly clumsier is just haveing a C/C++ compiler on the target system. But it's often easier to get that than a cross-compiler. -- hendrik From rodney_bates at lcwb.coop Tue Aug 11 23:01:40 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 11 Aug 2015 16:01:40 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: <55CA62B4.5000205@lcwb.coop> In my informal observation, it is additionally true that: On 08/11/2015 02:04 PM, Jay K wrote: > There are clearly three speeds to the compiler in my informal observation. > > The NT386 backend is fastest. > The m3cg backend is medium. > The C backed is a little slower. > gcc compiling m3cc is quite a bit slower than cm3 with m3cc backend. gcc compiling llvm/clang is a lot slower than that. clang compiling individual C++ units is noticeably slower than gcc on the same units. All of which is leading to my feeling that compiling Modula-3 code is already so much faster than compiling C or C++, that it's a pretty minor need to speed it up even more. > > But if this can all be divided by number of cores, and the I/O overlapped, > they might all be fast. :) > > > I would like to write more integrated backends, but the C backend needs a bit more work, > and we want to remove setjmp/longjmp for most targets still, > and the existing M3x86.m3 has a terrible debugging story -- only line numbers. > > > - Jay > > > > > Date: Tue, 11 Aug 2015 14:46:35 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > > I couldn't agree with you more. I think being able to compile or cross > > > compile the system without spending hours (or days) hacking > > > scripts/environments would be a huge step forward for the project. Or > > > having compiler binaries for more than two platforms (four if you count > > > different word sizes). Meanwhile I've never heard anyone complain that the > > > compiler is too slow. > > > > I'm even pleased how *fast* the compiler is. > > > > If I could make it generate portable C, it would satisy most of the > > uses I can imagine for it, not just the uses I actually have. > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 12 00:33:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811205721.GA9712@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , <20150811205721.GA9712@topoi.pooq.com> Message-ID: Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:37:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:37:10 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , Message-ID: It should be ok now. Have you been using Modula-3 on it? How?I can imagine a preexisting path of low resistance was to use gcc everywhere and reject Apple's tools.gcc as the "assembler driver" would have worked as it'd pick the working assembler instead of the LLVM assembler.As well as using (/usr/bin/)as directly, which is what I do now (it actually calls out to yet another as, but I didn'twant to hardcode that path -- /applications/xcode and all that .. which reminds me I need to uninstall xcodeand see what things do -- as there is an option to install command line tools w/o xcode).. But Apple's tools should work now. - Jay > Subject: Re: [M3devel] Mac 10.10? > From: hosking at purdue.edu > Date: Wed, 12 Aug 2015 07:49:58 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I am and I suspect it'll be a popular target! > > Sent from my iPad > > > On 11 Aug 2015, at 9:44 pm, Jay K wrote: > > > > Anyone else running 10.10? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:44:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:44:19 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , , , , <20150811205721.GA9712@topoi.pooq.com>, Message-ID: I suspect using Debian or Ubuntu is the quickest route to a suite of cross tools. https://wiki.debian.org/CrossToolchains at least to Linux, not necessarily to BSD/NT/Solaris/Mac. We could add "support" to that -- having the default config noticethat host = is debian or ubuntu and host != target and target is linux and invoke those toolsand tell you to apt-get install whatever if they aren't present.. But I still think the "fair" / "flat" / "consistent" story is to make boot archives something like 3.6and user then completes the build on the target with native tools. We'd distribute like three archives per target: boot -- no binaries, all assembly/C source, to just cm3/libm3/m3core (not just cm3 like current) binmin -- binaries -- cm3, cm3cg (except for NT386), m3core (maybe just static), libm3 (ditto) binall -- all binaries, nothing to build The C source might be somewhat target-independent, but getting it to be fully target-independentI can't yet commit to. binmin has a pretty narrow use case, maybe strike it - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: Re: [M3devel] multi-threaded m3front? Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 08:14:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 06:14:02 +0000 Subject: [M3devel] cm3cg coverage/profiling features? Message-ID: Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 12 08:40:40 2015 From: wagner at elego.de (Olaf Wagner) Date: Wed, 12 Aug 2015 08:40:40 +0200 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812084040.665db643b5756d863fb2eb2a@elego.de> On Wed, 12 Aug 2015 06:14:02 +0000 Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. I'm not sure what exactly you're going to remove, but keeping coverage and profiling support would be great. Though it's been years since I used them... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From hendrik at topoi.pooq.com Wed Aug 12 14:09:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 12 Aug 2015 08:09:19 -0400 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812120918.GA578@topoi.pooq.com> On Wed, Aug 12, 2015 at 06:14:02AM +0000, Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. > Thank you, - Jay Didn't know they were there. How to use them? -- hendrik From lists at darko.org Wed Aug 12 17:17:54 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 08:17:54 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: Why does it have to be removed? Is there some pressing reason that justifies removing functionality? How does it improve the compiler? Also, how does asking in the mailing list justify its removal? Not all users follow the mailing list, and future users do not get a say. On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > Does anyone use the coverage or profiling or > optimization-via-profiling-feedback features of cm3cg? > I'm removing dead stuff. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 12 17:21:55 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 12 Aug 2015 08:21:55 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812152155.6D9891A2062@async.async.caltech.edu> Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- From jay.krell at cornell.edu Wed Aug 12 17:33:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 15:33:24 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: , , <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: gcc is so big, I like to trim it down. For example, I removed the C preprocessor, libgcc (adding back a subsetted alternative when using Sun CC, in another place), the driver, most of the frontends, LTO which I assume nobody uses, all the dependency on gmp/mpfr/mpc, lots of "builtins" that we don't expose so you can't use in Modula-3, etc. This is stuff in gcc, not the stuff in cm3/m3front. But arguably we could (have) left it all alone, just apply a minimal patch. Note that anything here is going to be somewhat specific to this backend.Though LLVM may very well have similar functionality.The integrated backend does not -- though for coverage there exists post-link instrumentation options at least on Windows in Visual Studio.The C backend could get it via the underlying C compiler, e.g. gcc. Someone should try them out and report.The config files have long offered profile/-pg.I'll leave it for now. Thanks, - Jay > To: lists at darko.org > Date: Wed, 12 Aug 2015 08:21:55 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing rea= > >son that justifies removing functionality? How does it improve the compiler= > >?

Also, how does asking in the mailing list justify its = > >removal? Not all users follow the mailing list, and future users do not get= > > a say.

>">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= > >t; wrote:
> .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= > >uff.

Thank you,
=C2=A0- Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > >=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 18:11:10 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 09:11:10 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 19:41:26 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 17:41:26 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, Message-ID: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 20:21:42 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 11:21:42 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top; TYPE T <: ROOT; END Top. INTERFACE Pub; IMPORT Top; TYPE TPub = OBJECT [...] END; REVEAL Top.T <: TPub; END Pub. INTERFACE Pvt; IMPORT Top, Pub; TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt; END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 20:56:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 18:56:12 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , , Message-ID: > Error recovery is hard but the original authors laboured to implement it and it worked Within a module it is unbounded problem and such labor will only slightly work. That said, I know of no changes within modules or across modules wrt error recovery.We can/should investigate and fix. I don't think a compiler switch is a good idea here, as we should have a smaller matrixof behaviors and the behavior you desire is probably the one and only way it should be. Making switches for too many things is its own madness. Nobody runs the same thingand the test matrix explodes. Across modules we should probably just proceed, just being sure to error at the end (i.e. so we don't ship, and maybe so scripts don't proceed, thoughactually, scripts probably should proceed, but again error at the end -- builds tend to contain highly independent graph pieces, even if thereare some heavily shared nodes at the root like m3core) There are at least two schools of thought in software in general. There is "fail fast" and "muddle along". Generally older thinking is "muddle along", make a best effort attempt to work in the face of errors. Newer thinking is "fail fast", and generally acknowledging that once there is an error, some?many?most?all "bets"are off and best to stop, vs. doing the wrong thing. But there is a middle ground, which is to try to think about dependencies and if errors in one placereally affect another. Often times, for a simple example, iterations of a loop are independent and failurewithin one shouldn't stop the loop. Imagine you specify to copy many files, but a few fail. Should you stop the copy at the first failure?Should you rollback the earlier successful copies? Should you have some level of "transactionality" so that no intermediate state is visible unlessthe entire operation is guanteed to succeed? Anyway, compilers, even within a module, seem popular to place in the middle and muddle along.Their output is eventuall mission critical, but how they handle errors is not.People like to get as many errors as possible, fix them all, try again.Instead of the compiler stopping at the first. You know, to try to divide the quadratic natureof fixing errors down to almost linear. But it depends on the nature of the error. "breaking" a declaration is presumably removing it or changing it to a different valid declaration.Changing it to a declaration that doesn't parse is risky -- you have to assume error recovery works a certain way. I also discovered an incrementality problem -- if you move a module across packages.That is probably not a new regression though. I don't think we are losing functionality and all changes still come through m3commit. Imho review can occur before or after change, but I know, not everyone in the world agrees.Many development environments require review before commit. There is the matter of if rate of change exceeds rate of review and needs to be throttled down,but rate of change is really quite low (even if rate of review is even lower). - Jay Date: Wed, 12 Aug 2015 11:21:42 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top;TYPE T <: ROOT;END Top. INTERFACE Pub;IMPORT Top;TYPE TPub = OBJECT [...] END;REVEAL Top.T <: TPub;END Pub. INTERFACE Pvt;IMPORT Top, Pub;TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt;END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *)REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END;BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 21:41:43 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 12:41:43 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The error recovery doesn't matter. For instance I just rename a declaration so it no longer exists. The compiler will then produce an error for each usage of that symbol, saying it's no longer found, and it will report it at least once for every module where it used (I've noticed that the compiler will often only report one error if it is repeated multiple times in a module). I'm not sure the rate of change is that low. I think the best idea would be put changes in branches so each set can be reviewed and tested before being merged. This would also allow people to choose what they want in their build. With the new GitHub repository this should be a lot easier than with CVS. On Wed, Aug 12, 2015 at 11:56 AM, Jay K wrote: > > Error recovery is hard but the original authors laboured to implement > it and it worked > > > Within a module it is unbounded problem and such labor will only slightly > work. > > > That said, I know of no changes within modules or across modules wrt error > recovery. > We can/should investigate and fix. > > I don't think a compiler switch is a good idea here, as we should have a > smaller matrix > of behaviors and the behavior you desire is probably the one and only way > it should be. > > Making switches for too many things is its own madness. Nobody runs the > same thing > and the test matrix explodes. > > > Across modules we should probably just proceed, just being sure to error > at the end (i.e. so we don't ship, and maybe so scripts don't proceed, > though > actually, scripts probably should proceed, but again error at the end -- > builds tend to contain *highly* *independent* graph pieces, even if there > are some heavily shared nodes at the root like m3core) > > > There are at least two schools of thought in software in general. > > There is "fail fast" and "muddle along". > > > Generally older thinking is "muddle along", make a best effort attempt to > work in the face of errors. > > > Newer thinking is "fail fast", and generally acknowledging that once there > is an error, some?many?most?all "bets" > are off and best to stop, vs. doing the wrong thing. > > > But there is a middle ground, which is to try to think about dependencies > and if errors in one place > really affect another. Often times, for a simple example, iterations of a > loop are independent and failure > within one shouldn't stop the loop. > > > Imagine you specify to copy many files, but a few fail. Should you stop > the copy at the first failure? > Should you rollback the earlier successful copies? > > Should you have some level of "transactionality" so that no intermediate > state is visible unless > the entire operation is guanteed to succeed? > > > Anyway, compilers, even within a module, seem popular to place in the > middle and muddle along. > Their output is eventuall mission critical, but how they handle errors is > not. > People like to get as many errors as possible, fix them all, try again. > Instead of the compiler stopping at the first. You know, to try to divide > the quadratic nature > of fixing errors down to almost linear. But it depends on the nature of > the error. > > > "breaking" a declaration is presumably removing it or changing it to a > different valid declaration. > Changing it to a declaration that doesn't parse is risky -- you have to > assume error recovery works a certain way. > > > I also discovered an incrementality problem -- if you move a module across > packages. > That is probably not a new regression though. > > > I don't think we are losing functionality and all changes still come > through m3commit. > > Imho review can occur before or after change, but I know, not everyone in > the world agrees. > Many development environments require review before commit. > > > There is the matter of if rate of change exceeds rate of review and needs > to be throttled down, > but rate of change is *really quite low *(even if rate of review is even > lower). > > > - Jay > > > ------------------------------ > Date: Wed, 12 Aug 2015 11:21:42 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > The larger point is that is that changes are being made to the compiler > ad-hoc. There seems to be no consideration of the consequence of those > changes and how it affects users. These changes are being made on the trunk > instead of a branch and it may be hard to untangle them. If we're losing > functionality accidently then something is wrong with the development > process. > > On this issue, ignoring cascading errors is perfectly legitimate but so is > not ignoring them. I (used to) purposely break a declaration so see the > consequences of it throughout the code and to reveal where the declaration > was used. It's also very slow having to recompile after fixing a change in > each client module to reveal the next place it needs fixing, just because > it's in a different file. > > Error recovery is hard but the original authors laboured to implement it > and it worked, it shouldn't be lost. At the very least a compiler switch is > in order here. > > Either way this seems to be a bug because the compiler is also failing to > track dependencies in some causes causing files not to be recompiled. For > example it seems to be happening somewhere around using this sort of idiom: > > INTERFACE Top; > TYPE T <: ROOT; > END Top. > > INTERFACE Pub; > IMPORT Top; > TYPE TPub = OBJECT [...] END; > REVEAL Top.T <: TPub; > END Pub. > > INTERFACE Pvt; > IMPORT Top, Pub; > TYPE TPvt = Pub.TPub OBJECT [...] END; > REVEAL Top.T <: TPvt; > END Pvt. > > MODULE Top; > IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) > REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; > BEGIN > END Top. > > I suspect the problem has something to do with the "injection" of the > partial revelations through importing interfaces. > > On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:27:36 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:27:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: > I wasn?t aware this had changed. > When was this regression introduced? > > On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:29:03 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:29:03 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > I'm using: > > Critical Mass Modula-3 version d5.9.0 > > last updated: 2010-07-21 > > compiled: 2014-08-27 19:15:04 > > configuration: /usr/local/cm3/bin/cm3.cfg > > host: AMD64_DARWIN > > target: AMD64_DARWIN > > > > On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: > >> I wasn?t aware this had changed. >> When was this regression introduced? >> >> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >> >> For instance, cm3 used to keep on compiling files after errors were >> found. Now it seems to stop after one module. That loss of functionality >> seriously reduces productivity. I couldn't find any switch to reverse the >> change. When was this change decided? Is there a way to restore it? >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 13 03:00:22 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do. Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: > It's the one from the Releases section on GitHub. > > On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >> wrote: >> >>> I wasn?t aware this had changed. >>> When was this regression introduced? >>> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >>> >>> >>> >> > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 04:05:17 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 19:05:17 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Sorry, I was distracted by a phone call before - that is the compiler that exhibits the behaviour. Previous to that I was using something that was compiled in 2010, so not very helpful for pinpointing. On 8/12/15, Antony Hosking wrote: > Are you saying that the behavior was still working as of d5.9.0? > Or is that the version you are currently using in which it no longer does > what you want? > We should be able to figure out what commit broke things if we can pinpoint > a time when it worked. > >> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >> >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > > wrote: >> I wasn?t aware this had changed. >> When was this regression introduced? >> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >> > wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >> >> > > From jay.krell at cornell.edu Thu Aug 13 05:43:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Aug 2015 20:43:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> Message-ID: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? > >> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >> >> Sorry, I was distracted by a phone call before - that is the compiler >> that exhibits the behaviour. Previous to that I was using something >> that was compiled in 2010, so not very helpful for pinpointing. >> >> On 8/12/15, Antony Hosking wrote: >>> Are you saying that the behavior was still working as of d5.9.0? >>> Or is that the version you are currently using in which it no longer does >>> what you want? >>> We should be able to figure out what commit broke things if we can pinpoint >>> a time when it worked. >>> >>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>> >>>> I'm using: >>>> >>>> Critical Mass Modula-3 version d5.9.0 >>>> >>>> last updated: 2010-07-21 >>>> >>>> compiled: 2014-08-27 19:15:04 >>>> >>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>> >>>> host: AMD64_DARWIN >>>> >>>> target: AMD64_DARWIN >>>> >>>> >>>> >>>> >>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>> > wrote: >>>> I wasn?t aware this had changed. >>>> When was this regression introduced? >>>> >>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>> > wrote: >>>>> >>>>> For instance, cm3 used to keep on compiling files after errors were >>>>> found. Now it seems to stop after one module. That loss of functionality >>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>> change. When was this change decided? Is there a way to restore it? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 10:50:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 08:50:57 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , Message-ID: Note that this probably isn't the coverage/profiling I was talking about.I was talking about specifically in the gcc backend. I wonder if coverage -- a boolean per basic block; or maybe a counter,is cheap enough that we can't have it always enabled. And then some way of exposing it. I don't like having to recompile/relink everything multiple differentways to get all the features. - Jay From: peter.mckinna at gmail.com Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: Re: [M3devel] cm3cg coverage/profiling features? To: lists at darko.org CC: hosking at purdue.edu; m3devel at elegosoft.com; jay.krell at cornell.edu Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do.Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: I wasn?t aware this had changed.When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 11:23:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 09:23:46 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , ,,, , , , , , Message-ID: Eh, seems like some off by one in my testing -- things work now, including X apps. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: Re: [M3devel] Mac 10.10? Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 19:34:26 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Aug 2015 10:34:26 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> Message-ID: Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: > "Yes". > > I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. > > > I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) > > > - Jay > > On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > >> Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? >> >>> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >>> >>> Sorry, I was distracted by a phone call before - that is the compiler >>> that exhibits the behaviour. Previous to that I was using something >>> that was compiled in 2010, so not very helpful for pinpointing. >>> >>> On 8/12/15, Antony Hosking wrote: >>>> Are you saying that the behavior was still working as of d5.9.0? >>>> Or is that the version you are currently using in which it no longer does >>>> what you want? >>>> We should be able to figure out what commit broke things if we can pinpoint >>>> a time when it worked. >>>> >>>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>>> >>>>> I'm using: >>>>> >>>>> Critical Mass Modula-3 version d5.9.0 >>>>> >>>>> last updated: 2010-07-21 >>>>> >>>>> compiled: 2014-08-27 19:15:04 >>>>> >>>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>>> >>>>> host: AMD64_DARWIN >>>>> >>>>> target: AMD64_DARWIN >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>>> > wrote: >>>>> I wasn?t aware this had changed. >>>>> When was this regression introduced? >>>>> >>>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>>> > wrote: >>>>>> >>>>>> For instance, cm3 used to keep on compiling files after errors were >>>>>> found. Now it seems to stop after one module. That loss of functionality >>>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>>> change. When was this change decided? Is there a way to restore it? >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 20:47:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 18:47:13 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , , <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu>, <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com>, Message-ID: (Note: This was meant to report the unit path but I failed to commit that. This is good enough for today. Later...) From: jay.krell at cornell.edu CC: hosking at purdue.edu; lists at darko.org; jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? Date: Thu, 13 Aug 2015 10:34:26 -0700 To: jay.krell at cornell.edu Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote:Sorry, I was distracted by a phone call before - that is the compilerthat exhibits the behaviour. Previous to that I was using somethingthat was compiled in 2010, so not very helpful for pinpointing.On 8/12/15, Antony Hosking wrote:Are you saying that the behavior was still working as of d5.9.0? Or is that the version you are currently using in which it no longer does what you want? We should be able to figure out what commit broke things if we can pinpoint a time when it worked. On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: I wasn?t aware this had changed. When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric > wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 22:27:54 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 13:27:54 -0700 Subject: [M3devel] Build Server - Plan Message-ID: I'm setting up a server for building CM3 that takes a "minimalist" approach. It's a machine running several virtual machines, one for each platform supported by CM3. Each VM will contain clean install of the OS plus any external tool dependencies. It will have a minimal compiler install, basically enough to compile itself for the host target. I'm going create one VM for each target I have a bootstrap compiler for, so if you think you'll find this useful and you want a target supported PLEASE CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. Users can request any version of the compiler from the github repository and all new commits will be automatically built for all platforms. The publicly available build products will be: - minimal executables for bootstrap, eg the frontend and a backend - model compiler config files - compilation logs for bootstrap executables - compilation logs for most modules in the github repository - logs for certain tests Packages, libraries, scripts and non-essential tools or executables will not be built or used, the idea being that people take the minimal bootstraps and build from there. I'll post the URL when it's up and running and any suggestions are welcome. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 13 22:31:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 13 Aug 2015 16:31:29 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150813203129.GA20453@topoi.pooq.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. > > I'm going create one VM for each target I have a bootstrap compiler for, so > if you think you'll find this useful and you want a target supported PLEASE > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > Users can request any version of the compiler from the github repository > and all new commits will be automatically built for all platforms. > > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests > > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. > > I'll post the URL when it's up and running and any suggestions are welcome. Will the C backend be one of the platforms (or several, if they're not yet all compatible?) -- hendrik From jay.krell at cornell.edu Fri Aug 14 00:26:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 22:26:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: , <20150813203129.GA20453@topoi.pooq.com> Message-ID: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 02:14:16 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 17:14:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: I want to try to support every sort of backend I can configure. That's the focus, frontend and backend executables. In particular the idea is to be able to test backends against each other for correctness with a particular front end (and the other way around). On Thu, Aug 13, 2015 at 1:31 PM, Hendrik Boom wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, > so > > if you think you'll find this useful and you want a target supported > PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first > up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 04:11:08 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 19:11:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > > minimal executables for bootstrap, eg the frontend and a backend > > > While I have fiddled with this being just cm3, I currently propose it is > the "min" output of make-dist.py (which does output an "all" variant as > well, > it'll take just a bit of change to make that optional). > > > Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), > libm3 (static suffices). > > > make-dist.py also already outputs make-dist.py.log or such next to > itself I believe. > See "Tee" in pylib.py. > > > cm3 is a valid minimal bootstrap, and I have used it a number of times > all on its own, > but only IF you also have a matching source tree -- > m3cc/config/libm3/m3core. > It doesn't work if it is some "random old working" cm3 and not the rest. > > > "min" can be slightly more min I believe -- it has all the .m3 files for > m3core/libm3, > for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > > > > > Will the C backend be one of the platforms (or several, > > if they're not yet all compatible?) > > > Unfortunately, the C backend output is not compatible with the gcc > backend output. > They vary in how they pass the "display" to nested procedures (oh, if > only we didn't > have nested procedures...) > So we might want to do something about this. > I have considered appending "C" to BUILD_DIR, and have done so with mixed > success. > Alternatively, do everything twice. > > > Also, if isn't clear, the C backend output is still very > target-specific. > I'd like to fix that, but I don't have a full plan in mind and might > give up. > > > Note that the minimal bootstrap can be used no matter which backend > you are next going to use, i.e. any bootstrap can feed into C backend, > gcc backend, LLVM backend. > > > Can you really do Mac? > And if so, multiple versions? > > There is unfortunate combinatorial explosion beyond all this. > Specifically, you could n different C compiler per platform. > gcc x, gcc y, clang x, clang y, etc. > > > > logs for certain tests > > > We have been using m3-sys/m3tests as a start. Just a start. > > > Thank you, > - Jay > > > > > Date: Thu, 13 Aug 2015 16:31:29 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > > > It's a machine running several virtual machines, one for each platform > > > supported by CM3. Each VM will contain clean install of the OS plus any > > > external tool dependencies. It will have a minimal compiler install, > > > basically enough to compile itself for the host target. > > > > > > I'm going create one VM for each target I have a bootstrap compiler > for, so > > > if you think you'll find this useful and you want a target supported > PLEASE > > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be > first up. > > > > > > Users can request any version of the compiler from the github > repository > > > and all new commits will be automatically built for all platforms. > > > > > > The publicly available build products will be: > > > > > > - minimal executables for bootstrap, eg the frontend and a backend > > > - model compiler config files > > > - compilation logs for bootstrap executables > > > - compilation logs for most modules in the github repository > > > - logs for certain tests > > > > > > Packages, libraries, scripts and non-essential tools or executables > will > > > not be built or used, the idea being that people take the minimal > > > bootstraps and build from there. > > > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > > > Will the C backend be one of the platforms (or several, if they're not > > yet all compatible?) > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 08:36:29 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 06:36:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150814063629.GA3086@zoho.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. From jay.krell at cornell.edu Fri Aug 14 09:02:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 07:02:51 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: , <20150814063629.GA3086@zoho.com> Message-ID: It is very easy to install. "We" run make-dist.py and get out a "min" install and an "all" install (per platform).You pick one, extract it, set $PATH optionally, and you are done -- run cm3 either via $PATH or by full path. There is some chance our binaries built on a "new" system won't run on an "old" system.We can mitigate that by deferring some of the build to install time.I hope to have a better story here, a more "sourcey" distribution, that is easy to install.But the binary distributions usually work well. Sometimes you have to edit the config files, but I have endeavored to make that not necessary. There is another install path that prompts for path to compiler, linker, etc., even tries to figure it out. But I don't use that. I think it isn't needed. Yes we support significantly more than x86/amd64. We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw machines -- very recently even.In the 5.8.6 timeframe, I was setup to build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built tonight -- the gcc-based backend. Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Posix or NT for I/O, networking, threads, etc., and X windows or Win32.That is, while we don't run in a browser, we can run almost anywhere else, generally because other people have ported their underlying systems everywhere. (Run in a browser though: LLVM w/ emcriptem backend) The system was already very portable. I have made it easier to port. I assume the "difficult" goals are meant to be ensure that people don't have to jump through the hoops of "install lots of extra stuff" to get it to work. Most people will overshoot the minimal install requirements, but they will overshoot differently.I - Jay > Date: Fri, 14 Aug 2015 06:36:29 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 16:49:56 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 14:49:56 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150814144956.GA4391@zoho.com> Hi, please see comments within: On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > It is very easy to install. Ok, from what I read here I misunderstood. But I am glad to hear it. > Yes we support significantly more than x86/amd64. Thanks for the explanation. I am following projects that run on non-Intel platforms and I get a little concerned when there is movement to only run on Intel or when there seems to be the position Intel is all there is. While I concede Intel is overwhelmingly what people have there is also some very nice and increasingly cheap really good hardware based on other, better architectures and it's a shame to see that ignored into obvlivion. I was happy to see the broad range of hardware and OS when I looked at the CM3 download page so when somebody noted he was going to tackle the Big 3 I was concerned because many projects and OS are now pretty much Intel only which I think is both unfortunate and also technically limiting in terms of code quality. > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > tonight -- the gcc-based backend. I remember we talked about the confusion of the various flavors of SPARC listed in the downloads section. If people are in a consolidating mood it might be good to just use the Solaris Studio version instead of gcc on that platform. > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Well, LLVM is a significant inhibitor to running anywhere but Intel as far as I know. LLVM SPARC does not seem to even be in the alpha stage and doesn't look like it's on anybody's list of things to do. LLVM MIPS is also not working well from the looks of things on the Debian MIPS list. Other projects that have adopted LLVM seem to use it only on Intel and have stayed with gcc on other platforms. There is a concern that will eventually be too difficult to maintain and the natural solution will be to abandon non-Intel hardware. I understand and agree with the value of LLVM from the standpoint of reducing code complexity over gcc and also the more liberal license but it's disappointing to see only the Intel support is probably ever going to be worth anything as far as LLVM goes. > The system was already very portable. I have made it easier to port. > > I assume the "difficult" goals are meant to be ensure that people don't > have to jump through the hoops of "install lots of extra stuff" to get it > to work. I really don't know but I would agree with that. The install should go in standard paths and not require any prereqs, if possible. Thank you. From lists at darko.org Fri Aug 14 18:22:08 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:22:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 18:30:16 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:30:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 19:18:10 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 10:18:10 -0700 Subject: [M3devel] Build Server - Access and Architecture Message-ID: Anyone who wants to setup their own build VMs is welcome to do so and SSH access will be provided. The VMs are controlled from the host command line and you can jump into the command line of any VM. The host will have a local, full, and up to date copy of the github repository. There will be VMs with pristine installs of each of the supported OSes available on the host. These will include MacOS, Windows and various Unixes. To make a build VM you copy an OS VM and setup the build environment by installing the compiler and any required tools. This becomes the baseline for any actual compiling/building. The VM must also have a "build" script that performs the work. This script must pull the requested source version from the local git repository before building. The host build driver performs the following steps for each build VM: - it starts the build VM and calls the build script - when the build script has completed it copies the products (results) from the build VM onto the host where they are consequently uploaded to the publically accessible download server. - the host then rolls back/restores the build VM to its original state and stops it. This means there can be no contamination between builds and you always start building from a known, working state. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 20:15:37 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 11:15:37 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: > The server will be running MacOS 10.10 as the host. The VMs can handle any > MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need > for anything else? > > If you mean can we have multiple VMs for deploying different compiler > configs, then the answer is yes. Each VM will be suspended until it's > needed for a build. Once it's built the requested products and they are > uploaded to the download server, it shuts down. So there aren't any real > performance or memory limitations, all disks are SSDs. > > > On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > >> > minimal executables for bootstrap, eg the frontend and a backend >> >> >> While I have fiddled with this being just cm3, I currently propose it is >> the "min" output of make-dist.py (which does output an "all" variant as >> well, >> it'll take just a bit of change to make that optional). >> >> >> Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), >> libm3 (static suffices). >> >> >> make-dist.py also already outputs make-dist.py.log or such next to >> itself I believe. >> See "Tee" in pylib.py. >> >> >> cm3 is a valid minimal bootstrap, and I have used it a number of times >> all on its own, >> but only IF you also have a matching source tree -- >> m3cc/config/libm3/m3core. >> It doesn't work if it is some "random old working" cm3 and not the rest. >> >> >> "min" can be slightly more min I believe -- it has all the .m3 files >> for m3core/libm3, >> for debugging. I'm not sure if the .i3 files are needed, or the >> .ig/.mg. >> >> >> >> > Will the C backend be one of the platforms (or several, >> > if they're not yet all compatible?) >> >> >> Unfortunately, the C backend output is not compatible with the gcc >> backend output. >> They vary in how they pass the "display" to nested procedures (oh, if >> only we didn't >> have nested procedures...) >> So we might want to do something about this. >> I have considered appending "C" to BUILD_DIR, and have done so with >> mixed success. >> Alternatively, do everything twice. >> >> >> Also, if isn't clear, the C backend output is still very >> target-specific. >> I'd like to fix that, but I don't have a full plan in mind and might >> give up. >> >> >> Note that the minimal bootstrap can be used no matter which backend >> you are next going to use, i.e. any bootstrap can feed into C backend, >> gcc backend, LLVM backend. >> >> >> Can you really do Mac? >> And if so, multiple versions? >> >> There is unfortunate combinatorial explosion beyond all this. >> Specifically, you could n different C compiler per platform. >> gcc x, gcc y, clang x, clang y, etc. >> >> >> > logs for certain tests >> >> >> We have been using m3-sys/m3tests as a start. Just a start. >> >> >> Thank you, >> - Jay >> >> >> >> > Date: Thu, 13 Aug 2015 16:31:29 -0400 >> > From: hendrik at topoi.pooq.com >> > To: m3devel at elegosoft.com >> > Subject: Re: [M3devel] Build Server - Plan >> >> > >> > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: >> > > I'm setting up a server for building CM3 that takes a "minimalist" >> approach. >> > > >> > > It's a machine running several virtual machines, one for each platform >> > > supported by CM3. Each VM will contain clean install of the OS plus >> any >> > > external tool dependencies. It will have a minimal compiler install, >> > > basically enough to compile itself for the host target. >> > > >> > > I'm going create one VM for each target I have a bootstrap compiler >> for, so >> > > if you think you'll find this useful and you want a target supported >> PLEASE >> > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be >> first up. >> > > >> > > Users can request any version of the compiler from the github >> repository >> > > and all new commits will be automatically built for all platforms. >> > > >> > > The publicly available build products will be: >> > > >> > > - minimal executables for bootstrap, eg the frontend and a backend >> > > - model compiler config files >> > > - compilation logs for bootstrap executables >> > > - compilation logs for most modules in the github repository >> > > - logs for certain tests >> > > >> > > Packages, libraries, scripts and non-essential tools or executables >> will >> > > not be built or used, the idea being that people take the minimal >> > > bootstraps and build from there. >> > > >> > > I'll post the URL when it's up and running and any suggestions are >> welcome. >> > >> > Will the C backend be one of the platforms (or several, if they're not >> > yet all compatible?) >> > >> > -- hendrik >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 20:46:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 18:46:46 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150813203129.GA20453@topoi.pooq.com>, , , Message-ID: I have tried PPC_DARWIN on 10.5. It doesn't *quite* work. Almost. I don't believe Rosetta has adequate compatibility, i.e in suspending threadsand getting their context. When we switch to cooperative suspend that will probably work.And then we can remove a bunch more target-specific code. Generally you don't need to support Rosetta. It just works.But I don't believe Rosetta quite fully works. PPC_DARWIN definitely had a long somewhat recent run of workingon native hardware and probably still does.(Fyi, I386_DARWIN also worked before MacOSX/Intel was public.) - Jay Date: Fri, 14 Aug 2015 11:15:37 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:07:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:07:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: ARM_DARWIN I never quite finished.It was almost working on a jail-broken iPhone 3gs.Setting up and testing just a C development environmentis often the difficult part. If you do have a C cross compiler yes that can shift slightly,easily where some things happen. Let me explain.The compilation process has a few different steps it can go.Including that we have some C code.So that goes .c => .o, or internally .c => .s => .o. Ignoring LLVM...We have the NT386 backend.Code goes right from .m3 => .obj.cm3 does "everything". We have the gcc backend.Code goes.m3 => .mc => .ms => .o .ms files are just assembly source.mc files are an internal format that cm3 writes out. We have the C backend.Code goes.m3 => .c => .s => .ocm3 writes the .c files. And then there is linking. This is all *hightly* automated.From the users point of view, cm3 seems to go from .m3 files to a .a/.dylib/executable. But cm3 only has within it some of the stages.It doesn't have a C compiler, an assembler, or a linker.We depend on a "C" toolchain for this (I'm calling assembler/linker part of the C toolchain). If you don't have the C toolchain, then you are stuck part way through those pipelines.Either with a bunch of .o files you can't link.Or a bunch of .ms files you can't assemble and link.Or a bunch of .c files you can't compile/assemble/link. Native toolchains are historically more common than cross tool chains.So we have a "boot" workflow -- that needs work -- where on the "first" machine,you stop where I say you are "stuck", you archive up the files, copy themto the target machine, which has a native toolchain, and then resume the pipeline --running C compiler/assembler/linker. Now, if you have a cross toolchain there is not quite reason for that. But, it comes down mainly to the config files and the builder, and another detail I'll get to. The "boot" workflow was well automated 20 years ago.Back when "quake" was written in C. The 3.6 release.I don't remember what the 4.0/4.1 release looked like. I have good automated flow for producing a cm3 for cross scenarios,and it is useful, but it requires a matching source tree on the other side. This is boot1.py and boot2.sh. If there is an incompatibility, like I recently added to m3core, then it doesn't work.So it dawns on me now, that boot1.py should produce m3core/libm3, and maybe cm3cg.Though cm3cg would require a cross compiler, so sometimes a pain. "another detail I'll get to".We have a small number of tools that when we build the system, we buildthem and run time. Other than cm3. There is mklib, which I already handle.But there is also m3bundle and PklFonts and some RPC stub generators. Given a cross C toolset, we don't have conventions/automation around,hey, I'm targeting x, building on y, my y-hosted tools might need an update.I need to build all of y, then all of x, and when I'm building x, what is the cm3I use? What is the C compiler I use? Again it is mostly just a thing with config files.The config files tend to run "gcc" and find it in $PATH.They assume it is for the current host/target.But there needs to be arguably some convention, I'm hosting on I386_LINUX,targeting AMD64_DARWIN -- where are the native tools and where are the cross tools?In gcc-land, this is basically you run ad64-apple-darwin-gcc for the AMD64 target toolset,instead of just "gcc" You might also run e.g. /usrl/ocal/cm3/bin//cm3cg. So, very long story short -- we largely need a file system layout. The "GNU platform" mapping I386_DARWIN => i386-apple-darwin8 for example,is somewhat encoded in our tree, but for this to work it needs more formalization. With LLVM, you get all the backends together in on tree and you givethe executable a command line switch and it finds the right .so, I imagine,so this sort of gets easier. We still have to know the switches and work that into config and such.We should just do that for native targets anyway and then cross will reuse that. Again, sorry, long story short -- paths and switches to compiler/assembler/linker is largely the thing. Apple has a good system here btw.You say gcc -arch ppc.And that combines with -x assembler.So you can cross compile with gcc -arch ppc.You can cross assemble with gcc -arch -x assembler.They put the work in the gcc driver. It then farms out to the correct cc1/cc1plus/as. If only this was the norm.. Another thing -- a cross C compiler that can handle something with no #includes andno linking, is really easy to build. It is the headers and libraries that make it more difficult.The assembler/linker are also sometimes a sticking point, because sometimes, if youread the gcc config/install manual, they prefer the native closed source assembler/linker. I have a working VMS C cross compiler from Mac, but I had to copy various filesfrom a running VMS machine to do it -- the headers and libraries. In as much as the assembler/linker is GNU, it is easier.And Apple's stuff is open source and people have ported it.But imagine targeting HP-UX or AIX, or NT using the Microsoft toolset.There you can't just conjure cross tools with arbitrary host, only what the vendor ships. - Jay Date: Fri, 14 Aug 2015 09:30:16 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:11:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:11:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:24:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:24:54 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814144956.GA4391@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com> Message-ID: > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. I already mostly solved this years ago. SOLgnu and SOLsun were always already the same, e.g. same threading,same jmpbuf, except the config files picked a different C compiler and differentoutput directory name. I decided it shouldn't be separate paths everywhere in cm3/config files,only which C compiler to run and output directory names.That is, only the config files should vary. We have a somewhat separate notion of "TARGET" and "BUILD_DIR".We might need to better about that.For example profile build get "p" appended to BUILD_DIR, but still the same TARGET.We need BUILD_DIR=SOLgnu but TARGET=SPARC32_SOLARIS. The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun compiler.I invented a separate variable with better granularity. Just look at SPARC32_SOLARIS: readonly TARGET = "SPARC32_SOLARIS"readonly C_COMPILER = "SUN" % or GNU (SOLgnu configuration)include("SPARC32_SOLARIS.common") and then SPARC32_SOLARIS.common looks for the compiler you ask for,and makes the switches match also. Rodney gave me the term for this elegance the other day -- "Cartesian factoring". Or, only factor out what varies. Where things are the same, don't duplicate. In the old days the config files were very monolithic and there was duplication all over the place.For example, in the old days, HPPA implies HP-UX, but now HPPA can be Linux, NetBSD, FreeBSD, etc.The HPPA part is one file, the Linux/NetBSD/FreeBSD parts are their own single files.We only have a combinatorial explosion of small config files, not large ones.Even this could be improved -- runtime composition? The old days had somewhat less sharing. i.e. we probably used "vendor" cc/as/ld more and GNUless. But there is still good allowance for that now. I similarly refactor I386_NT.It was called NT386.The "mingw" platform is largely the same -- Win32 threads, Win32 I/O, Win32 GUI, GNU compiler/assembler/linker.There I came up with several variables for the factoring: See NT.commonTHREAD_LIBRARY = MS or PTHREADSWINDOW_LIBRARY = MS or XLINKER = MS or GNUC_COMPILER = MS or GNU (should add "clang" or "llvm" as an option here) LINUXLIBC6 is now I386_LINUX. NT386GNU is I386_CYGWIN or I386_MINGW A problem I ran into though wrt SOLgnu/SOLsun/LINUXLIBC6/NT386 is that nobodyusing them wanted to switch the name. So I fixed n systems by renaming to m, leaving us with n + m, but they are justlayered over very common stuff. It doesn't take n + m work to implement.But the system is cluttered. Has anyone switched from LINUXLIBC6 to I386_LINUX? Can we delete LINUXLIBC6?Has anyone/everyone switched from SOLsun/SOLgnu to SPARC32_SOLARIS?FreeBSD4 to I386_FREEBSD? Well, at least all the AMD64 and DARWIN targets are well named.The others maybe aren't used much either way.. - Jay > Date: Fri, 14 Aug 2015 14:49:56 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > Hi, please see comments within: > > On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > > > It is very easy to install. > > Ok, from what I read here I misunderstood. But I am glad to hear it. > > > Yes we support significantly more than x86/amd64. > > Thanks for the explanation. I am following projects that run on non-Intel > platforms and I get a little concerned when there is movement to only run on > Intel or when there seems to be the position Intel is all there is. While I > concede Intel is overwhelmingly what people have there is also some very > nice and increasingly cheap really good hardware based on other, better > architectures and it's a shame to see that ignored into obvlivion. > > I was happy to see the broad range of hardware and OS when I looked at the > CM3 download page so when somebody noted he was going to tackle the Big 3 I > was concerned because many projects and OS are now pretty much Intel only > which I think is both unfortunate and also technically limiting in terms of > code quality. > > > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > > tonight -- the gcc-based backend. > > I remember we talked about the confusion of the various flavors of SPARC > listed in the downloads section. If people are in a consolidating mood it > might be good to just use the Solaris Studio version instead of gcc on that > platform. > > > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, > > Well, LLVM is a significant inhibitor to running anywhere but Intel as far > as I know. LLVM SPARC does not seem to even be in the alpha stage and > doesn't look like it's on anybody's list of things to do. LLVM MIPS is also > not working well from the looks of things on the Debian MIPS list. Other > projects that have adopted LLVM seem to use it only on Intel and have stayed > with gcc on other platforms. There is a concern that will eventually be too > difficult to maintain and the natural solution will be to abandon non-Intel > hardware. > > I understand and agree with the value of LLVM from the standpoint of > reducing code complexity over gcc and also the more liberal license but it's > disappointing to see only the Intel support is probably ever going to be > worth anything as far as LLVM goes. > > > The system was already very portable. I have made it easier to port. > > > > I assume the "difficult" goals are meant to be ensure that people don't > > have to jump through the hoops of "install lots of extra stuff" to get it > > to work. > > I really don't know but I would agree with that. The install should go in > standard paths and not require any prereqs, if possible. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:32:02 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:32:02 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:42:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:42:00 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , Message-ID: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:48:29 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: > Reasonable but painful. And varied. > > > Do you want the Apple-supplied tools or fetch various cc/ld source and > build them? > > > For Windows, Microsoft or Cygwin or Mingw? > > Probe the machine to see if the tools are already present? > Run cc/gcc/cl, see if it works? > > > I like the autoconf model here -- which is indeed that last point. > It tries to compile "int main() { return 0; }", if that works, great a > working C compiler. > If it doesn't, it might fish around for a few, but ultimately it gives up, > and the > user can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. > > > It doesn't know how to apt-get install build-essentials or other. > > > I must say -- autoconf -- I don't like sh or m4, and it is slow, but it > does kinda work well. > I wrestle with this. I output C and I need int8/int16/int32/int64. > I have the following choices > 1 assume char/short/int/__int64/long long > 2 #if on compiler for __int64 vs. long long > 3 try to check limits.h -- paying for the cost of #include for every > compile > 4 check for STDC_VERSION and use inttypes.h/stdint.h > 5 autoconf > 6 something like autoconf > > > For now I do some mix of 1/2/3, but I have this nagging feeling that > autoconf would > gain portability to more systems. > > > Modula-3 follows "the Perl way" and is ported everywhere. > Whereas autoconf can kind of cons up a port if the system is the > combination > of already known elements. Only individual elements need to be known about, > not combinations. > > > autoconf again is ugly and slow, but it is better in terms of "cartesian > factoring". > > And it doesn't handle Microsoft tools well or at all. > > > Also I have a similar nagging dilemma on bootstrap archives. > I want some incrementally. So I want e.g. make. > I want (currently lack) some hierarchy. So I want recursive make or maybe > GNU make-specific. > Maybe automake? Which still doesn't buy me MS make. > cmake? > I don't really want to reinvent portability beyond GNU make. > out-of-source builds? With bootstrapping? Overkill? > So I wonder if I should, gasp, generate automake input. > > make does though cater to the new generation and their IDEs, while also > catering to the old ways. > KDE's use of it is a big gote > > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 12:32:02 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > I think the Mac and Win installers should also install the required > external tools, so it all "just works", which makes it a bit more > difficult. Either way not a priority right now. > > On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:22:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:22:10 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , , , Message-ID: Good thinking...but not easy to pick the one.People will complain no matter what you chose.MS? Cygwin? Mingw? Cross inhibits MS.Cross that isn't prebuilt will take a while to build.MS probably not redistributable. Have to point people at a web page. Chasing annual versions.. - Jay Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:39:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:39:10 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? Message-ID: Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:43:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:43:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? Message-ID: air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Aug 15 22:43:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 16:43:59 -0400 Subject: [M3devel] Current releas(es) Message-ID: <20150815204359.GA15497@topoi.pooq.com> I'm currently installing stuff on an effecively new machine. Where do I find the current Modula 3 release? Is it still http://www.opencm3.net/releng/ ? If not, I'd better update the wikipedia entry. And is CM3 5.8.6 the current release? -- hendrik From jay.krell at cornell.edu Sun Aug 16 00:15:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 15 Aug 2015 22:15:30 +0000 Subject: [M3devel] Current releas(es) In-Reply-To: <20150815204359.GA15497@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: 5.8.6 is unfortunately probably current for most architectures.We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 before: foreach root in [ m3cc, bin ] after:foreach root in [ bin ] - Jay > Date: Sat, 15 Aug 2015 16:43:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: [M3devel] Current releas(es) > > I'm currently installing stuff on an effecively new machine. > > Where do I find the current Modula 3 release? > > Is it still > > http://www.opencm3.net/releng/ > > ? If not, I'd better update the wikipedia entry. > > And is CM3 5.8.6 the current release? > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Aug 16 01:42:07 2015 From: adacore at marino.st (John Marino) Date: Sun, 16 Aug 2015 01:42:07 +0200 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <55CFCE4F.2070104@marino.st> Don't forget FreeBSD has 5.10.0 easily available: http://www.freshports.org/lang/modula3 John On 8/16/2015 12:15 AM, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. > We have a newer build for MacOSX. > You will need one or two patches, if you want to run upgrade.py to get > latest. > I remember this one: > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > > before: > > foreach root in [ m3cc, bin ] > > after: > foreach root in [ bin ] > > > - Jay > >> Date: Sat, 15 Aug 2015 16:43:59 -0400 >> From: hendrik at topoi.pooq.com >> To: m3devel at elegosoft.com >> Subject: [M3devel] Current releas(es) >> >> I'm currently installing stuff on an effecively new machine. >> >> Where do I find the current Modula 3 release? >> >> Is it still >> >> http://www.opencm3.net/releng/ >> >> ? If not, I'd better update the wikipedia entry. >> >> And is CM3 5.8.6 the current release? >> >> -- hendrik >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From hendrik at topoi.pooq.com Sun Aug 16 04:15:41 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 22:15:41 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <20150816021541.GA22514@topoi.pooq.com> On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should really carry a version number. Should I try and find the official Debian syntax for version numbers? -- hendrik We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > before: > foreach root in [ m3cc, bin ] > after:foreach root in [ bin ] > > > - Jay > > > Date: Sat, 15 Aug 2015 16:43:59 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: [M3devel] Current releas(es) > > > > I'm currently installing stuff on an effecively new machine. > > > > Where do I find the current Modula 3 release? > > > > Is it still > > > > http://www.opencm3.net/releng/ > > > > ? If not, I'd better update the wikipedia entry. > > > > And is CM3 5.8.6 the current release? > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From jay.krell at cornell.edu Sun Aug 16 04:20:05 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 15 Aug 2015 19:20:05 -0700 Subject: [M3devel] Current releas(es) In-Reply-To: <20150816021541.GA22514@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: "Yes please take over packaging?" :) Start with scripts/python/make-dist.py? - Jay On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: >> 5.8.6 is unfortunately probably current for most architectures. > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > really carry a version number. Should I try and find the official > Debian syntax for version numbers? > > -- hendrik > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 >> >> before: >> foreach root in [ m3cc, bin ] >> after:foreach root in [ bin ] >> >> >> - Jay >> >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 >>> From: hendrik at topoi.pooq.com >>> To: m3devel at elegosoft.com >>> Subject: [M3devel] Current releas(es) >>> >>> I'm currently installing stuff on an effecively new machine. >>> >>> Where do I find the current Modula 3 release? >>> >>> Is it still >>> >>> http://www.opencm3.net/releng/ >>> >>> ? If not, I'd better update the wikipedia entry. >>> >>> And is CM3 5.8.6 the current release? >>> >>> -- hendrik >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> From microcode at zoho.com Sun Aug 16 09:16:08 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:16:08 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> Message-ID: <20150816071607.GA3479@zoho.com> On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > I already mostly solved this years ago. Oh, that is good news. > SOLgnu and SOLsun were always already the same, e.g. same threading,same > jmpbuf, except the config files picked a different C compiler and > differentoutput directory name. I had misunderstood there was a choice because of the download names, suggesting you actually got a compiler built with Studio or gcc depending on which file you downloaded. > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > compiler.I invented a separate variable with better granularity. That sounds much better, thanks. But for the downloadable stuff where presumably you get a working setup already built and just have to untar the tarball, which compiler(s) are you going forward with now? In other words, since you have consolidate the various Solaris versions as you listed above and it is no longer necessarily indicative of Studio v. gcc, are you going to make turnkey builds available as in the past and if so what will they be built with? And I would vote for using Studio if possible... Thanks. From microcode at zoho.com Sun Aug 16 09:27:30 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:27:30 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150816072730.GB3479@zoho.com> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. From jay.krell at cornell.edu Sun Aug 16 09:52:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 07:52:13 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com> Message-ID: The output of autoconf/automake should have lightweight dependencies.They might stress make, might require GNU make.They might stress the sh, but I think there are adequate shells out there. They are meant to be easy for people building stuff to use.They aren't meant to be easy for people developing stuff to use. Look at this way..while people complain and there are widelyused alternatives like cmake, autoconf/automake are in widespreaduse, and they do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, Aix, Irix, etc. Furthermore, consider any platform that has a native gcc, is likely buildingthat with autoconf/automake. I remain very template for "bootstrap" archives to consist of C and automake/autoconf/libtool output.Or assembly, but still automake/autoconf -- at least to generate makefiles and decide how to run the assembler. - Jay > Date: Sun, 16 Aug 2015 07:27:30 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 16 10:24:09 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 08:24:09 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150816082409.GC3479@zoho.com> On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > The output of autoconf/automake should have lightweight dependencies.They > might stress make, might require GNU make.They might stress the sh, but I > think there are adequate shells out there. That is typically one issue with autoconf, requiring gnu tools in the path. On Solaris this can be annoying since most Solaris people don't use gcc or bash or have them in their path and not all (none of?) the gnu pieces are up to date or even current by any stretch of the imagination. > They are meant to be easy for people building stuff to use.They aren't > meant to be easy for people developing stuff to use. Not sure what you meant here... > Look at this way..while people complain and there are widelyused > alternatives like cmake, autoconf/automake are in widespreaduse, and they > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > Aix, Irix, etc. They often don't "work" for Solaris as-installed but they can most often be made to work. Increasingly, as automake and its prereqs get version bumps there are problems building apps on Solaris because Solaris installs with a very back-level version of gcc and a rather incomplete set of gnu tools. I ran into a problem with the last year where Solaris awk was not good enough to install(!) an app that compiled on Solaris as part of autotools so I had to download a newish copy of gawk and install it. So really, it is much better not to go there if you can avoid it. I am not suggesting an alternative to autotools but just pointing out it is not really accurate to say they work on Linux, Solaris, BSD, etc. except for Linux. I also run BSD on several platforms and because of the same issues as with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, shells, etc. not being good enough for autotools) it is sometimes non-trivial and painful to get Linux apps built on BSD. There is obviously no good/easy solution to this, just to point out it is not the slam-dunk as might be thought. > Furthermore, consider any platform that has a native gcc, is likely > buildingthat with autoconf/automake. Conceded, yet on Solaris it is not clear why gcc is there. It is old and it is in a non-standard location and is often not used. Ideally, apps to be run on Solaris should be able to be built with native (non-gnu) tools. The Studio compilers are very good and have optimizations for SPARC that should be better than what gcc can provide and it is also somewhat of a test of C portability since Studio doesn't necessarily provide all non-standard gcc extensions. From jay.krell at cornell.edu Sun Aug 16 12:04:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:04:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816071607.GA3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com>, , <20150816071607.GA3479@zoho.com> Message-ID: It is like you thought:The SOLsun build used the Sun C compiler, and probably assembler and linker.The SOLgnu build used the GNU C compiler, and maybe assembler, and maybe linker.It isn't a big deal.Turnkey is based on what people have installed.I don't know how to acquire/install the Sun compilers. Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need configurations..just a collection of factors. - Jay > Date: Sun, 16 Aug 2015 07:16:08 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > I already mostly solved this years ago. > > Oh, that is good news. > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > jmpbuf, except the config files picked a different C compiler and > > differentoutput directory name. > > I had misunderstood there was a choice because of the download names, > suggesting you actually got a compiler built with Studio or gcc depending on > which file you downloaded. > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > compiler.I invented a separate variable with better granularity. > > That sounds much better, thanks. But for the downloadable stuff where > presumably you get a working setup already built and just have to untar the > tarball, which compiler(s) are you going forward with now? In other words, > since you have consolidate the various Solaris versions as you listed above > and it is no longer necessarily indicative of Studio v. gcc, are you going > to make turnkey builds available as in the past and if so what will they be > built with? And I would vote for using Studio if possible... > > Thanks. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:11:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:11:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816082409.GC3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com> Message-ID: Autoconf output has little/none dependency.Sun's SPARC optimization are mostly irrelevant as we have little C code,unless you use the C backend. > somewhat of a test of C portability This I appreciate. We have some C code and the C backend.More compilers have "helped".The Tru64 compiler was another.On HP-UX in-box I had only K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And now clang -- having found and worked around a bug in its assembler.Some time soon I'll expand to Metrowerks and Digital Mars.. - Jay > Date: Sun, 16 Aug 2015 08:24:09 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > The output of autoconf/automake should have lightweight dependencies.They > > might stress make, might require GNU make.They might stress the sh, but I > > think there are adequate shells out there. > > That is typically one issue with autoconf, requiring gnu tools in the > path. On Solaris this can be annoying since most Solaris people don't use > gcc or bash or have them in their path and not all (none of?) the gnu pieces > are up to date or even current by any stretch of the imagination. > > > They are meant to be easy for people building stuff to use.They aren't > > meant to be easy for people developing stuff to use. > > Not sure what you meant here... > > > Look at this way..while people complain and there are widelyused > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > Aix, Irix, etc. > > They often don't "work" for Solaris as-installed but they can most often be > made to work. Increasingly, as automake and its prereqs get version bumps > there are problems building apps on Solaris because Solaris installs with a > very back-level version of gcc and a rather incomplete set of gnu tools. I > ran into a problem with the last year where Solaris awk was not good enough > to install(!) an app that compiled on Solaris as part of autotools so I had > to download a newish copy of gawk and install it. So really, it is much > better not to go there if you can avoid it. > > I am not suggesting an alternative to autotools but just pointing out it is > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > Linux. I also run BSD on several platforms and because of the same issues as > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > shells, etc. not being good enough for autotools) it is sometimes > non-trivial and painful to get Linux apps built on BSD. There is obviously > no good/easy solution to this, just to point out it is not the slam-dunk as > might be thought. > > > Furthermore, consider any platform that has a native gcc, is likely > > buildingthat with autoconf/automake. > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > is in a non-standard location and is often not used. Ideally, apps to be run > on Solaris should be able to be built with native (non-gnu) tools. The > Studio compilers are very good and have optimizations for SPARC that should > be better than what gcc can provide and it is also somewhat of a test of C > portability since Studio doesn't necessarily provide all non-standard gcc > extensions. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:12:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? In-Reply-To: References: Message-ID: ok, so this structure_size_boundary isn't entirely bogus.In fact, as is typical, it might have been correct, but itsrelevance has drastically declined as the targets have shifted.Like so much of cm3, it is based closely on gcc. It appears that OpenBSD/68k rounds struct sizes up to a multiple of 2. Other 68k targets do not. Perhaps historical 68k targets did -- perhaps that is a lot of what cm3 was handling. ARM sometimes rounds up to a multiple of 4, but sometimes not, depending on exact platform. sh used to? and still optionally rounds up to a multiple of 4, but I think defaults to not? The only non-8 case we had in cm3 was for HPPA and that was wrong seeming. Getting this value wrong should only lead to a lack of C interop on unusual records.i.e. if you have an array of records with an odd size, the adress of than the firstmight mismatch.It can have an affect in non-array-of-record cases too, if you use BYTESIZE.Or, when the C backend does layout, possibly disagreeing with the frontend. Maybe I'm naive but this all seems like a mistake. sizeof(struct {char a;}) should be 1 for all targets. sizeof(struct {char a[3];}) should be 3 for all targets. but I guess various ABI developers have disagreed through the decadesand we are kinda somewhat stuck with it. There is a gcc command line switch, at least for ARM, to vary the rounding. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: Target.Structure_size_boundary? Date: Fri, 14 Aug 2015 20:43:18 +0000 air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 16 14:18:42 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 16 Aug 2015 08:18:42 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: <20150816121842.GA4830@topoi.pooq.com> On Sat, Aug 15, 2015 at 07:20:05PM -0700, Jay wrote: > "Yes please take over packaging?" :) :) > Start with scripts/python/make-dist.py? As far as I can tell looking at my package cache, all that's needed is to change the name of the file to cm3-LINUXLIBC6-REL_5.8.6_i386.deb When I install it with dpkg, it appears to know internally what revision it is, so not much would need changing. Havint the version number there would help me know whether to bother downloading another copy if I seem to already have one. I probably wasted somoe of your server time by redownloading. -- hendrik > > - Jay > > On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > > > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > >> 5.8.6 is unfortunately probably current for most architectures. > > > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > > really carry a version number. Should I try and find the official > > Debian syntax for version numbers? > > > > -- hendrik > > > > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > >> > >> before: > >> foreach root in [ m3cc, bin ] > >> after:foreach root in [ bin ] > >> > >> > >> - Jay > >> > >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 > >>> From: hendrik at topoi.pooq.com > >>> To: m3devel at elegosoft.com > >>> Subject: [M3devel] Current releas(es) > >>> > >>> I'm currently installing stuff on an effecively new machine. > >>> > >>> Where do I find the current Modula 3 release? > >>> > >>> Is it still > >>> > >>> http://www.opencm3.net/releng/ > >>> > >>> ? If not, I'd better update the wikipedia entry. > >>> > >>> And is CM3 5.8.6 the current release? > >>> > >>> -- hendrik > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> From microcode at zoho.com Sun Aug 16 15:39:31 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:39:31 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> <20150816071607.GA3479@zoho.com> Message-ID: <20150816133931.GA5008@zoho.com> On Sun, Aug 16, 2015 at 10:04:29AM +0000, Jay K wrote: > It is like you thought:The SOLsun build used the Sun C compiler, and > probably assembler and linker.The SOLgnu build used the GNU C compiler, and > maybe assembler, and maybe linker.It isn't a big deal. Turnkey is based on > what people have installed. Are you saying now that you have consolidated the builds into the 4 you listed below they will use what the user has installed? If so that sounds ideal, except for possible search order issues. The gcc that "comes with" Solaris is always installed in the same location (/usr/sfw) but the path to Studio depends on the version and I believe you could create non-standard paths using the tarball installers vs. the package installers (Studio offers both options IIRC). And then there is what the user has in their PATH which might be all, one, or none of the above. And there are some users who install new(er) versions of gcc, usually in addition to the stock version and there is no convention for where this gcc should go. So...there should probably be a documented way to use the toolchain of your choice, if this isn't already explained somewhere. > I don't know how to acquire/install the Sun compilers. They are a free download from Oracle (free to use and free runtime). If they are not already installed on your opencsw box(es) and you would like to have them on there I can do that. It is just a matter of downloading and installing the packages. It should take 20 to 30 minutes. > Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, > AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need > configurations..just a collection of factors. I think this is the right thing to do, I just don't understand how to select which toolchain you want to use, and further, I thought the downloads contained a prebuilt cm3, so I don't understand why a C toolchain would be required at all, unless you are saying in the future the user will compile cm3 rather than get a prebuilt binary. > - Jay > > > > Date: Sun, 16 Aug 2015 07:16:08 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > > I already mostly solved this years ago. > > > > Oh, that is good news. > > > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > > jmpbuf, except the config files picked a different C compiler and > > > differentoutput directory name. > > > > I had misunderstood there was a choice because of the download names, > > suggesting you actually got a compiler built with Studio or gcc depending on > > which file you downloaded. > > > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > > compiler.I invented a separate variable with better granularity. > > > > That sounds much better, thanks. But for the downloadable stuff where > > presumably you get a working setup already built and just have to untar the > > tarball, which compiler(s) are you going forward with now? In other words, > > since you have consolidate the various Solaris versions as you listed above > > and it is no longer necessarily indicative of Studio v. gcc, are you going > > to make turnkey builds available as in the past and if so what will they be > > built with? And I would vote for using Studio if possible... > > > > Thanks. > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From microcode at zoho.com Sun Aug 16 15:43:06 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:43:06 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> Message-ID: <20150816134306.GB5008@zoho.com> On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > Autoconf output has little/none dependency.Sun's SPARC optimization are > mostly irrelevant as we have little C code,unless you use the C backend. Ok, well, it sounds like it will be increasingly important as you go to your C backend though. > > somewhat of a test of C portability > This I appreciate. We have some C code and the C backend.More compilers > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > now clang -- having found and worked around a bug in its assembler.Some > time soon I'll expand to Metrowerks and Digital Mars.. I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have the latest versions. C/C++ are not my thing but if I can help by just building stuff to see what messages we get or if it breaks etc. let me know and I will try to participate. I didn't know Metrowerks was still around. I thought they got swallowed up by Motorola and then removed from all the non-embedded space. > - Jay > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > The output of autoconf/automake should have lightweight dependencies.They > > > might stress make, might require GNU make.They might stress the sh, but I > > > think there are adequate shells out there. > > > > That is typically one issue with autoconf, requiring gnu tools in the > > path. On Solaris this can be annoying since most Solaris people don't use > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > are up to date or even current by any stretch of the imagination. > > > > > They are meant to be easy for people building stuff to use.They aren't > > > meant to be easy for people developing stuff to use. > > > > Not sure what you meant here... > > > > > Look at this way..while people complain and there are widelyused > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > Aix, Irix, etc. > > > > They often don't "work" for Solaris as-installed but they can most often be > > made to work. Increasingly, as automake and its prereqs get version bumps > > there are problems building apps on Solaris because Solaris installs with a > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > ran into a problem with the last year where Solaris awk was not good enough > > to install(!) an app that compiled on Solaris as part of autotools so I had > > to download a newish copy of gawk and install it. So really, it is much > > better not to go there if you can avoid it. > > > > I am not suggesting an alternative to autotools but just pointing out it is > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > Linux. I also run BSD on several platforms and because of the same issues as > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > shells, etc. not being good enough for autotools) it is sometimes > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > no good/easy solution to this, just to point out it is not the slam-dunk as > > might be thought. > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > buildingthat with autoconf/automake. > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > is in a non-standard location and is often not used. Ideally, apps to be run > > on Solaris should be able to be built with native (non-gnu) tools. The > > Studio compilers are very good and have optimizations for SPARC that should > > be better than what gcc can provide and it is also somewhat of a test of C > > portability since Studio doesn't necessarily provide all non-standard gcc > > extensions. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From jay.krell at cornell.edu Sun Aug 16 20:28:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 18:28:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: Message-ID: Actually, I suggest that endianness might be removable as a factor. I know of the following endian-dependentness: 1) bitfield layout This matters "only" so that bitfields can be declared that line up with C to interoperate with. Or, sort of, to interoperate with hardware -- but hardware probably has no endianness, i.e. much hardware can be used with both little and big endian CPUs, and this variability actually makes portabililiy more difficult. I suggest this might not matter and we can pick an arbitrary endianness, at least for the C backend. 2) To pick code that picks apart floating point numbers. I suggest this code can be rewritten in C and/or do a runtime detection of endianness, in C or Modula-3. I don't think endianess matters in initialization non-bitfields. Or at least integers. I have to check. Potentially leaving us with just 4: 32bit posix 64bit posix 32bit windows 64bit windows I know at least one more Target factor to bring up, later (alignment of closures). - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: posix/nt/32/64/endian platforms for C backend? Date: Fri, 14 Aug 2015 20:39:10 +0000 Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Sun Aug 16 22:59:15 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 13:59:15 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 23:40:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 21:40:45 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, Message-ID: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 02:04:03 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 17:04:03 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, > including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically > get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't > have X installed, it fails. We can make it do nothing. We can link to the > stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap > archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if > using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how > to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse > it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU > build system" > It is slow and bad to author in, but produces easy to use somewhat slow > idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would > be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: > install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. > Even gcc. > > > - Jay > > > > ------------------------------ > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus > different distributions, there are different views on how CM3 should be > setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I > want to take it further than the current installer. In particular I want to > reduce the external, internal and environmental dependencies so that you > can build anything, anywhere with just the minimal set of executables and a > standard set of tools. Others want a more elaborate setup which gives them > more flexibility; that can also be derived from that same minimal set. > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 17 03:24:07 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Aug 2015 18:24:07 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU build system" >> It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. Even gcc. >> >> >> - Jay >> >> >> >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and so >> forth. >> >> > What this system spits out is everything you need to compile anything you >> > want - a complete and working compiler. Not everyone wants or needs to use >> > all the packages. If you do, you can. Here are the instructions for doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I do >> I think anybody new is going to have similar questions/misunderstandings of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 05:14:27 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link > it all. > > > The compiler has to come with a matching m3core/libm3, binary and/or > source. The cm3cg binary can be built from source, but again it has to be > matching -- or use C backend. > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can > be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, > minimal toolset (mostly just gcc), installed in a particular place. I'm not > trying to support or test every configuration, just one. It's for people > who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know > there are no external source dependencies in the frontend and the backend > depends only on the gcc source which is in the repository. The build server > only builds the frontend and backend executables, nothing else, so there > are no other dependencies. > > If you're talking about building an X Windows application, I still don't > get it. Why do I need to support someone wanting to write an X program? I'm > not supporting people who want to do anything other than run the compiler > on their platform. If they want to make X programs they can download and > configure whatever is needed themselves. The CM3 compiler can compile (but > not link) M3 interfaces and modules for X without having X installed. > That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, >> including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically >> get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and >> don't have X installed, it fails. We can make it do nothing. We can link to >> the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap >> archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if >> using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to >> how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse >> it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU >> build system" >> It is slow and bad to author in, but produces easy to use somewhat slow >> idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would >> be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: >> install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. >> Even gcc. >> >> >> - Jay >> >> >> >> ------------------------------ >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and >> thus different distributions, there are different views on how CM3 should >> be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I >> want to take it further than the current installer. In particular I want to >> reduce the external, internal and environmental dependencies so that you >> can build anything, anywhere with just the minimal set of executables and a >> standard set of tools. Others want a more elaborate setup which gives them >> more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend >> just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to >> me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from >> the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and >> so >> forth. >> >> > What this system spits out is everything you need to compile anything >> you >> > want - a complete and working compiler. Not everyone wants or needs to >> use >> > all the packages. If you do, you can. Here are the instructions for >> doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been >> following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I >> do >> I think anybody new is going to have similar questions/misunderstandings >> of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> >> _______________________________________________ M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:42:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:42:03 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , Message-ID: I think you want is roughly: boot1.py over each architecture package those up along with matching m3core/libm3/m3cc source boot2.sh or such on the target system -- but this currently assumes an entire matching source tree, so this needs work But fair enough, you just want compilers, to get around the "crossing". Crossing is easy as far as we do it -- just see boot1.py. It only gets as far as assembly files. Then you move them to the target machine and run native assembler and linker. ARM_LINUX is a sore point for me. Debian has been through at least three different arm ports. For this all to work reasonably cleanly we have to track something like this: https://wiki.debian.org/Multiarch/Tuples Meanwhile, m3front barely cares about platform at all. Almost all platforms generate identical code until they hit the backend. And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. Our "porting" and "supported platforms" is much ado about little. We support "everything and nothing", by leveraging underlying gcc or C compiler and pthreads or Win32, and X or Win32. - Jay Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com; microcode at zoho.com If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:43:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:43:37 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> Message-ID: But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else.It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 18 03:39:09 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 17 Aug 2015 21:39:09 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150818013909.GA16240@topoi.pooq.com> On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > I think you want is roughly: > boot1.py over each architecture > package those up > along with matching m3core/libm3/m3cc source > boot2.sh or such on the target system -- but this currently > assumes an entire matching source tree, so this needs work > > But fair enough, you just want compilers, to get around the "crossing". > Crossing is easy as far as we do it -- just see boot1.py. > It only gets as far as assembly files. Then you move them to the target > machine and run native assembler and linker. Suppose you used C instead of assembler for this. Would that get you more machine types for less effort? I've had a machine on which there was a dispute which assembler to use! > > ARM_LINUX is a sore point for me. > Debian has been through at least three different arm ports. > For this all to work reasonably cleanly we have to track > something like this: > https://wiki.debian.org/Multiarch/Tuples > > Meanwhile, m3front barely cares about platform at all. > Almost all platforms generate identical code until they hit the backend. > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > Our "porting" and "supported platforms" is much ado about little. > We support "everything and nothing", by leveraging underlying gcc > or C compiler and pthreads or Win32, and X or Win32. > > - Jay > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > Subject: Re: [M3devel] Build Server - Plan > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; microcode at zoho.com > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link it all. > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > Python is convenient for cross-platform scripting, but optional. > - Jay > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. Even gcc. > > > - Jay > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > The solution for SPARC might be to create a cross-compiler. That's not > > > ideal but would be useful for verifying that things compile for the > > > platform at least, short of having the right hardware. > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > machine time but I can't keep the machine running all the time. Hopefully > > the current setup (opencsw) is still viable. > > > > > I disagree with your "only for experts" assessment. The point of this > > > server is that you don't have to compiler the compiler and a backend just > > > to get the latest (or even a properly working) compiler. That's the very > > > "expert" work I'm trying to automate. > > > > That is good to hear because from your list of deliverables it seemed to me > > there would still be a lot of tinkering to get a complete install. If you > > are scripting that and it can be built effortlessly and mindlessly from the > > deliverables then good. I think it would be ideal and very nice if people > > could download a turnkey tarball ready to run on their platform, or source > > code ready to build and an installer script that installs the final > > product. Jay mentioned autotools and they are usually far from optimal on > > Solaris because of autotools reliance on so much gnu stuff that is not > > always necessarily present on Solaris but that is the general idea- to be > > able to configure && make && make install with all the standard stuff > > happening like everything being installed in /usr/local/bin|lib|share and so > > forth. > > > > > What this system spits out is everything you need to compile anything you > > > want - a complete and working compiler. Not everyone wants or needs to use > > > all the packages. If you do, you can. Here are the instructions for doing > > > that: "cd ; cm3 " > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > sorry if I wasted anybody's time with this diversion. I have been following > > the mailing list and it looks like Jay and Rodney are mostly discussing > > things with a very small number of regulars. Coming from the outside as I do > > I think anybody new is going to have similar questions/misunderstandings of > > how cm3 is built and installed, etc. > > > > > I'll be opening up the server to anyone who wants to do something > > > different. If someone wants to produce tarballs they can setup their own > > > VMs and they will get built too. > > > > Hopefully this can be integrated with what you already have running on > > opencsw. > > > > Thank you. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Aug 18 04:31:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:31:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu>, , Message-ID: Is bitfield layout really portable enough and predictable enough that interop is achievable?I don't know the layout algorithms myself.I don't really know what packed does.Change the alignment rules for the record?Remove all padding for alignment? And then access fields piecemeal in case anything is no longer aligned?I should experiment. - Jay Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? From: hosking at purdue.edu Date: Tue, 18 Aug 2015 11:57:52 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 18, 2015, at 9:43 AM, Jay K wrote:But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else. And doesn?t interrupt make sense wherever possible. I would hope that there is a reasonable correspondence between C types and M3 types, particularly for RECORD and PACKED elements. It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 04:37:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:37:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150818013909.GA16240@topoi.pooq.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , , , <20150818013909.GA16240@topoi.pooq.com> Message-ID: Yes, but use of C is tied to the C backend.If you want to use cm3cg or LLVM it isn't an option.cm3cg was implied, but the C backend is quite good and workingand I recommend it for all targets. The reason for assembly is that is cm3cg output and probablyoptional LLVM output. I'm not sure people are ready/willing to give up cm3cg yet. ? In fact, with the C backend, we are close to having just a small number, like 6-8,service all targets, as discussed in another thread.The only factors become word size, possibly endian, NT vs. posix.(I have to send a mail yet on "aligned procedure types" -- which correlateswith processor, i.e. the same for all x86 targets). "exact" target doesn't matter.esp. with the jmpbuf thing removed. > I've had a machine on which there was a dispute which assembler to use! Which assembler to use is even a problem on modern Mac OSX 10.10.4 Yosemite.The default LLVM assembler, invoked by the default "gcc" (which is actually clang),is not compatible with gcc/cm3cg. It produces the wrong code.The default assembler works. I put a workaround into cm3cg though. It is really just a mess that any compiler these days doesn't default to outputing.o files directly, but that is gcc. - Jay > Date: Mon, 17 Aug 2015 21:39:09 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > > I think you want is roughly: > > boot1.py over each architecture > > package those up > > along with matching m3core/libm3/m3cc source > > boot2.sh or such on the target system -- but this currently > > assumes an entire matching source tree, so this needs work > > > > But fair enough, you just want compilers, to get around the "crossing". > > Crossing is easy as far as we do it -- just see boot1.py. > > It only gets as far as assembly files. Then you move them to the target > > machine and run native assembler and linker. > > Suppose you used C instead of assembler for this. Would that get > you more machine types for less effort? I've had a machine on which > there was a dispute which assembler to use! > > > > > ARM_LINUX is a sore point for me. > > Debian has been through at least three different arm ports. > > For this all to work reasonably cleanly we have to track > > something like this: > > https://wiki.debian.org/Multiarch/Tuples > > > > Meanwhile, m3front barely cares about platform at all. > > Almost all platforms generate identical code until they hit the backend. > > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > > > Our "porting" and "supported platforms" is much ado about little. > > We support "everything and nothing", by leveraging underlying gcc > > or C compiler and pthreads or Win32, and X or Win32. > > > > - Jay > > > > > > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > > Subject: Re: [M3devel] Build Server - Plan > > From: lists at darko.org > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com; microcode at zoho.com > > > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > > I meant like "the entire system" not "just the compiler". Right, to link it all. > > > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > > On many systems, even X is just an "apt get whatever" away. > > And, if we form our .deb correctly, it is fairly automatic. > > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > > > > How to ease that apart? A sort of dependency on X? > > Likely separate packages, like cm3 and cm3-gui? > > > > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > > Imagine if, you know, our config files dropped away entirely. > > Imagine if the config file was generated at install time by autoconf. > > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > > Install from binaries is faster. > > Install from source is problematic for compilers written in themselves. Even gcc. > > > > > > - Jay > > > > > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > > From: lists at darko.org > > To: m3devel at elegosoft.com; microcode at zoho.com > > Subject: Re: [M3devel] Build Server - Plan > > > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > > > The solution for SPARC might be to create a cross-compiler. That's not > > > > > ideal but would be useful for verifying that things compile for the > > > > > platform at least, short of having the right hardware. > > > > > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > > > machine time but I can't keep the machine running all the time. Hopefully > > > > the current setup (opencsw) is still viable. > > > > > > > > > I disagree with your "only for experts" assessment. The point of this > > > > > server is that you don't have to compiler the compiler and a backend just > > > > > to get the latest (or even a properly working) compiler. That's the very > > > > > "expert" work I'm trying to automate. > > > > > > > > That is good to hear because from your list of deliverables it seemed to me > > > > there would still be a lot of tinkering to get a complete install. If you > > > > are scripting that and it can be built effortlessly and mindlessly from the > > > > deliverables then good. I think it would be ideal and very nice if people > > > > could download a turnkey tarball ready to run on their platform, or source > > > > code ready to build and an installer script that installs the final > > > > product. Jay mentioned autotools and they are usually far from optimal on > > > > Solaris because of autotools reliance on so much gnu stuff that is not > > > > always necessarily present on Solaris but that is the general idea- to be > > > > able to configure && make && make install with all the standard stuff > > > > happening like everything being installed in /usr/local/bin|lib|share and so > > > > forth. > > > > > > > > > What this system spits out is everything you need to compile anything you > > > > > want - a complete and working compiler. Not everyone wants or needs to use > > > > > all the packages. If you do, you can. Here are the instructions for doing > > > > > that: "cd ; cm3 " > > > > > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > > > sorry if I wasted anybody's time with this diversion. I have been following > > > > the mailing list and it looks like Jay and Rodney are mostly discussing > > > > things with a very small number of regulars. Coming from the outside as I do > > > > I think anybody new is going to have similar questions/misunderstandings of > > > > how cm3 is built and installed, etc. > > > > > > > > > I'll be opening up the server to anyone who wants to do something > > > > > different. If someone wants to produce tarballs they can setup their own > > > > > VMs and they will get built too. > > > > > > > > Hopefully this can be integrated with what you already have running on > > > > opencsw. > > > > > > > > Thank you. > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 00:20:25 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 19 Aug 2015 17:20:25 -0500 Subject: [M3devel] cm3 is broken Message-ID: <55D50129.6030600@lcwb.coop> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into runaway recursion trying to report a fault. This happens immediately upon running it, before it produces any output. Here is one cycle of the backtrace: #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 #1748 0x00007fd9b0b0ccfe in Raise (act= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) at ../src/runtime/common/RTHooks.m3:79 #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 So far, I haven't had the patience to keep hitting return until I get to the bottom of it. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 01:57:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 19 Aug 2015 23:57:54 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: Did you upgrade.py? (or possibly upgrade.sh?) I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. I run upgrade.py frequently and recommend it. - Jay > Date: Wed, 19 Aug 2015 17:20:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: [M3devel] cm3 is broken > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 20 11:38:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 20 Aug 2015 09:38:37 +0000 Subject: [M3devel] First_readable_addr? Message-ID: First_readable_addr looks off by a factor of 8.But I haven't gotten it to trigger at all yet. I'll debug further. Proposals: - remove First_readable_addr as wrong or non functional? - fix First_readable_addr - Most importantly, First_readable_addr is I believe usually 4k, sometimes 8k.It is 8k for sparc. It "should" also be 8k for ia64 and alpha. However, the proposal is to make it always 4k, for more commonality among targets.Lower is safe, but a possible very slight loss in efficiency when it is too low. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 17:44:39 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 10:44:39 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> Message-ID: <55D5F5E7.5000109@lcwb.coop> On 08/19/2015 06:57 PM, Jay K wrote: > Did you upgrade.py? (or possibly upgrade.sh?) > upgrade.py does this: rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py Traceback (most recent call last): File "upgrade.py", line 4, in import pylib File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in if Target.startswith("NT386"): AttributeError: 'NoneType' object has no attribute 'startswith' I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', with front removed from the m3cc line in pkginfo.txt, since there were no changes to it, and it takes longer to compile than everything else combined. upgrade.sh now segfaults after cleaning things out, but that is no doubt just the now-broken cm3. I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing a compiler, but restoring what I had before does not fix the problem. I suppose it is in m3core, which I guess I need to add to my backup habit. > I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > I run upgrade.py frequently and recommend it. > > - Jay > > > > Date: Wed, 19 Aug 2015 17:20:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: [M3devel] cm3 is broken > > > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > runaway recursion trying to report a fault. This happens immediately upon > > running it, before it produces any output. > > > > Here is one cycle of the backtrace: > > > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > #1748 0x00007fd9b0b0ccfe in Raise (act= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > at ../src/runtime/common/RTHooks.m3:79 > > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 19:19:01 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 20 Aug 2015 10:19:01 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D5F5E7.5000109@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: Upgrade is failing because cm3 is already broken. I can see about making the error clearer. I often upgrade.py nocleangcc to speed it up. Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. Do you have recourse? Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) - Jay On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > > On 08/19/2015 06:57 PM, Jay K wrote: >> Did you upgrade.py? (or possibly upgrade.sh?) > > upgrade.py does this: > > rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > Traceback (most recent call last): > File "upgrade.py", line 4, in > import pylib > File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > if Target.startswith("NT386"): > AttributeError: 'NoneType' object has no attribute 'startswith' > > I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > with front removed from the m3cc line in pkginfo.txt, since there were no > changes to it, and it takes longer to compile than everything else combined. > > upgrade.sh now segfaults after cleaning things out, but that is no doubt just > the now-broken cm3. > > I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > a compiler, but restoring what I had before does not fix the problem. I suppose > it is in m3core, which I guess I need to add to my backup habit. > >> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> >> I run upgrade.py frequently and recommend it. >> >> - Jay >> >> >> > Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] cm3 is broken >> > >> > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > runaway recursion trying to report a fault. This happens immediately upon >> > running it, before it produces any output. >> > >> > Here is one cycle of the backtrace: >> > >> > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > #1748 0x00007fd9b0b0ccfe in Raise (act= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > at ../src/runtime/common/RTHooks.m3:79 >> > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 20 20:39:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 13:39:58 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: <55D61EFE.1040302@lcwb.coop> I have some old complete installations. One of them will probably be enough to build a compiler. No time just yet. On 08/20/2015 12:19 PM, Jay wrote: > Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > > I often upgrade.py nocleangcc to speed it up. > > > Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > > m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > > Do you have recourse? > Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > - Jay > > On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/19/2015 06:57 PM, Jay K wrote: >>> Did you upgrade.py? (or possibly upgrade.sh?) >> >> upgrade.py does this: >> >> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> Traceback (most recent call last): >> File "upgrade.py", line 4, in >> import pylib >> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> if Target.startswith("NT386"): >> AttributeError: 'NoneType' object has no attribute 'startswith' >> >> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> with front removed from the m3cc line in pkginfo.txt, since there were no >> changes to it, and it takes longer to compile than everything else combined. >> >> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> the now-broken cm3. >> >> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> a compiler, but restoring what I had before does not fix the problem. I suppose >> it is in m3core, which I guess I need to add to my backup habit. >> >>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>> >>> I run upgrade.py frequently and recommend it. >>> >>> - Jay >>> >>> >>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com >>>> Subject: [M3devel] cm3 is broken >>>> >>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>> runaway recursion trying to report a fault. This happens immediately upon >>>> running it, before it produces any output. >>>> >>>> Here is one cycle of the backtrace: >>>> >>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>> at ../src/runtime/common/RTHooks.m3:79 >>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> >>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From wagner at elego.de Fri Aug 21 14:46:44 2015 From: wagner at elego.de (Olaf Wagner) Date: Fri, 21 Aug 2015 14:46:44 +0200 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: References: Message-ID: <20150821144644.345ac1a25b429e6c35854d32@elego.de> On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rodney_bates at lcwb.coop Fri Aug 21 18:38:32 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 21 Aug 2015 11:38:32 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> Message-ID: <55D75408.3010900@lcwb.coop> On 08/20/2015 06:32 PM, Antony Hosking wrote: > cm3 should build with static link to m3core so as to allow backup as you describe. > Did this change recently? Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting a breakpoint at one of these places gives: (m3gdb) b RTExFrame.m3:175 No source file named RTExFrame.m3. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (RTExFrame.m3:175) pending. (m3gdb) run ........ Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. Pending breakpoint "RTExFrame.m3:175" resolved > >> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> >> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> >> >> I often upgrade.py nocleangcc to speed it up. >> >> >> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> >> >> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> >> >> Do you have recourse? >> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> >> - Jay >> >> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> >>> >>> >>> On 08/19/2015 06:57 PM, Jay K wrote: >>>> Did you upgrade.py? (or possibly upgrade.sh?) >>> >>> upgrade.py does this: >>> >>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >>> Traceback (most recent call last): >>> File "upgrade.py", line 4, in >>> import pylib >>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >>> if Target.startswith("NT386"): >>> AttributeError: 'NoneType' object has no attribute 'startswith' >>> >>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >>> with front removed from the m3cc line in pkginfo.txt, since there were no >>> changes to it, and it takes longer to compile than everything else combined. >>> >>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >>> the now-broken cm3. >>> >>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >>> a compiler, but restoring what I had before does not fix the problem. I suppose >>> it is in m3core, which I guess I need to add to my backup habit. >>> >>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>>> >>>> I run upgrade.py frequently and recommend it. >>>> >>>> - Jay >>>> >>>> >>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>>> From: rodney_bates at lcwb.coop >>>>> To: m3devel at elegosoft.com >>>>> Subject: [M3devel] cm3 is broken >>>>> >>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>> running it, before it produces any output. >>>>> >>>>> Here is one cycle of the backtrace: >>>>> >>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> >>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>> >>>>> -- >>>>> Rodney Bates >>>>> rodney.m.bates at acm.org >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 11:28:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 09:28:01 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 12:00:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 10:00:18 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: It works for me. Can you go back to a working install, maybe 5.8.6 with that slight config patch,run upgrade.py, add see what you get?Send the build log maybe? jair:python jay$ otool -L /cm3/bin/cm3/cm3/bin/cm3: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) jair:python jay$ /cm3/bin/cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-22 09:57:19 configuration: /cm3/bin/cm3.cfg host: I386_DARWIN target: I386_DARWIN On most other systems, use ldd.On Windows use link /dump /imports or link /dump /dependents. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu; rodney.m.bates at acm.org Date: Sat, 22 Aug 2015 09:28:01 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3 is broken This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Aug 22 17:44:04 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 10:44:04 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: <55D898C4.90305@lcwb.coop> Going back several cm3 backups until I see a change, I have: root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 linux-vdso.so.1 => (0x00007fff211fe000) libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-07-31 18:44:16 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 linux-vdso.so.1 => (0x00007fffa47a7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-06-04 14:24:48 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX So I compiled the change to dynamic between June 4 and July 31. On 08/22/2015 05:00 AM, Jay K wrote: > It works for me. > > Can you go back to a working install, maybe 5.8.6 with that slight config patch, > run upgrade.py, add see what you get? > Send the build log maybe? > > > jair:python jay$ otool -L /cm3/bin/cm3 > /cm3/bin/cm3: > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) > /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) > > jair:python jay$ /cm3/bin/cm3 -version > > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-22 09:57:19 > configuration: /cm3/bin/cm3.cfg > host: I386_DARWIN > target: I386_DARWIN > > > On most other systems, use ldd. > On Windows use link /dump /imports or link /dump /dependents. > > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: hosking at purdue.edu; rodney.m.bates at acm.org > Date: Sat, 22 Aug 2015 09:28:01 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken > > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:07:01 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:07:01 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: <55D89E25.9000002@lcwb.coop> On 08/22/2015 04:28 AM, Jay K wrote: > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". I think this is a very good idea. The in-place rebuilds always make me very nervous. I never feel like I quite know what is going on, and I never quite trust any of the processes we have (and rightly so, from experience). When it works, or appears to, it always is more a matter of experiment than understanding. Whenever I am the least bit suspicious, I feel the need to copy two whole subtrees (source & installed) to update in place, keeping the originals to make another copy from, if problems arise. But that's a pretty tedious process to go through every time, so I have to evaluate the risks and decide when to take them and when to play safe. And when I touch the compiler or runtime, I feel the need to rebuild all of front twice, by whatever process I am using, then look for at least identical sized executables and libraries. > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:33:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:33:12 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: <55D8A448.30703@lcwb.coop> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = VAR p := LOOPHOLE (f, PF1); BEGIN IF DEBUG THEN PutExcept ("INVOKE HANDLER", a); RTIO.PutText (" frame="); RTIO.PutAddr (f); RTIO.PutText (" class="); RTIO.PutInt (f.class); RTIO.PutText ("\n"); RTIO.Flush (); END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *) <* ASSERT p.jmpbuf # NIL *> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; (m3gdb) frame 0 #0 InvokeHandler (f=16_00007fff509f4af0, a= RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 175 <* ASSERT p.jmpbuf # NIL *> (m3gdb) p p.jmpbuf $31 = NIL --------^^^ (m3gdb) The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = VAR status: File.Status; fname := M3toC.SharedTtoS(pn); BEGIN IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; ---------^^^^^^^^^^^^^^^^^^^^^^ M3toC.FreeSharedS(pn, fname); RETURN status END Status; (m3gdb) #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p pn $30 = (*16_00000000013e4ac0*) "./cm3.cfg" --------------------------------^^^^^^^^^^^ (m3gdb) I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? Also, I notice one other thing, that probably isn't part of this problem, but maybe needs to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, after the failing CStatus above. Obviously not thread safe. Does it matter here? As I recall from many years ago, there was a change in C library that provided a thread safe alternative to this, and I even think I had to change something in pm3 or SRC to adapt to it. Do we care? PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = VAR err := Cerrno.GetErrno(); ---------------^^^^^^^^^^^^^^^^^ BEGIN M3toC.FreeSharedS(p, f); OSErrorPosix.Raise0(err); END Fail; On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:41:14 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:41:14 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A448.30703@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> Message-ID: <55D8A62A.8030205@lcwb.coop> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > VAR p := LOOPHOLE (f, PF1); > BEGIN > IF DEBUG THEN > PutExcept ("INVOKE HANDLER", a); > RTIO.PutText (" frame="); RTIO.PutAddr (f); > RTIO.PutText (" class="); RTIO.PutInt (f.class); > RTIO.PutText ("\n"); > RTIO.Flush (); > END; > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > p.info := a; (* copy the exception to the new frame *) > <* ASSERT p.jmpbuf # NIL *> > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > RAISE OUCH; > END InvokeHandler; > > (m3gdb) frame 0 > #0 InvokeHandler (f=16_00007fff509f4af0, a= > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > 175 <* ASSERT p.jmpbuf # NIL *> > (m3gdb) p p.jmpbuf > $31 = NIL > --------^^^ > (m3gdb) > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > BEGIN > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > ---------^^^^^^^^^^^^^^^^^^^^^^ > M3toC.FreeSharedS(pn, fname); > RETURN status > END Status; > > (m3gdb) > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p pn > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > --------------------------------^^^^^^^^^^^ > (m3gdb) > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? A bit more info: (m3gdb) down #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p Process.GetWorkingDirectory() $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (m3gdb) This is where I ran cm3 from. > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > after the failing CStatus above. Obviously not thread safe. Does it matter here? > As I recall from many years ago, there was a change in C library that provided a thread > safe alternative to this, and I even think I had to change something in pm3 or SRC to > adapt to it. Do we care? > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > VAR err := Cerrno.GetErrno(); > ---------------^^^^^^^^^^^^^^^^^ > BEGIN > M3toC.FreeSharedS(p, f); > OSErrorPosix.Raise0(err); > END Fail; > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> runaway recursion trying to report a fault. This happens immediately upon >> running it, before it produces any output. >> >> Here is one cycle of the backtrace: >> >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> at ../src/runtime/common/RTHooks.m3:79 >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 19:26:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 17:26:33 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> Message-ID: It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) The interface here between cm3's generated code/data and m3core has changed. You cannot mix them. It will fail. - Jay > Date: Sat, 22 Aug 2015 11:41:14 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] cm3 is broken: More info > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > VAR p := LOOPHOLE (f, PF1); > > BEGIN > > IF DEBUG THEN > > PutExcept ("INVOKE HANDLER", a); > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > RTIO.PutText ("\n"); > > RTIO.Flush (); > > END; > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > p.info := a; (* copy the exception to the new frame *) > > <* ASSERT p.jmpbuf # NIL *> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > RAISE OUCH; > > END InvokeHandler; > > > > (m3gdb) frame 0 > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > 175 <* ASSERT p.jmpbuf # NIL *> > > (m3gdb) p p.jmpbuf > > $31 = NIL > > --------^^^ > > (m3gdb) > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > BEGIN > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > M3toC.FreeSharedS(pn, fname); > > RETURN status > > END Status; > > > > (m3gdb) > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p pn > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > --------------------------------^^^^^^^^^^^ > > (m3gdb) > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > As I recall from many years ago, there was a change in C library that provided a thread > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > adapt to it. Do we care? > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > VAR err := Cerrno.GetErrno(); > > ---------------^^^^^^^^^^^^^^^^^ > > BEGIN > > M3toC.FreeSharedS(p, f); > > OSErrorPosix.Raise0(err); > > END Fail; > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >> runaway recursion trying to report a fault. This happens immediately upon > >> running it, before it produces any output. > >> > >> Here is one cycle of the backtrace: > >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >> at ../src/runtime/common/RTHooks.m3:79 > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >> > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 20:47:00 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 11:47:00 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> Message-ID: On the other matter -- Cerrno.GetErrno() is thread safe. It ends up as a function that accesses a thread local. At least on Windows and surely that is the only logical approach. It is still a bad design. Thread locals don't make things all better. They are still "fragile" (easy to change by accident), slow, and don't provide for reentrance. - Jay On Aug 22, 2015, at 9:41 AM, "Rodney M. Bates" wrote: > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> >> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> VAR p := LOOPHOLE (f, PF1); >> BEGIN >> IF DEBUG THEN >> PutExcept ("INVOKE HANDLER", a); >> RTIO.PutText (" frame="); RTIO.PutAddr (f); >> RTIO.PutText (" class="); RTIO.PutInt (f.class); >> RTIO.PutText ("\n"); >> RTIO.Flush (); >> END; >> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> p.info := a; (* copy the exception to the new frame *) >> <* ASSERT p.jmpbuf # NIL *> >> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> RAISE OUCH; >> END InvokeHandler; >> >> (m3gdb) frame 0 >> #0 InvokeHandler (f=16_00007fff509f4af0, a= >> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> 175 <* ASSERT p.jmpbuf # NIL *> >> (m3gdb) p p.jmpbuf >> $31 = NIL >> --------^^^ >> (m3gdb) >> >> >> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> >> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> BEGIN >> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> ---------^^^^^^^^^^^^^^^^^^^^^^ >> M3toC.FreeSharedS(pn, fname); >> RETURN status >> END Status; >> >> (m3gdb) >> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> at ../src/os/POSIX/FSPosix.m3:328 >> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> (m3gdb) p pn >> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> --------------------------------^^^^^^^^^^^ >> (m3gdb) >> >> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > >> >> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> after the failing CStatus above. Obviously not thread safe. Does it matter here? >> As I recall from many years ago, there was a change in C library that provided a thread >> safe alternative to this, and I even think I had to change something in pm3 or SRC to >> adapt to it. Do we care? >> >> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> VAR err := Cerrno.GetErrno(); >> ---------------^^^^^^^^^^^^^^^^^ >> BEGIN >> M3toC.FreeSharedS(p, f); >> OSErrorPosix.Raise0(err); >> END Fail; >> >> >> >> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>> runaway recursion trying to report a fault. This happens immediately upon >>> running it, before it produces any output. >>> >>> Here is one cycle of the backtrace: >>> >>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>> at ../src/runtime/common/RTHooks.m3:79 >>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> >>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 23:50:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 14:50:42 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D898C4.90305@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> <55D75408.3010900@lcwb.coop> <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> <55D898C4.90305@lcwb.coop> Message-ID: <2A7A1326-A801-4BFF-8C1D-4E50DB646272@gmail.com> Go back to one that is statically linked and upgrade from that? upgrade.py nocleangcc doesn't take too long. The backup must have, in general, cm3 cm3cg config libm3 m3core. We can relax the libm3 requirement I believe, but let's deal with that later separately. - Jay On Aug 22, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > Going back several cm3 backups until I see a change, I have: > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 > linux-vdso.so.1 => (0x00007fff211fe000) > libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) > libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) > libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) > /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-07-31 18:44:16 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 > linux-vdso.so.1 => (0x00007fffa47a7000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) > /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-06-04 14:24:48 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > So I compiled the change to dynamic between June 4 and July 31. > > > On 08/22/2015 05:00 AM, Jay K wrote: >> It works for me. >> >> Can you go back to a working install, maybe 5.8.6 with that slight config patch, >> run upgrade.py, add see what you get? >> Send the build log maybe? >> >> >> jair:python jay$ otool -L /cm3/bin/cm3 >> /cm3/bin/cm3: >> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) >> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) >> >> jair:python jay$ /cm3/bin/cm3 -version >> >> Critical Mass Modula-3 version d5.10.0 >> last updated: 2015-05-21 >> compiled: 2015-08-22 09:57:19 >> configuration: /cm3/bin/cm3.cfg >> host: I386_DARWIN >> target: I386_DARWIN >> >> >> On most other systems, use ldd. >> On Windows use link /dump /imports or link /dump /dependents. >> >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: hosking at purdue.edu; rodney.m.bates at acm.org >> Date: Sat, 22 Aug 2015 09:28:01 +0000 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 is broken >> >> This was not meant to change and I haven't seen that it did change. >> Maybe my cleaning up of the config files. I'll check. >> >> >> I understand the risks well. I have a potential plan to change it. But that is another matter. >> >> You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. >> >> The wrong uprade sequence still breaks cm3. That is at it should be. >> >> >> Tangentially, here is my plan: >> The upgrade process must be "to the side and switch all at once", not "in place". >> It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all >> into the install in tight sequence (not atomically). The trick on NT would be to unmap the >> executable, and avoid returning to it -- some sort of "continuation passing style", >> possibly off into a little bit of C++, whose main task would be to move a few files >> and then exit. >> >> >> That'd still leave all the platforms to test on to see if a running executable can be replaced. >> esp. including AIX. >> >> >> Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this >> and just leave it as an idea/plan that I don't get to >> >> - Jay >> >> >> >> >> > Subject: Re: [M3devel] cm3 is broken >> > From: hosking at purdue.edu >> > Date: Sat, 22 Aug 2015 06:48:15 +1000 >> > CC: jay.krell at cornell.edu; m3devel at elegosoft.com >> > To: rodney.m.bates at acm.org >> > >> > That is a regression. >> > >> > Sent from my iPad >> > >> > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: >> > > >> > > >> > > >> > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: >> > >> cm3 should build with static link to m3core so as to allow backup as you describe. >> > >> Did this change recently? >> > > >> > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting >> > > a breakpoint at one of these places gives: >> > > >> > > >> > > (m3gdb) b RTExFrame.m3:175 >> > > No source file named RTExFrame.m3. >> > > Make breakpoint pending on future shared library load? (y or [n]) y >> > > Breakpoint 1 (RTExFrame.m3:175) pending. >> > > (m3gdb) run >> > > >> > > ........ >> > > >> > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. >> > > Pending breakpoint "RTExFrame.m3:175" resolved >> > > >> > > >> > > >> > >> >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> > >>> >> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> > >>> >> > >>> >> > >>> I often upgrade.py nocleangcc to speed it up. >> > >>> >> > >>> >> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> > >>> >> > >>> >> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> > >>> >> > >>> >> > >>> Do you have recourse? >> > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> > >>> >> > >>> - Jay >> > >>> >> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> > >>>> >> > >>>> >> > >>>> >> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: >> > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) >> > >>>> >> > >>>> upgrade.py does this: >> > >>>> >> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> > >>>> Traceback (most recent call last): >> > >>>> File "upgrade.py", line 4, in >> > >>>> import pylib >> > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> > >>>> if Target.startswith("NT386"): >> > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' >> > >>>> >> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no >> > >>>> changes to it, and it takes longer to compile than everything else combined. >> > >>>> >> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> > >>>> the now-broken cm3. >> > >>>> >> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose >> > >>>> it is in m3core, which I guess I need to add to my backup habit. >> > >>>> >> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> > >>>>> >> > >>>>> I run upgrade.py frequently and recommend it. >> > >>>>> >> > >>>>> - Jay >> > >>>>> >> > >>>>> >> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > >>>>>> From: rodney_bates at lcwb.coop >> > >>>>>> To: m3devel at elegosoft.com >> > >>>>>> Subject: [M3devel] cm3 is broken >> > >>>>>> >> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >>>>>> runaway recursion trying to report a fault. This happens immediately upon >> > >>>>>> running it, before it produces any output. >> > >>>>>> >> > >>>>>> Here is one cycle of the backtrace: >> > >>>>>> >> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >>>>>> at ../src/runtime/common/RTHooks.m3:79 >> > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> >> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Rodney Bates >> > >>>>>> rodney.m.bates at acm.org >> > >>>>>> _______________________________________________ >> > >>>>>> M3devel mailing list >> > >>>>>> M3devel at elegosoft.com >> > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >>>> >> > >>>> -- >> > >>>> Rodney Bates >> > >>>> rodney.m.bates at acm.org >> > >>> _______________________________________________ >> > >>> M3devel mailing list >> > >>> M3devel at elegosoft.com >> > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >> > >> >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 23:54:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 16:54:26 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop> Message-ID: <55D8EF92.20403@lcwb.coop> On 08/22/2015 12:26 PM, Jay K wrote: > It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > > > Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run independently. After a good lot of successful-looking output, I get: new source -> compiling LLGen.i3 new source -> compiling LLGen.m3 new source -> compiling Version.i3 new source -> compiling M3Backend.i3 new source -> compiling Arg.i3 new source -> compiling Utils.i3 new source -> compiling Msg.i3 new source -> compiling M3Backend.m3 new source -> compiling UtilsPosix.m3 new source -> compiling Arg.m3 new source -> compiling M3Loc.i3 new source -> compiling M3Unit.i3 new source -> compiling Builder.i3 new source -> compiling M3Options.i3 new source -> compiling WebFile.i3 new source -> compiling Dirs.i3 new source -> compiling Builder.m3 new source -> compiling Dirs.m3 new source -> compiling M3Build.i3 new source -> compiling M3Build.m3 new source -> compiling M3Loc.m3 new source -> compiling M3Options.m3 new source -> compiling M3Unit.m3 new source -> compiling Makefile.i3 new source -> compiling Makefile.m3 new source -> compiling Msg.m3 new source -> compiling Utils.m3 new source -> compiling WebFile.m3 new source -> compiling Main.m3 "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure 1 warning encountered new source -> compiling cm3unix.c new exporters -> recompiling Utils.i3 -> linking cm3 --- shipping from AMD64_LINUX --- . => /usr/local/cm3/pkg/cm3/AMD64_LINUX .M3EXPORTS .M3WEB ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX cm3unix.c Version.i3 ../src => /usr/local/cm3/pkg/cm3/src M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 Arg.m3 Builder.m3 Builder.i3 Dirs.i3 Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 Msg.m3 Utils.m3 Utils.i3 WebFile.i3 WebFile.m3 Main.m3 ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy LLGen.m3 LLGen.i3 . => /usr/local/cm3/bin cm3 . => /usr/local/cm3/man/man1 cm3.1 . => /usr/local/cm3/man/man7 m3makefile.7 Segmentation fault (core dumped) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done ------------------------------------------------------------------------------------- several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: ------------------------------------------------------------------------------------- /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ *** execution of [, ] failed *** > The interface here between cm3's generated code/data and m3core has changed. > You cannot mix them. It will fail. > > > - Jay > > > > > > Date: Sat, 22 Aug 2015 11:41:14 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] cm3 is broken: More info > > > > > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > > VAR p := LOOPHOLE (f, PF1); > > > BEGIN > > > IF DEBUG THEN > > > PutExcept ("INVOKE HANDLER", a); > > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > > RTIO.PutText ("\n"); > > > RTIO.Flush (); > > > END; > > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > > p.info := a; (* copy the exception to the new frame *) > > > <* ASSERT p.jmpbuf # NIL *> > > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > > RAISE OUCH; > > > END InvokeHandler; > > > > > > (m3gdb) frame 0 > > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > > 175 <* ASSERT p.jmpbuf # NIL *> > > > (m3gdb) p p.jmpbuf > > > $31 = NIL > > > --------^^^ > > > (m3gdb) > > > > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > > BEGIN > > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > > M3toC.FreeSharedS(pn, fname); > > > RETURN status > > > END Status; > > > > > > (m3gdb) > > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > > at ../src/os/POSIX/FSPosix.m3:328 > > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > (m3gdb) p pn > > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > > --------------------------------^^^^^^^^^^^ > > > (m3gdb) > > > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > > > A bit more info: > > > > (m3gdb) down > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p Process.GetWorkingDirectory() > > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > (m3gdb) > > > > This is where I ran cm3 from. > > > > > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > > As I recall from many years ago, there was a change in C library that provided a thread > > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > > adapt to it. Do we care? > > > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > > VAR err := Cerrno.GetErrno(); > > > ---------------^^^^^^^^^^^^^^^^^ > > > BEGIN > > > M3toC.FreeSharedS(p, f); > > > OSErrorPosix.Raise0(err); > > > END Fail; > > > > > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >> runaway recursion trying to report a fault. This happens immediately upon > > >> running it, before it produces any output. > > >> > > >> Here is one cycle of the backtrace: > > >> > > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >> at ../src/runtime/common/RTHooks.m3:79 > > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> > > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >> > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 23 01:37:19 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 16:37:19 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8EF92.20403@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> Message-ID: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. I upgraded multiple installations on multiple targets. I'll try again from 5.8.6. I suggest upgrade also check for the bad dynamic linking. Anyone else able/unable to upgrade? - Jay On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > On 08/22/2015 12:26 PM, Jay K wrote: >> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >> >> >> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install > directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > independently. > > After a good lot of successful-looking output, I get: > > new source -> compiling LLGen.i3 > new source -> compiling LLGen.m3 > new source -> compiling Version.i3 > new source -> compiling M3Backend.i3 > new source -> compiling Arg.i3 > new source -> compiling Utils.i3 > new source -> compiling Msg.i3 > new source -> compiling M3Backend.m3 > new source -> compiling UtilsPosix.m3 > new source -> compiling Arg.m3 > new source -> compiling M3Loc.i3 > new source -> compiling M3Unit.i3 > new source -> compiling Builder.i3 > new source -> compiling M3Options.i3 > new source -> compiling WebFile.i3 > new source -> compiling Dirs.i3 > new source -> compiling Builder.m3 > new source -> compiling Dirs.m3 > new source -> compiling M3Build.i3 > new source -> compiling M3Build.m3 > new source -> compiling M3Loc.m3 > new source -> compiling M3Options.m3 > new source -> compiling M3Unit.m3 > new source -> compiling Makefile.i3 > new source -> compiling Makefile.m3 > new source -> compiling Msg.m3 > new source -> compiling Utils.m3 > new source -> compiling WebFile.m3 > new source -> compiling Main.m3 > "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > 1 warning encountered > new source -> compiling cm3unix.c > new exporters -> recompiling Utils.i3 > -> linking cm3 > --- shipping from AMD64_LINUX --- > > . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > .M3EXPORTS .M3WEB > ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > cm3unix.c Version.i3 > ../src => /usr/local/cm3/pkg/cm3/src > M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > WebFile.m3 Main.m3 > ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > LLGen.m3 LLGen.i3 > . => /usr/local/cm3/bin > cm3 > . => /usr/local/cm3/man/man1 > cm3.1 > . => /usr/local/cm3/man/man7 > m3makefile.7 > Segmentation fault (core dumped) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > > > ------------------------------------------------------------------------------------- > several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > ------------------------------------------------------------------------------------- > > > /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > *** execution of [, ] failed *** > > > > > > > > > >> The interface here between cm3's generated code/data and m3core has changed. >> You cannot mix them. It will fail. >> >> >> - Jay >> >> >> >> >> > Date: Sat, 22 Aug 2015 11:41:14 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: Re: [M3devel] cm3 is broken: More info >> > >> > >> > >> > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> > > >> > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> > > VAR p := LOOPHOLE (f, PF1); >> > > BEGIN >> > > IF DEBUG THEN >> > > PutExcept ("INVOKE HANDLER", a); >> > > RTIO.PutText (" frame="); RTIO.PutAddr (f); >> > > RTIO.PutText (" class="); RTIO.PutInt (f.class); >> > > RTIO.PutText ("\n"); >> > > RTIO.Flush (); >> > > END; >> > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> > > p.info := a; (* copy the exception to the new frame *) >> > > <* ASSERT p.jmpbuf # NIL *> >> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> > > RAISE OUCH; >> > > END InvokeHandler; >> > > >> > > (m3gdb) frame 0 >> > > #0 InvokeHandler (f=16_00007fff509f4af0, a= >> > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > > 175 <* ASSERT p.jmpbuf # NIL *> >> > > (m3gdb) p p.jmpbuf >> > > $31 = NIL >> > > --------^^^ >> > > (m3gdb) >> > > >> > > >> > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> > > >> > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> > > BEGIN >> > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > ---------^^^^^^^^^^^^^^^^^^^^^^ >> > > M3toC.FreeSharedS(pn, fname); >> > > RETURN status >> > > END Status; >> > > >> > > (m3gdb) >> > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > > at ../src/os/POSIX/FSPosix.m3:328 >> > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > (m3gdb) p pn >> > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> > > --------------------------------^^^^^^^^^^^ >> > > (m3gdb) >> > > >> > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >> > >> > A bit more info: >> > >> > (m3gdb) down >> > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > at ../src/os/POSIX/FSPosix.m3:328 >> > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > (m3gdb) p Process.GetWorkingDirectory() >> > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >> > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > (m3gdb) >> > >> > This is where I ran cm3 from. >> > >> > >> > > >> > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> > > after the failing CStatus above. Obviously not thread safe. Does it matter here? >> > > As I recall from many years ago, there was a change in C library that provided a thread >> > > safe alternative to this, and I even think I had to change something in pm3 or SRC to >> > > adapt to it. Do we care? >> > > >> > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> > > VAR err := Cerrno.GetErrno(); >> > > ---------------^^^^^^^^^^^^^^^^^ >> > > BEGIN >> > > M3toC.FreeSharedS(p, f); >> > > OSErrorPosix.Raise0(err); >> > > END Fail; >> > > >> > > >> > > >> > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >> runaway recursion trying to report a fault. This happens immediately upon >> > >> running it, before it produces any output. >> > >> >> > >> Here is one cycle of the backtrace: >> > >> >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >> at ../src/runtime/common/RTHooks.m3:79 >> > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 24 19:41:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 24 Aug 2015 17:41:51 +0000 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: <20150821144644.345ac1a25b429e6c35854d32@elego.de> References: , <20150821144644.345ac1a25b429e6c35854d32@elego.de> Message-ID: It is a bug and there is a workaround. See: https://connect.microsoft.com/VisualStudio/feedback/details/1651822/incorrect-mspdb140-dll-version-picked-in-x86-x64-cross-tools-environment I can probably workaround it in pylib.py also. - Jay Date: Fri, 21 Aug 2015 14:46:44 +0200 From: wagner at elego.de To: m3devel at elegosoft.com CC: sevettel at hotmail.de Subject: Re: [M3devel] critical mass 3 auf window10 On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 04:08:09 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 24 Aug 2015 21:08:09 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Message-ID: <55DBCE09.3020005@lcwb.coop> Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field jmpbuf of an exception frame being NIL. The only place I find that this field could be set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, in procedure CaptureState. From a version without this rework, it will require compiling the compiler twice, once to get a compiler that would generate this code, and again to get a compiler that executes it when run. You are saying the middle compiler legitimately raises and catches an exception, in order to run, so we can't get past this step. And it could do that in other places too. I am guessing you have re-bootstrapped your compiler through more than one intermediate state of source code, so you don't get this problem. But it has to work somehow from the original version, for everybody else. Will it work with the new compiler-generated code but old runtime? Seems like a long shot. On 08/22/2015 06:37 PM, Jay wrote: > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > I upgraded multiple installations on multiple targets. > I'll try again from 5.8.6. > > > I suggest upgrade also check for the bad dynamic linking. > > > Anyone else able/unable to upgrade? > > - Jay > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > >> >> >> On 08/22/2015 12:26 PM, Jay K wrote: >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >>> >>> >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) >> >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run >> independently. >> >> After a good lot of successful-looking output, I get: >> >> new source -> compiling LLGen.i3 >> new source -> compiling LLGen.m3 >> new source -> compiling Version.i3 >> new source -> compiling M3Backend.i3 >> new source -> compiling Arg.i3 >> new source -> compiling Utils.i3 >> new source -> compiling Msg.i3 >> new source -> compiling M3Backend.m3 >> new source -> compiling UtilsPosix.m3 >> new source -> compiling Arg.m3 >> new source -> compiling M3Loc.i3 >> new source -> compiling M3Unit.i3 >> new source -> compiling Builder.i3 >> new source -> compiling M3Options.i3 >> new source -> compiling WebFile.i3 >> new source -> compiling Dirs.i3 >> new source -> compiling Builder.m3 >> new source -> compiling Dirs.m3 >> new source -> compiling M3Build.i3 >> new source -> compiling M3Build.m3 >> new source -> compiling M3Loc.m3 >> new source -> compiling M3Options.m3 >> new source -> compiling M3Unit.m3 >> new source -> compiling Makefile.i3 >> new source -> compiling Makefile.m3 >> new source -> compiling Msg.m3 >> new source -> compiling Utils.m3 >> new source -> compiling WebFile.m3 >> new source -> compiling Main.m3 >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure >> 1 warning encountered >> new source -> compiling cm3unix.c >> new exporters -> recompiling Utils.i3 >> -> linking cm3 >> --- shipping from AMD64_LINUX --- >> >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> .M3EXPORTS .M3WEB >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> cm3unix.c Version.i3 >> ../src => /usr/local/cm3/pkg/cm3/src >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 >> WebFile.m3 Main.m3 >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy >> LLGen.m3 LLGen.i3 >> . => /usr/local/cm3/bin >> cm3 >> . => /usr/local/cm3/man/man1 >> cm3.1 >> . => /usr/local/cm3/man/man7 >> m3makefile.7 >> Segmentation fault (core dumped) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done >> >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done >> >> >> ------------------------------------------------------------------------------------- >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: >> ------------------------------------------------------------------------------------- >> >> >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> *** execution of [, ] failed *** >> >> >> >> >> >> >> >> >> >>> The interface here between cm3's generated code/data and m3core has changed. >>> You cannot mix them. It will fail. >>> >>> >>> - Jay >>> >>> >>> >>> >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu >>>> Subject: Re: [M3devel] cm3 is broken: More info >>>> >>>> >>>> >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >>>>> >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >>>>> VAR p := LOOPHOLE (f, PF1); >>>>> BEGIN >>>>> IF DEBUG THEN >>>>> PutExcept ("INVOKE HANDLER", a); >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); >>>>> RTIO.PutText ("\n"); >>>>> RTIO.Flush (); >>>>> END; >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >>>>> p.info := a; (* copy the exception to the new frame *) >>>>> <* ASSERT p.jmpbuf # NIL *> >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >>>>> RAISE OUCH; >>>>> END InvokeHandler; >>>>> >>>>> (m3gdb) frame 0 >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> 175 <* ASSERT p.jmpbuf # NIL *> >>>>> (m3gdb) p p.jmpbuf >>>>> $31 = NIL >>>>> --------^^^ >>>>> (m3gdb) >>>>> >>>>> >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >>>>> >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >>>>> BEGIN >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ >>>>> M3toC.FreeSharedS(pn, fname); >>>>> RETURN status >>>>> END Status; >>>>> >>>>> (m3gdb) >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>>> at ../src/os/POSIX/FSPosix.m3:328 >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> (m3gdb) p pn >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >>>>> --------------------------------^^^^^^^^^^^ >>>>> (m3gdb) >>>>> >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >>>> >>>> A bit more info: >>>> >>>> (m3gdb) down >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>> at ../src/os/POSIX/FSPosix.m3:328 >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>> (m3gdb) p Process.GetWorkingDirectory() >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> (m3gdb) >>>> >>>> This is where I ran cm3 from. >>>> >>>> >>>>> >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? >>>>> As I recall from many years ago, there was a change in C library that provided a thread >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to >>>>> adapt to it. Do we care? >>>>> >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >>>>> VAR err := Cerrno.GetErrno(); >>>>> ---------------^^^^^^^^^^^^^^^^^ >>>>> BEGIN >>>>> M3toC.FreeSharedS(p, f); >>>>> OSErrorPosix.Raise0(err); >>>>> END Fail; >>>>> >>>>> >>>>> >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>>> running it, before it produces any output. >>>>>> >>>>>> Here is one cycle of the backtrace: >>>>>> >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>>> >>>>> >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 25 06:12:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55DBCE09.3020005@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> ,<55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop> Message-ID: Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 25 11:56:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 09:56:36 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: I should have said you are close, it makes mostly sense, but not entirely. Anyway.. upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: % cm3 -versionCritical Mass Modula-3 version 5.8.6 last updated: 2010-04-11 compiled: 2010-07-12 20:10:34 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX to % cm3 -versionCritical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 09:33:26 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX in "one" step (automating many steps) Here is from scripts/python/capture-boot.py: https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz Do you have any local edits? Removed stuff or reordered in pkginfo.txt? (I edited one line in 5.8.6's config, to stop looking for m3cc). - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: Re: [M3devel] cm3 is broken: More info Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 22:08:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 25 Aug 2015 15:08:41 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: <55DCCB49.80103@lcwb.coop> OK, I got a working compiler from 5.8.6, and it is now statically linked to all modula-3 libs. rodney at allegheny:~/proj/m3/exp/trytemp/src$ cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 19:02:01 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX rodney at allegheny:~/proj/m3/exp/trytemp/src$ ldd /usr/local/cm3/bin/cm3 linux-vdso.so.1 => (0x00007fff54bfe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8cca5b8000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8cca39a000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8cc9fd2000) /lib64/ld-linux-x86-64.so.2 (0x00007f8cca8de000) I did a fresh pull of modula3-cm3, and removed and re-checked out a couple of locally edited git-tracked files, so my working is as github. git diff HEAD shows only a prompt. Then I make a full backup of my 5.8.6 installation directory, switched to it (the original), went to scripts/python, and did ./upgrade-full.sh 2>&1 | tee upgrade-full.sh.log. I did not do anything to disable rebuilding of m3cc. I had some changes in the last pull, and lots since 5.8.6. Eventually the rebuild of everything failed in cvsup, which is probably entirely irrelevant. There are many clean steps that look like: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ rm -rf AMD64_LINUX +++ ==> [] returned 0 Are they cleaning or failing to clean? In selected cases, they are followed by build and ship steps that do nothing: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done +++ /usr/local/cm3/bin/cm3 -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done So I must have had some non-bootstrappable intermediate version of something? > On 08/25/2015 04:56 AM, Jay K wrote: > I should have said you are close, it makes mostly sense, but not entirely. Anyway.. > > > upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: > > > > % cm3 -version > Critical Mass Modula-3 version 5.8.6 > last updated: 2010-04-11 > compiled: 2010-07-12 20:10:34 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > to > > > % cm3 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-25 09:33:26 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > in "one" step (automating many steps) > > Here is from scripts/python/capture-boot.py: > https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz > > > Do you have any local edits? > > > Removed stuff or reordered in pkginfo.txt? > > > (I edited one line in 5.8.6's config, to stop looking for m3cc). > > - Jay > > > >-- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 02:28:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 00:28:48 +0000 Subject: [M3devel] Target.First_readable_addr Message-ID: Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 26 07:54:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 05:54:25 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> References: , <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> Message-ID: At the very least, can we use the same number for all targets, 4K? I don't like this inconsistency: IF RTParams.IsPresent("checked") THEN EVAL F4096x8p2.F1(NIL); (* large -- gets null checked *) END; IF RTParams.IsPresent("unchecked") THEN EVAL F0.F1(NIL); (* small -- does not get null check *) END; jair:p263 jay$ ./I386_DARWIN/pgm @M3checked ****** runtime error:*** Attempt to reference an illegal memory location.*** file "../F4096x8p2.m3", line 8*** Abort trap: 6jair:p263 jay$ ./I386_DARWIN/pgm @M3unchecked ****** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0xb8dc5 = F1 + 0x6 in ../F0.m3*** I wonder if we have even implemented this for Windows, the unchecked form.The inefficient checked form is easy.Notice how "large" gives the file/line of the null deref, "small" does not. What do Java, gcj, C#, mono, .NET Native, go, rust do? (Extra credit to anyone who can show meboth their compiler outputting the checksand their generated code in a debugger..For Modula-3/cm3cg, grep for fault in the *ms files for *.c files) - Jay Subject: Re: [M3devel] Target.First_readable_addr From: hosking at purdue.edu Date: Wed, 26 Aug 2015 14:44:30 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu This optimization has some performance benefit.Surely we can simply fix the problems rather than removing the capability. On Aug 26, 2015, at 10:28 AM, Jay K wrote:Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 26 17:37:54 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 10:37:54 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: References: Message-ID: <55DDDD52.5090008@lcwb.coop> I have always thought "segfault, could it be a NIL pointer deref?", or whatever words to that effect, was a rather lame excuse of an error message, and not much help. Since this scheme doesn't fault at the point of the real deref, but afterwards, when a memory reference is actually made, possibly computed from that address, could the location given for the fault (whether line no or just code displacement) be wrong? Very wrong? TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; PROCEDURE Q ( ) = VAR Ptr : REF A := NIL; BEGIN P(Ptr^); <---deref occurs here END Q; .... lots of other code, different module, etc. ... PROCEDURE P ( VAR x : A ) = BEGIN x[119] := 17 <---memory ref & segfault don't happen until here. END P; Or is this scheme only used in selective places, such as implicit deref when putting a subscript onto a pointer? e.g.: Ptr[117] := 43; On 08/25/2015 07:28 PM, Jay K wrote: > Of course, my agenda is to remove target-specificity. > > > > Target.First_readable_addr is a target-specific optimization > to omit null checks. Or maybe range checks.. > > > The assumption is that dereferencing "actual" null is checked > by the hardware, so code doesn't have to be generated to check for null. > > > I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > > > The assumption is further that accesses to the first page, at least, > are subject to the same hardware checks, and can also be optimized away. > > > At first, glance, I thought this was based on offset + size of a static access. > > > For example: > > a: ARRAY [0...large] OF CHAR > > > a[0..4095] no check > a[4096 and up] would check > > > Target.First_deadable_addr is 4k on many targets; 8k on others. > Setting it too low would yield extra checks. > Setting it too high would yield missing checks and a loss of safety. > > > Here is what I actually found though. > > > - The check is based on the size of the type being accessed. > > - It is off by a factor of 8 -- there is confusing between m3middle and m3front > as to if it is bit number of a byte number. > > > small: ARRAY[0..100] OF CHAR > large:ARRAY[0..100000] OF CHAR > > no access to small gets checked and every access to larger gets checked. > > Should we do anything about this? > > In m3-sys/m3tests/src/p2/p263: > cm3 -realclean > cm3 -keep > grep fault /*ms > > > All the accesses are to offset 0. > So, by some expectation, no null checks are needed. > Null checks are output when the size of the > containing type is, for x86, larger than 4096*8. > > > The checks have been largely but not completely wrong/missing. > Safety behooves us to check though. > > - fix the factor of 8? > - make it 0?? too slow? > - make it 4k on all target? until such time as a target manifests with a smaller page size? > - base the checks on access offset + size, not containing size? > Containing size is conservative. It checks more than i think is meant. > > > I couldn't actually figure out the code here, I added various: > > IF RTParams.IsPresent("qual") THEN > RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > RTIO.Flush(); > END; > > and such to m3front to figure it out. > > > Thanks, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 18:10:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 26 Aug 2015 09:10:36 -0700 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DDDD52.5090008@lcwb.coop> References: <55DDDD52.5090008@lcwb.coop> Message-ID: <08316510-3124-4372-A514-3D4881D26966@gmail.com> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. - Jay On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > I have always thought "segfault, could it be a NIL pointer deref?", or whatever > words to that effect, was a rather lame excuse of an error message, and not > much help. > > Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > memory reference is actually made, possibly computed from that address, could the location > given for the fault (whether line no or just code displacement) be wrong? Very wrong? > > TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > > PROCEDURE Q ( ) > = VAR Ptr : REF A := NIL; > BEGIN > P(Ptr^); <---deref occurs here > END Q; > > .... lots of other code, different module, etc. ... > > PROCEDURE P ( VAR x : A ) > = BEGIN > x[119] := 17 <---memory ref & segfault don't happen until here. > END P; > > Or is this scheme only used in selective places, such as implicit deref > when putting a subscript onto a pointer? e.g.: > > Ptr[117] := 43; > > On 08/25/2015 07:28 PM, Jay K wrote: >> Of course, my agenda is to remove target-specificity. >> >> >> >> Target.First_readable_addr is a target-specific optimization >> to omit null checks. Or maybe range checks.. >> >> >> The assumption is that dereferencing "actual" null is checked >> by the hardware, so code doesn't have to be generated to check for null. >> >> >> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >> >> >> The assumption is further that accesses to the first page, at least, >> are subject to the same hardware checks, and can also be optimized away. >> >> >> At first, glance, I thought this was based on offset + size of a static access. >> >> >> For example: >> >> a: ARRAY [0...large] OF CHAR >> >> >> a[0..4095] no check >> a[4096 and up] would check >> >> >> Target.First_deadable_addr is 4k on many targets; 8k on others. >> Setting it too low would yield extra checks. >> Setting it too high would yield missing checks and a loss of safety. >> >> >> Here is what I actually found though. >> >> >> - The check is based on the size of the type being accessed. >> >> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >> as to if it is bit number of a byte number. >> >> >> small: ARRAY[0..100] OF CHAR >> large:ARRAY[0..100000] OF CHAR >> >> no access to small gets checked and every access to larger gets checked. >> >> Should we do anything about this? >> >> In m3-sys/m3tests/src/p2/p263: >> cm3 -realclean >> cm3 -keep >> grep fault /*ms >> >> >> All the accesses are to offset 0. >> So, by some expectation, no null checks are needed. >> Null checks are output when the size of the >> containing type is, for x86, larger than 4096*8. >> >> >> The checks have been largely but not completely wrong/missing. >> Safety behooves us to check though. >> >> - fix the factor of 8? >> - make it 0?? too slow? >> - make it 4k on all target? until such time as a target manifests with a smaller page size? >> - base the checks on access offset + size, not containing size? >> Containing size is conservative. It checks more than i think is meant. >> >> >> I couldn't actually figure out the code here, I added various: >> >> IF RTParams.IsPresent("qual") THEN >> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >> RTIO.Flush(); >> END; >> >> and such to m3front to figure it out. >> >> >> Thanks, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 27 04:23:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 21:23:05 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <08316510-3124-4372-A514-3D4881D26966@gmail.com> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> Message-ID: <55DE7489.9000609@lcwb.coop> On 08/26/2015 11:10 AM, Jay wrote: > Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > No, I disagree adamantly. The whole idea of linguistic support of parameters passed by reference, instead of the programmer manually concocting it by taking addresses and contents of pointers, is that the language ensures to the called procedure that the hidden pointer will be neither NIL, uninitialized, nor point to something of the wrong type. In fact, pass by ref is not even defined in terms of the compiler lowering into pointer twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the type system that should only happen in UNSAFE code. NIL deref, as a runtime error, should in concept happen when and where ^ is applied to a pointer, including the implied dereference when a subscript or field selection is applied directly without the ^ operator. It's purely an implementation question how to do this. If a segfault message will point to the line of code where the dereference happens, and before execution proceeds beyond that code, that would be marginally OK. (Only marginally, because the error message should be able to say with certainty that it's a NIL deref. Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his obligation, its effects can escape into safe code. > > - Jay > > On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >> words to that effect, was a rather lame excuse of an error message, and not >> much help. >> >> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >> memory reference is actually made, possibly computed from that address, could the location >> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >> >> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >> >> PROCEDURE Q ( ) >> = VAR Ptr : REF A := NIL; >> BEGIN >> P(Ptr^); <---deref occurs here >> END Q; >> >> .... lots of other code, different module, etc. ... >> >> PROCEDURE P ( VAR x : A ) >> = BEGIN >> x[119] := 17 <---memory ref & segfault don't happen until here. >> END P; >> >> Or is this scheme only used in selective places, such as implicit deref >> when putting a subscript onto a pointer? e.g.: >> >> Ptr[117] := 43; >> >> On 08/25/2015 07:28 PM, Jay K wrote: >>> Of course, my agenda is to remove target-specificity. >>> >>> >>> >>> Target.First_readable_addr is a target-specific optimization >>> to omit null checks. Or maybe range checks.. >>> >>> >>> The assumption is that dereferencing "actual" null is checked >>> by the hardware, so code doesn't have to be generated to check for null. >>> >>> >>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>> >>> >>> The assumption is further that accesses to the first page, at least, >>> are subject to the same hardware checks, and can also be optimized away. >>> >>> >>> At first, glance, I thought this was based on offset + size of a static access. >>> >>> >>> For example: >>> >>> a: ARRAY [0...large] OF CHAR >>> >>> >>> a[0..4095] no check >>> a[4096 and up] would check >>> >>> >>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>> Setting it too low would yield extra checks. >>> Setting it too high would yield missing checks and a loss of safety. >>> >>> >>> Here is what I actually found though. >>> >>> >>> - The check is based on the size of the type being accessed. >>> >>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>> as to if it is bit number of a byte number. >>> >>> >>> small: ARRAY[0..100] OF CHAR >>> large:ARRAY[0..100000] OF CHAR >>> >>> no access to small gets checked and every access to larger gets checked. >>> >>> Should we do anything about this? >>> >>> In m3-sys/m3tests/src/p2/p263: >>> cm3 -realclean >>> cm3 -keep >>> grep fault /*ms >>> >>> >>> All the accesses are to offset 0. >>> So, by some expectation, no null checks are needed. >>> Null checks are output when the size of the >>> containing type is, for x86, larger than 4096*8. >>> >>> >>> The checks have been largely but not completely wrong/missing. >>> Safety behooves us to check though. >>> >>> - fix the factor of 8? >>> - make it 0?? too slow? >>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>> - base the checks on access offset + size, not containing size? >>> Containing size is conservative. It checks more than i think is meant. >>> >>> >>> I couldn't actually figure out the code here, I added various: >>> >>> IF RTParams.IsPresent("qual") THEN >>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>> RTIO.Flush(); >>> END; >>> >>> and such to m3front to figure it out. >>> >>> >>> Thanks, >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Aug 28 00:20:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 27 Aug 2015 17:20:58 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> <55DE7489.9000609@lcwb.coop> <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> Message-ID: <55DF8D4A.5050707@lcwb.coop> In any case, I would shed no tears if First_readable_addr were set target-indepenently to the lowest value on any target, since that would only increase the number of cases that give the proper error at the proper place & time. On 08/26/2015 10:07 PM, Antony Hosking wrote: > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: >> >> >> >> On 08/26/2015 11:10 AM, Jay wrote: >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. >>> >> >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by >> reference, instead of the programmer manually concocting it by taking addresses and >> contents of pointers, is that the language ensures to the called procedure that the >> hidden pointer will be neither NIL, uninitialized, nor point to something of the >> wrong type. >> >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the >> type system that should only happen in UNSAFE code. >> >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied >> to a pointer, including the implied dereference when a subscript or field selection is >> applied directly without the ^ operator. It's purely an implementation question how to >> do this. If a segfault message will point to the line of code where the dereference >> happens, and before execution proceeds beyond that code, that would be marginally OK. >> (Only marginally, because the error message should be able to say with certainty that >> it's a NIL deref. >> >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his >> obligation, its effects can escape into safe code. >> >>> >>> - Jay >>> >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: >>> >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >>>> words to that effect, was a rather lame excuse of an error message, and not >>>> much help. >>>> >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >>>> memory reference is actually made, possibly computed from that address, could the location >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >>>> >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >>>> >>>> PROCEDURE Q ( ) >>>> = VAR Ptr : REF A := NIL; >>>> BEGIN >>>> P(Ptr^); <---deref occurs here >>>> END Q; >>>> >>>> .... lots of other code, different module, etc. ... >>>> >>>> PROCEDURE P ( VAR x : A ) >>>> = BEGIN >>>> x[119] := 17 <---memory ref & segfault don't happen until here. >>>> END P; >>>> >>>> Or is this scheme only used in selective places, such as implicit deref >>>> when putting a subscript onto a pointer? e.g.: >>>> >>>> Ptr[117] := 43; >>>> >>>> On 08/25/2015 07:28 PM, Jay K wrote: >>>>> Of course, my agenda is to remove target-specificity. >>>>> >>>>> >>>>> >>>>> Target.First_readable_addr is a target-specific optimization >>>>> to omit null checks. Or maybe range checks.. >>>>> >>>>> >>>>> The assumption is that dereferencing "actual" null is checked >>>>> by the hardware, so code doesn't have to be generated to check for null. >>>>> >>>>> >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>>>> >>>>> >>>>> The assumption is further that accesses to the first page, at least, >>>>> are subject to the same hardware checks, and can also be optimized away. >>>>> >>>>> >>>>> At first, glance, I thought this was based on offset + size of a static access. >>>>> >>>>> >>>>> For example: >>>>> >>>>> a: ARRAY [0...large] OF CHAR >>>>> >>>>> >>>>> a[0..4095] no check >>>>> a[4096 and up] would check >>>>> >>>>> >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>>>> Setting it too low would yield extra checks. >>>>> Setting it too high would yield missing checks and a loss of safety. >>>>> >>>>> >>>>> Here is what I actually found though. >>>>> >>>>> >>>>> - The check is based on the size of the type being accessed. >>>>> >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>>>> as to if it is bit number of a byte number. >>>>> >>>>> >>>>> small: ARRAY[0..100] OF CHAR >>>>> large:ARRAY[0..100000] OF CHAR >>>>> >>>>> no access to small gets checked and every access to larger gets checked. >>>>> >>>>> Should we do anything about this? >>>>> >>>>> In m3-sys/m3tests/src/p2/p263: >>>>> cm3 -realclean >>>>> cm3 -keep >>>>> grep fault /*ms >>>>> >>>>> >>>>> All the accesses are to offset 0. >>>>> So, by some expectation, no null checks are needed. >>>>> Null checks are output when the size of the >>>>> containing type is, for x86, larger than 4096*8. >>>>> >>>>> >>>>> The checks have been largely but not completely wrong/missing. >>>>> Safety behooves us to check though. >>>>> >>>>> - fix the factor of 8? >>>>> - make it 0?? too slow? >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>>>> - base the checks on access offset + size, not containing size? >>>>> Containing size is conservative. It checks more than i think is meant. >>>>> >>>>> >>>>> I couldn't actually figure out the code here, I added various: >>>>> >>>>> IF RTParams.IsPresent("qual") THEN >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>>>> RTIO.Flush(); >>>>> END; >>>>> >>>>> and such to m3front to figure it out. >>>>> >>>>> >>>>> Thanks, >>>>> - Jay >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Aug 28 02:32:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:32:22 +0000 Subject: [M3devel] github history In-Reply-To: References: <73FE1A29-7F80-46DB-8B3C-1CA168C6FC92@purdue.edu>, , Message-ID: I don't really know what I'm doing with git. And it seems dangerous. I do know what I'm doing with Perforce, but oh well. I am a big fan of preserving history, so if I lost any, you can assume it was an accident. (That said, "local history" on my machine maybe isn't so valuable, noisy.) I really don't understand git merging. I tried using stash when there was a conflict, and that didn't go well. I don't know how to have multiple "checkouts" w/o multiple repositories. Wasteful. The web suggests some ways, but they didn't work for me -- e.g. cloning a local; I couldn't push from that. I gave up and created another large local repository. At least it was easy and fast enough. I don't want little temporary branches for this -- I don't want my one working directory to bounce between branches, I want multiple independent working directories against the same branch. - Jay Subject: Re: github history From: hosking at purdue.edu Date: Thu, 27 Aug 2015 14:54:57 +1000 CC: m3devel at elegosoft.com; jay.krell at cornell.edu; dragisha at m3w.org To: hosking at purdue.edu Update: I have successfully brought back history by: 1. setting the default branch to our old trunk,2. merging from master,3. deleting master,4. branching a new master from the (up-to-date) trunk, and5. setting the default branch back to master. On Aug 27, 2015, at 1:46 PM, Antony Hosking wrote:PS Jay, in poking around on github it seems that the source of this weirdness is a merge you did earlier this year. I suspect you were not using proper github workflow and it messed up the history (perhaps you inverted the merge?). Does anyone out there have better github fu than I do, who can dig into this and try to repair? Dragisha? On Aug 27, 2015, at 1:14 PM, Antony Hosking wrote:I am dismayed to find that the full commit histories on github do not appear to have made it over from the transition from svn to github. For example, I find the following very abbreviated history in the master tree for NilChkExpr.m3 in package m3front (see below). Did something strange happen in a merge somewhere along the way from the initial import into github? It looks like it occurred at the time of the commit "jaykrell on Jan 4, 2008 initial diff from ../config/PPC_LINUX" when it looks like the master got clobbered somehow. Anyone have any ideas if we can recover? Commits on May 4, 2008<1635728.png>add more information printed for some assertion failures .jaykrell authored on May 4, 200866139d4 Commits on Jan 4, 2008<1635728.png>initial diff from ../config/PPC_LINUXjaykrell authored on Jan 4, 20082412437 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 28 02:40:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:40:32 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DF8D4A.5050707@lcwb.coop> References: , <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com>, <55DE7489.9000609@lcwb.coop>, <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu>, <55DF8D4A.5050707@lcwb.coop> Message-ID: That is done -- it is a constant 4k now. Hypothetically Sparc/IA64/Alpha could be 8k. Sparc used to be. I'm still leary of the omitted checks and suggest we compare against C# (JIT and native, Microsoft and Mono), Java, Go, Rust, Gnat, etc. If most/all of them check aggressively, we should to imho. If they actually omit more aggressively, which I believe is possible, we should consider that. In particular, I think the omission should be based on the offset+size of the access, not of the containing type. i.e. the field and not the record. However this might be a fallacy also. e.g. TYPE BigRecord = RECORD ...lots.... INTEGER smallFieldPast4K END; PROCEDURE F1(VAR a:INTEGER); PROCEDURE F2(b:REF BigRecord) BEGIN F1(b.a); END F2; Hm. I see. Maybe you have to check at b.a and a (not shown here)? The recipient of a "small" type doesn't know its offset, if any, within a larger type, and it can vary. I can have: TYPE SmallRecord = RECORD .. nothing .. INTEGER smallFieldAtStart END; F1(smallRecord.a); So maybe you are right anyway -- check needed at the "address generation" not just the deref. I need to study this more. Another idea, perhaps, is that if there is any dereference of a parameter within a function, do all the checks up front, and then none in the function. You know -- in case there are loops. But I also don't know the guarantees around "dynamic paths" -- turning conditional dereferences into unconditional dereferences. I need to study this more. :) - Jay > Date: Thu, 27 Aug 2015 17:20:58 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] Target.First_readable_addr > > In any case, I would shed no tears if First_readable_addr were set target-indepenently > to the lowest value on any target, since that would only increase the number of cases > that give the proper error at the proper place & time. > > On 08/26/2015 10:07 PM, Antony Hosking wrote: > > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > > > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: > >> > >> > >> > >> On 08/26/2015 11:10 AM, Jay wrote: > >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > >>> > >> > >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by > >> reference, instead of the programmer manually concocting it by taking addresses and > >> contents of pointers, is that the language ensures to the called procedure that the > >> hidden pointer will be neither NIL, uninitialized, nor point to something of the > >> wrong type. > >> > >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer > >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual > >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. > >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the > >> type system that should only happen in UNSAFE code. > >> > >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied > >> to a pointer, including the implied dereference when a subscript or field selection is > >> applied directly without the ^ operator. It's purely an implementation question how to > >> do this. If a segfault message will point to the line of code where the dereference > >> happens, and before execution proceeds beyond that code, that would be marginally OK. > >> (Only marginally, because the error message should be able to say with certainty that > >> it's a NIL deref. > >> > >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault > >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his > >> obligation, its effects can escape into safe code. > >> > >>> > >>> - Jay > >>> > >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >>> > >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever > >>>> words to that effect, was a rather lame excuse of an error message, and not > >>>> much help. > >>>> > >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > >>>> memory reference is actually made, possibly computed from that address, could the location > >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? > >>>> > >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > >>>> > >>>> PROCEDURE Q ( ) > >>>> = VAR Ptr : REF A := NIL; > >>>> BEGIN > >>>> P(Ptr^); <---deref occurs here > >>>> END Q; > >>>> > >>>> .... lots of other code, different module, etc. ... > >>>> > >>>> PROCEDURE P ( VAR x : A ) > >>>> = BEGIN > >>>> x[119] := 17 <---memory ref & segfault don't happen until here. > >>>> END P; > >>>> > >>>> Or is this scheme only used in selective places, such as implicit deref > >>>> when putting a subscript onto a pointer? e.g.: > >>>> > >>>> Ptr[117] := 43; > >>>> > >>>> On 08/25/2015 07:28 PM, Jay K wrote: > >>>>> Of course, my agenda is to remove target-specificity. > >>>>> > >>>>> > >>>>> > >>>>> Target.First_readable_addr is a target-specific optimization > >>>>> to omit null checks. Or maybe range checks.. > >>>>> > >>>>> > >>>>> The assumption is that dereferencing "actual" null is checked > >>>>> by the hardware, so code doesn't have to be generated to check for null. > >>>>> > >>>>> > >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > >>>>> > >>>>> > >>>>> The assumption is further that accesses to the first page, at least, > >>>>> are subject to the same hardware checks, and can also be optimized away. > >>>>> > >>>>> > >>>>> At first, glance, I thought this was based on offset + size of a static access. > >>>>> > >>>>> > >>>>> For example: > >>>>> > >>>>> a: ARRAY [0...large] OF CHAR > >>>>> > >>>>> > >>>>> a[0..4095] no check > >>>>> a[4096 and up] would check > >>>>> > >>>>> > >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. > >>>>> Setting it too low would yield extra checks. > >>>>> Setting it too high would yield missing checks and a loss of safety. > >>>>> > >>>>> > >>>>> Here is what I actually found though. > >>>>> > >>>>> > >>>>> - The check is based on the size of the type being accessed. > >>>>> > >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front > >>>>> as to if it is bit number of a byte number. > >>>>> > >>>>> > >>>>> small: ARRAY[0..100] OF CHAR > >>>>> large:ARRAY[0..100000] OF CHAR > >>>>> > >>>>> no access to small gets checked and every access to larger gets checked. > >>>>> > >>>>> Should we do anything about this? > >>>>> > >>>>> In m3-sys/m3tests/src/p2/p263: > >>>>> cm3 -realclean > >>>>> cm3 -keep > >>>>> grep fault /*ms > >>>>> > >>>>> > >>>>> All the accesses are to offset 0. > >>>>> So, by some expectation, no null checks are needed. > >>>>> Null checks are output when the size of the > >>>>> containing type is, for x86, larger than 4096*8. > >>>>> > >>>>> > >>>>> The checks have been largely but not completely wrong/missing. > >>>>> Safety behooves us to check though. > >>>>> > >>>>> - fix the factor of 8? > >>>>> - make it 0?? too slow? > >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? > >>>>> - base the checks on access offset + size, not containing size? > >>>>> Containing size is conservative. It checks more than i think is meant. > >>>>> > >>>>> > >>>>> I couldn't actually figure out the code here, I added various: > >>>>> > >>>>> IF RTParams.IsPresent("qual") THEN > >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > >>>>> RTIO.Flush(); > >>>>> END; > >>>>> > >>>>> and such to m3front to figure it out. > >>>>> > >>>>> > >>>>> Thanks, > >>>>> - Jay > >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> M3devel mailing list > >>>>> M3devel at elegosoft.com > >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Aug 28 21:45:16 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 28 Aug 2015 14:45:16 -0500 Subject: [M3devel] llvm backend integration Message-ID: <55E0BA4C.7060802@lcwb.coop> The llvm back end now has an initial degree of integration into cm3. To get it, add M3_BACKEND_MODE="StAloneLlvmAsm" near the front of the m3makefile, or elsewhere in quake code that will get interpreted. It does the following: Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm bitcode IR, in file *.mb or *.ib. Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. Runs 'as' to assemble to object code, in file *.mo or *.io. Prerequisites: llc on the path. m3llvm on the path. as on the path. In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, see the m3makefile of llvm, which must give their locations. 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR and llvm IR are created, but you can get human-readable versions of them with 'm3cgcat' and 'llvm-dis'. Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line parameters to m3llvm. (Other targets are not supported). -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:17:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:17:05 -0500 Subject: [M3devel] llvm backend integration In-Reply-To: <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> References: <55E0BA4C.7060802@lcwb.coop> <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> Message-ID: <55E1F721.1090106@lcwb.coop> On 08/28/2015 07:08 PM, Antony Hosking wrote: > Sounds great! What's missing? > No provision for starting from an intermediate file format, as there is for m3cc. Also, it might be nice to get the human-readable forms of the two IRs as side outputs, for development work. Just a convenience, though. And, it just feels like Builder.m3 could use a good code review in general. > Sent from my iPad > >> On 29 Aug 2015, at 5:45 am, Rodney M. Bates wrote: >> >> The llvm back end now has an initial degree of integration into cm3. To get it, add >> >> M3_BACKEND_MODE="StAloneLlvmAsm" >> >> near the front of the m3makefile, or elsewhere in quake code that will get interpreted. >> >> It does the following: >> >> Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. >> >> Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm >> bitcode IR, in file *.mb or *.ib. >> >> Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. >> >> Runs 'as' to assemble to object code, in file *.mo or *.io. >> >> Prerequisites: >> llc on the path. >> m3llvm on the path. >> as on the path. >> >> In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, >> see the m3makefile of llvm, which must give their locations. >> >> 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR >> and llvm IR are created, but you can get human-readable versions of them with >> 'm3cgcat' and 'llvm-dis'. >> >> Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to >> /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line >> parameters to m3llvm. (Other targets are not supported). >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:54:46 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:54:46 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop> Message-ID: <55E1FFF6.1070102@lcwb.coop> On 08/29/2015 06:02 AM, Peter McKinna wrote: > Hi Rodney, > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, instead of the 3.5.0. > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > upgrade.py worked OK for me after I went back to the release compiler and did it from there, which is the case we really care about. Llvm changes themselves should not affect the upgrade process, since no llvm mode is used in building the distribution. > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm changed the debug info data structure design to make it not derived from Value. Not only did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, though systematic and simple. These are in separate M3 package llvmbindings. In the case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather than just what is currently used. I still am uncomfortable about how the complex C++ type hierarchies should be reflected in the bindings. I doesn't look feasible to reflect these into true M3 hierarcharies. But clients need some kind of conversions. Some functions return results that need to be passed back in later, but to a parameter having a static subtype or supertype thereof. The unwrap functions used in llvm appear to sometimes just silently convert (the equivalent of) failing NARROWs into NIL pointers, rather than runtime errors. This will inevitably make some bugs much harder to track down. I'm not even sure dynamic type checks alway happen. Maybe some undetected type errors can propagate into C++ and give inscrutable segfaults. I am hoping future llvm releases will not affect this so much. They do change things a lot. I also like to avoid chasing the new releases too often. Just as I was getting llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. This all compiled and linked, as last I checked, but I have done very little execution checking. These do require getting some llvm stuff in the right places and states. I have had similar frustrations as Jay has with trying to use git branches, so for now, I just made this a separate package. They generate alternative executables with comparable function, so should be easy to switch between. Eventually, we will have to decide what to do about getting needed llvm stuff in, in a standard place and also, how much work will be done outside of the cm3 build system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc would be cleanest for users, but it is very big, takes hours to compile, and maybe not everybody will want to use it badly enough to pay that price. Right now, you have to do building and installing of llvm on the side, then edit a few paths in m3makefiles. Keeping m3llvm as a completely separate executable means those who don't want to use it can be unbothered. With its still being in major development, this seems best for now. I originally set out on the path of having the functions of m3llvm linked in to cm3, as with the C backend, but am skeptical about all the llvm libraries that have to be built and linked in. > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > Those things will take some work, but should not be conceptually difficult. > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > Regards Peter > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > Peter, > > How actively are you working on the llvm back end? I don't see a lot of updates to > the github repository. I have checked in some minor things and think I am about > ready to do more. But I think it best we keep things merged as much as reasonable. > > One change is adding specialized support for calls on alloca. These are very > recently being generated as library calls by the front end, and come through > the whole process as link failures. I think it best to detect these and convert > to llvm alloca instructions. m3cc does something like this now. > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From lemming at henning-thielemann.de Sat Aug 29 21:56:33 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Sat, 29 Aug 2015 21:56:33 +0200 (CEST) Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> Message-ID: On Sat, 29 Aug 2015, Rodney M. Bates wrote: > I still am uncomfortable about how the complex C++ type hierarchies > should be reflected in the bindings. In Haskell I am calling the C interface to LLVM, like certainly many other high-level language bindings do. Would this be an option for Modula-3 bindings to LLVM, too? However, the C interface of LLVM is not well documented and maybe not complete. From jay.krell at cornell.edu Sun Aug 30 00:12:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:12:37 +0000 Subject: [M3devel] paths too truncated in assertion failures Message-ID: These messages: ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** 1) It should really be a full path. I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. Yes, full paths could "leak" across machines but I think that is ok. I did work on this long ago but people disagreed with at the time. 2) We could/should trim the root of the git checkout, so it sayse.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. Some C compilers have that feature, when people are conceredwith cross machine consistency for the sake of caching and similarbuild optimizations. I still prefer #1. 3) Or at least m3front/src/convert/Convert.m33b) or ../../m3front/src/convert/Convert.m3 if we really want it to be a correct relative path from the target directory. A big part of the problem is the build system is geared toward packages,not multiple packages. I'd like to solve this problem too somehow someday. Really -- #1 -- source paths should be full paths. C compilers vary here btw, in terms of what __FILE__ is and what isin debugging information.Sometimes whatever is passed on the command line is used.Sometimes full paths are computed by the compiler. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 00:20:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:20:16 +0000 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: I understand your dilemas.Static linking vs. dynamic.Cloning the source or not.Significantly growing our repository or not.Forking or not -- do we need any patches? Hopefully not but ok if we do. Clearly the C backend cheats -- the C compiler appears on the systemas if by magic, or various external means. I am somewhat keen on seeing this LLVM stuff work.And somewhat interested in helping. First, you know, there are many sayings and their redundancies and opposites: 1a too many cooks spoil the broth 1b 9 women cannot have a baby in one month (but they can have 9 in 9) 2 more/many hands make less/little work Which is it here, 1 or 2?i.e. do you need/want help? I have built LLVM now a few times.Yes it is large and takes time.Do we have any required patches for it?I'm still on the fence as to if we should "import" it, vs. depend on its presence"somehow". If there really is version sensitivity, that argues for importing.While "apt-get install llvm" might be in widespread availability,the exact "apt-get install llvm-3.5.0" I imagine isn't. The current setup obviously needs work.I'll likely commit something at least a little better. Do we really need 3.5.0, or can I use 3.5.2? > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > problems other people have had, guess I will have to trust upgrade.py. Really, I run this stuff all the time. On various hosts. What you/we/I do need though is a safe way to rebuild the compiler when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. In particular, if you have an ABI change, you need two different m3cores. The one for the working compiler and the one for the new compiler. Maybe I just need to use "buildlocal" more. It doesn't help that the C backend isn't ABI compatible with cm3cg, due to nested functions and static links. cm3cg uses a target-dependent extra register not accessible to C as far as I know. C backend uses an extra last parameter. - Jay > Date: Sat, 29 Aug 2015 13:54:46 -0500 > From: rodney_bates at lcwb.coop > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > Subject: Re: [M3devel] Modula-3 llvm backend > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > Hi Rodney, > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > instead of the 3.5.0. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > from there, which is the case we really care about. Llvm changes themselves should > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > changed the debug info data structure design to make it not derived from Value. Not only > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > though systematic and simple. These are in separate M3 package llvmbindings. In the > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > than just what is currently used. > > I still am uncomfortable about how the complex C++ type hierarchies should be > reflected in the bindings. I doesn't look feasible to reflect these into true > M3 hierarcharies. But clients need some kind of conversions. Some functions > return results that need to be passed back in later, but to a parameter having > a static subtype or supertype thereof. The unwrap functions used in llvm appear > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > pointers, rather than runtime errors. This will inevitably make some bugs much > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > I am hoping future llvm releases will not affect this so much. They do change things > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > This all compiled and linked, as last I checked, but I have done very little execution > checking. These do require getting some llvm stuff in the right places and states. > I have had similar frustrations as Jay has with trying to use git branches, so > for now, I just made this a separate package. They generate alternative executables > with comparable function, so should be easy to switch between. > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > in a standard place and also, how much work will be done outside of the cm3 build > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > would be cleanest for users, but it is very big, takes hours to compile, and maybe > not everybody will want to use it badly enough to pay that price. Right now, you > have to do building and installing of llvm on the side, then edit a few paths in > m3makefiles. > > Keeping m3llvm as a completely separate executable means those who don't want to use > it can be unbothered. With its still being in major development, this seems best > for now. I originally set out on the path of having the functions of m3llvm linked > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > have to be built and linked in. > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > Those things will take some work, but should not be conceptually difficult. > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > Regards Peter > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > Peter, > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > the github repository. I have checked in some minor things and think I am about > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > One change is adding specialized support for calls on alloca. These are very > > recently being generated as library calls by the front end, and come through > > the whole process as link failures. I think it best to detect these and convert > > to llvm alloca instructions. m3cc does something like this now. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:45:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? Message-ID: The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:57:58 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:57:58 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: On the IA64 matter, it appears the bundle is the lower 5 bits.And the bundles 6, 7, 14, 15, 1A, 1B, 1E, 1F are invalid.It appears IA64 can be little or big endian, but the instructionsare always little endian.Therefore a 64bit value of -1 would seem invalid and ok for a marker,but 0 is less clear -- might still be invalid. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 30 19:26:21 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:26:21 -0500 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <55E33CBD.307@lcwb.coop> I too have long wished for a longer path in these messages. Although all source files in a package are required to have unique simple names, I doubt this is true between packages, except for visible interfaces. Even when unique, unless you have it all memorized, it is just an extra step to find a source file from only a simple name, when you are trying concentrate on the bug. "../src/" usually provides no information. I would be happy if the paths only went back as far as, say, cm3, for things in the repo, but how would we define where to stop for outside code? In a way, I think a full path as on the machine where the program was compiled might be more helpful anyway. Otherwise, that is really lost information, if you have any reason to care about it. A partial path that translates in an apparent way from the compiling to the executing machine could even be misleading, as the files could be quite different. OTOH, this can happen even when it's all the same machine, though probably less likely. On 08/29/2015 05:12 PM, Jay K wrote: > These messages: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/convert/Convert.m3", line 47 > *** > > > 1) It should really be a full path. > > I know people will disagree with me. > You want more commonality across machines. > I'm not sure that is worth it. > In particular, debuggers always work more easily with full paths, for local private builds. > Hopefully for debugging someone else's, some search path > with "prefix replacement" is viable. > But debugging your own build is more common and ideally > no special setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > > 2) We could/should trim the root of the git checkout, so it says > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > Some C compilers have that feature, when people are concered > with cross machine consistency for the sake of caching and similar > build optimizations. > > > I still prefer #1. > > > 3) Or at least m3front/src/convert/Convert.m3 > 3b) or ../../m3front/src/convert/Convert.m3 > > > if we really want it to be a correct relative path from the target directory. > > > A big part of the problem is the build system is geared toward packages, > not multiple packages. I'd like to solve this problem too somehow someday. > > > Really -- #1 -- source paths should be full paths. > > C compilers vary here btw, in terms of what __FILE__ is and what is > in debugging information. > Sometimes whatever is passed on the command line is used. > Sometimes full paths are computed by the compiler. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 19:52:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:52:59 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: <55E342FB.7060503@lcwb.coop> On 08/29/2015 05:20 PM, Jay K wrote: > I understand your dilemas. > Static linking vs. dynamic. > Cloning the source or not. > Significantly growing our repository or not. > Forking or not -- do we need any patches? Hopefully not but ok if we do. > > > Clearly the C backend cheats -- the C compiler appears on the system > as if by magic, or various external means. > > > I am somewhat keen on seeing this LLVM stuff work. > And somewhat interested in helping. > > > First, you know, there are many sayings and their redundancies and opposites: > > 1a too many cooks spoil the broth > 1b 9 women cannot have a baby in one month (but they can have 9 in 9) > > 2 more/many hands make less/little work > Which is it here, 1 or 2? > i.e. do you need/want help? > Help is surely needed/wanted, as long as we don't start too much tripping over each other. Just exactly where needs some thought. > I have built LLVM now a few times. > Yes it is large and takes time. > Do we have any required patches for it? I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain it is to retrofit patches to new versions. But that may be unrealistic. The last I checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be fixed in the llvm source tree. Maybe we can get them to do that for us. > I'm still on the fence as to if we should "import" it, vs. depend on its presence > "somehow". > > If there really is version sensitivity, that argues for importing. There is definitely version sensitivity. There were big changes in the data structure for debug info and the calls to construct it, somewhere between 3.5.0 and 3.6.1, require big rework of bindings (which they do not provide.) They do not hesitate to change interface things. There has been a roaring discussion about whether/how to start keeping the "Core" interface stable--something apparently has not be a goal so far. Core is a C interface they provide to certain things, notably not debug info. > While "apt-get install llvm" might be in widespread availability, > the exact "apt-get install llvm-3.5.0" I imagine isn't. > > > The current setup obviously needs work. > I'll likely commit something at least a little better. > > Do we really need 3.5.0, or can I use 3.5.2? > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything pretending to be a binding for. I prefer to at least start with 3.6.1, since it is later and significantly different, so will require just that much less retrofitting. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > > problems other people have had, guess I will have to trust upgrade.py. > > Really, I run this stuff all the time. > On various hosts. > What you/we/I do need though is a safe way to rebuild the compiler > when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. > In particular, if you have an ABI change, you need two different m3cores. > The one for the working compiler and the one for the new compiler. > Maybe I just need to use "buildlocal" more. > It doesn't help that the C backend isn't ABI compatible with cm3cg, due to > nested functions and static links. cm3cg uses a target-dependent extra > register not accessible to C as far as I know. C backend uses an extra > last parameter. > > - Jay > > > > > > Date: Sat, 29 Aug 2015 13:54:46 -0500 > > From: rodney_bates at lcwb.coop > > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > > Subject: Re: [M3devel] Modula-3 llvm backend > > > > > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > > Hi Rodney, > > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > > instead of the 3.5.0. > > > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > > from there, which is the case we really care about. Llvm changes themselves should > > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > > changed the debug info data structure design to make it not derived from Value. Not only > > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > > though systematic and simple. These are in separate M3 package llvmbindings. In the > > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > > than just what is currently used. > > > > I still am uncomfortable about how the complex C++ type hierarchies should be > > reflected in the bindings. I doesn't look feasible to reflect these into true > > M3 hierarcharies. But clients need some kind of conversions. Some functions > > return results that need to be passed back in later, but to a parameter having > > a static subtype or supertype thereof. The unwrap functions used in llvm appear > > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > > pointers, rather than runtime errors. This will inevitably make some bugs much > > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > > > I am hoping future llvm releases will not affect this so much. They do change things > > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > > This all compiled and linked, as last I checked, but I have done very little execution > > checking. These do require getting some llvm stuff in the right places and states. > > I have had similar frustrations as Jay has with trying to use git branches, so > > for now, I just made this a separate package. They generate alternative executables > > with comparable function, so should be easy to switch between. > > > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > > in a standard place and also, how much work will be done outside of the cm3 build > > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > > would be cleanest for users, but it is very big, takes hours to compile, and maybe > > not everybody will want to use it badly enough to pay that price. Right now, you > > have to do building and installing of llvm on the side, then edit a few paths in > > m3makefiles. > > > > Keeping m3llvm as a completely separate executable means those who don't want to use > > it can be unbothered. With its still being in major development, this seems best > > for now. I originally set out on the path of having the functions of m3llvm linked > > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > > have to be built and linked in. > > > > > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > > > > Those things will take some work, but should not be conceptually difficult. > > > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > > > Regards Peter > > > > > > > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > > > Peter, > > > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > > the github repository. I have checked in some minor things and think I am about > > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > > > One change is adding specialized support for calls on alloca. These are very > > > recently being generated as library calls by the front end, and come through > > > the whole process as link failures. I think it best to detect these and convert > > > to llvm alloca instructions. m3cc does something like this now. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 20:24:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 13:24:55 -0500 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: <55E34A77.1050001@lcwb.coop> On 08/30/2015 02:45 AM, Jay K wrote: > The agenda remains seeing if Target variables can be made constants. > > The discussion in this case is more complicated and some facts are unclear. > > > Background: > > > Nested functions are a problem. > In particular, if you can take their address. > Taking the address of a nested function presents a problem. > You are presented with two or three solutions. > > > - runtime code gen > - either on the stack > > - or somewhere more expensive, possibly with garbage collection > > - possibly "templated" instead of "arbitrary"; the meaning > of this is a lot to explain -- related to libffi and mremap, which > isn't entirely portable, but is e.g. portable to Windows > > > - or instead of runtime codegen, altering how function pointers are > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > The sequence is roughly: > Check if it is a "regular" function pointer or a "closure". > If it is a "regular" function pointer, just call it. > If it is a "closure", it contains a function pointer and a "static link". > Call the function pointer, passing the static link. > > > To tell if it is "regular" function pointer or a "closure", roughly > what is done is the data at the function pointer is read and compared > against a marker value. If it equals the marker value, it is a closure. > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > The value of the marker checked for is 0 or -1, I'd have to check. > The alignment of the pointer might be a factor. In particular, on most > architectures, all instructions have a certain alignment. If the pointer has > less alignment, it can't be an instruction. Or maybe on those architectures, > the bytes are read one at a time to avoid alignment faults. > > > In particular, as far as I know, the following: > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > 4 bytes and 4 byte aligned, so functions are at least also as much > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > pointer is actually odd as well, and the low bit is removed to really find the instructions > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > I could use confirmation on much of this. > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > value that works on all. > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > then checking for two in a row reduces the odds of a false positive. > > > However, given that the closure is a marker and two pointers, it isn't like you are going > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a valid instruction or sequence thereof. So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity here. Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for certain (especially if we want to be able to call procedures produced by other compilers) but much less likely to result in a problem. Just a wild guess, but I would not be surprised if ELF and other object formats would require the machine code of a function/procedure to begin on a native word boundary, even if the hardware instruction set does not. Where so, this would obviate checking the alignment before checking for a closure, though probably target-dependently. > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > However, I want less target-variation not more. > > > Here are some my lingering questions: > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > - Is a 64bit marker value actually sufficient on IA64? > The way to help here, I think, is to ensure that a 64bit marker, > not a 128bit marker, contains the "template", and an invalid "template". > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > i believe all of these function pointers are rare. > I hope/believe the object method calls do not check for closures -- though actually > that is related to a useful language construct, that I doubt we have. > Method body procedures are required to be top-level, ensured statically, so there is no need for method call code to consider the possibility of the pointer in the object type to be a closure. > > The simplest solution is likely: > - ignore IA64, or research it further > - keep marker size at integer Pointer would be target-independent in getting the following two pointers aligned. > - for the C backend, assume no alignment of function pointers -- give up > any of the optimization, esp. x86/amd64. I think this optimization both applies to a low-frequency situation and has a very small benefit, so I would not worry about giving up on it. > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > Thoughts? > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 30 22:45:18 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 30 Aug 2015 13:45:18 -0700 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E342FB.7060503@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> <55E342FB.7060503@lcwb.coop> Message-ID: <6B3E0C66-4A5A-44FF-9910-42DD01413214@gmail.com> I'm kind of impatient, sorry, that will be reflected here: - if you have a known working version, stick with it for now? - maintain multiple bindings? - patches: good & bad: it can be fun to fork, but also wasteful; maintaining diff/patch files may be an option. - Proceed without dwarf? If much else to do? You certain the problem is in LLVM? Your patches totally fix the problem. - Aomewhat interesting, m3x86 & m3cc for *some* platforms, e.g. Darwin, produce a minimum of debugging info, kinda surprising, & people don't seem to notice; I have largely fixed this with the C backend though & intend to fix it more. - Presumably I can/should expand beyond AMD_LINUX. I was going to use I386_DARWIN, but until I convince LLVM to be that, probably AMD64_DARWIN. - Jay On Aug 30, 2015, at 10:52 AM, "Rodney M. Bates" wrote: > > > On 08/29/2015 05:20 PM, Jay K wrote: >> I understand your dilemas. >> Static linking vs. dynamic. >> Cloning the source or not. >> Significantly growing our repository or not. >> Forking or not -- do we need any patches? Hopefully not but ok if we do. >> >> >> Clearly the C backend cheats -- the C compiler appears on the system >> as if by magic, or various external means. >> >> >> I am somewhat keen on seeing this LLVM stuff work. >> And somewhat interested in helping. >> >> >> First, you know, there are many sayings and their redundancies and opposites: >> >> 1a too many cooks spoil the broth >> 1b 9 women cannot have a baby in one month (but they can have 9 in 9) >> >> 2 more/many hands make less/little work >> Which is it here, 1 or 2? >> i.e. do you need/want help? > > Help is surely needed/wanted, as long as we don't start too much tripping > over each other. Just exactly where needs some thought. > >> I have built LLVM now a few times. >> Yes it is large and takes time. >> Do we have any required patches for it? > > I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain > it is to retrofit patches to new versions. But that may be unrealistic. The last I > checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be > fixed in the llvm source tree. Maybe we can get them to do that for us. > >> I'm still on the fence as to if we should "import" it, vs. depend on its presence >> "somehow". >> >> If there really is version sensitivity, that argues for importing. > > There is definitely version sensitivity. There were big changes in the > data structure for debug info and the calls to construct it, somewhere > between 3.5.0 and 3.6.1, require big rework of bindings (which they do > not provide.) They do not hesitate to change interface things. There > has been a roaring discussion about whether/how to start keeping the > "Core" interface stable--something apparently has not be a goal so far. > Core is a C interface they provide to certain things, notably not > debug info. > >> While "apt-get install llvm" might be in widespread availability, >> the exact "apt-get install llvm-3.5.0" I imagine isn't. >> >> >> The current setup obviously needs work. >> I'll likely commit something at least a little better. >> >> Do we really need 3.5.0, or can I use 3.5.2? > > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything > pretending to be a binding for. I prefer to at least start with 3.6.1, since it is > later and significantly different, so will require just that much less retrofitting. > >> >> > So I will be upgrading asap. I am always a bit wary of upgrading seeing the >> > problems other people have had, guess I will have to trust upgrade.py. >> >> Really, I run this stuff all the time. >> On various hosts. >> What you/we/I do need though is a safe way to rebuild the compiler >> when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. >> In particular, if you have an ABI change, you need two different m3cores. >> The one for the working compiler and the one for the new compiler. >> Maybe I just need to use "buildlocal" more. >> It doesn't help that the C backend isn't ABI compatible with cm3cg, due to >> nested functions and static links. cm3cg uses a target-dependent extra >> register not accessible to C as far as I know. C backend uses an extra >> last parameter. >> >> - Jay >> >> >> >> >> > Date: Sat, 29 Aug 2015 13:54:46 -0500 >> > From: rodney_bates at lcwb.coop >> > To: peter.mckinna at gmail.com; m3devel at elegosoft.com >> > Subject: Re: [M3devel] Modula-3 llvm backend >> > >> > >> > >> > On 08/29/2015 06:02 AM, Peter McKinna wrote: >> > > Hi Rodney, >> > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. >> > > >> > >> > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, >> > instead of the 3.5.0. >> > >> > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. >> > > >> > >> > upgrade.py worked OK for me after I went back to the release compiler and did it >> > from there, which is the case we really care about. Llvm changes themselves should >> > not affect the upgrade process, since no llvm mode is used in building the distribution. >> > >> > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. >> > > >> > >> > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm >> > changed the debug info data structure design to make it not derived from Value. Not only >> > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, >> > though systematic and simple. These are in separate M3 package llvmbindings. In the >> > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather >> > than just what is currently used. >> > >> > I still am uncomfortable about how the complex C++ type hierarchies should be >> > reflected in the bindings. I doesn't look feasible to reflect these into true >> > M3 hierarcharies. But clients need some kind of conversions. Some functions >> > return results that need to be passed back in later, but to a parameter having >> > a static subtype or supertype thereof. The unwrap functions used in llvm appear >> > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL >> > pointers, rather than runtime errors. This will inevitably make some bugs much >> > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe >> > some undetected type errors can propagate into C++ and give inscrutable segfaults. >> > >> > I am hoping future llvm releases will not affect this so much. They do change things >> > a lot. I also like to avoid chasing the new releases too often. Just as I was getting >> > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. >> > >> > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. >> > This all compiled and linked, as last I checked, but I have done very little execution >> > checking. These do require getting some llvm stuff in the right places and states. >> > I have had similar frustrations as Jay has with trying to use git branches, so >> > for now, I just made this a separate package. They generate alternative executables >> > with comparable function, so should be easy to switch between. >> > >> > Eventually, we will have to decide what to do about getting needed llvm stuff in, >> > in a standard place and also, how much work will be done outside of the cm3 build >> > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc >> > would be cleanest for users, but it is very big, takes hours to compile, and maybe >> > not everybody will want to use it badly enough to pay that price. Right now, you >> > have to do building and installing of llvm on the side, then edit a few paths in >> > m3makefiles. >> > >> > Keeping m3llvm as a completely separate executable means those who don't want to use >> > it can be unbothered. With its still being in major development, this seems best >> > for now. I originally set out on the path of having the functions of m3llvm linked >> > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that >> > have to be built and linked in. >> > >> > >> > >> > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. >> > > >> > >> > Those things will take some work, but should not be conceptually difficult. >> > >> > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. >> > > >> > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. >> > > >> > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. >> > > >> > > Regards Peter >> > > >> > >> > >> > >> > > >> > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: >> > > >> > > Peter, >> > > >> > > How actively are you working on the llvm back end? I don't see a lot of updates to >> > > the github repository. I have checked in some minor things and think I am about >> > > ready to do more. But I think it best we keep things merged as much as reasonable. >> > > >> > > One change is adding specialized support for calls on alloca. These are very >> > > recently being generated as library calls by the front end, and come through >> > > the whole process as link failures. I think it best to detect these and convert >> > > to llvm alloca instructions. m3cc does something like this now. >> > > >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> > > >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 08:26:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 06:26:32 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <55E33CBD.307@lcwb.coop> References: , <55E33CBD.307@lcwb.coop> Message-ID: It isn't perfectly scientific, as even full paths on a machinecan have variable meaning through time. For example, I can delete some source, put other source in its place,similar or different. And on Windows there is the "net use" feature where drive lettersmap to the network. Then you delete the use, and make another one,same letter, different place. Similar to mounting on Unixy systems. Here is another story in this area.Hopefully I have my facts correct.Perhaps it is just apocryphal and I filled in details to suit my agenda.We can double check it all with a bit of time. Visual C++, has, like any other system, some value for __FILE__and some values in the debugging information. A long time ago, they were both whatever you passed on the command line. At some point, a switch was added /FC that meant "full path the valuesin the debugging information". I believe this was added around Visual C++ 5.0.Later however, it was realized, that should always be how it works. The switchno longer does anything. Or maybe /FC still serves to full path __FILE__ but no longer has anyaffect on debugging information.I'd have to go back and try every compiler I have to verify. So, I think this actually motivate my other point from years ago,which is to full path the file names in the debug info, but not necessarily here.But I want both. :) I debug a lot of stuff, and it is unavoidably difficult, but some thingsjust make it unnecessarily more difficult and tedious. We should actually be recording crcs of source files and the assertcan go off and find the source and report if it matches.Or at least the assert can report the crc and user can verify it himself.i.e. once an assert fails, you can't do much reliably. And yes, when there is no CM3_ROOT or such, unclear what partof the path to truncate.Which motivates the other option -- don't truncate anything. I believe the names have to be unique "across the link".Visible or not, I'm not sure it matters.I don't believe we use the "static" feature of C/.obj files,so even non public functions, and the globals, are in a shared globalnamespace. And with dynamic linking, you can't be sure they don't surviveinto the dynamic link search space at least on Unixy systems.. - Jay > Date: Sun, 30 Aug 2015 12:26:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] paths too truncated in assertion failures > > I too have long wished for a longer path in these messages. Although > all source files in a package are required to have unique simple names, > I doubt this is true between packages, except for visible interfaces. > Even when unique, unless you have it all memorized, it is just an extra > step to find a source file from only a simple name, when you are trying > concentrate on the bug. "../src/" usually provides no information. > > I would be happy if the paths only went back as far as, say, cm3, for > things in the repo, but how would we define where to stop for outside > code? In a way, I think a full path as on the machine where the program > was compiled might be more helpful anyway. Otherwise, that is really lost > information, if you have any reason to care about it. A partial path > that translates in an apparent way from the compiling to the executing > machine could even be misleading, as the files could be quite different. > > OTOH, this can happen even when it's all the same machine, though probably > less likely. > > On 08/29/2015 05:12 PM, Jay K wrote: > > These messages: > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file "../src/convert/Convert.m3", line 47 > > *** > > > > > > 1) It should really be a full path. > > > > I know people will disagree with me. > > You want more commonality across machines. > > I'm not sure that is worth it. > > In particular, debuggers always work more easily with full paths, for local private builds. > > Hopefully for debugging someone else's, some search path > > with "prefix replacement" is viable. > > But debugging your own build is more common and ideally > > no special setting is needed to make that work. > > > > Yes, full paths could "leak" across machines but I think that is ok. > > > > I did work on this long ago but people disagreed with at the time. > > > > > > 2) We could/should trim the root of the git checkout, so it says > > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > > > > Some C compilers have that feature, when people are concered > > with cross machine consistency for the sake of caching and similar > > build optimizations. > > > > > > I still prefer #1. > > > > > > 3) Or at least m3front/src/convert/Convert.m3 > > 3b) or ../../m3front/src/convert/Convert.m3 > > > > > > if we really want it to be a correct relative path from the target directory. > > > > > > A big part of the problem is the build system is geared toward packages, > > not multiple packages. I'd like to solve this problem too somehow someday. > > > > > > Really -- #1 -- source paths should be full paths. > > > > C compilers vary here btw, in terms of what __FILE__ is and what is > > in debugging information. > > Sometimes whatever is passed on the command line is used. > > Sometimes full paths are computed by the compiler. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 15:31:29 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 15:31:29 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Mon Aug 31 16:26:59 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: > On Sat, 29 Aug 2015 22:12:37 +0000 > Jay K wrote: > > > These messages: > > ****** runtime error:*** <*ASSERT*> failed.*** file > "../src/convert/Convert.m3", line 47*** > > > > 1) It should really be a full path. > > I know people will disagree with me.You want more commonality across > machines.I'm not sure that is worth it.In particular, debuggers always work > more easily with full paths, for local private builds.Hopefully for > debugging someone else's, some search pathwith "prefix replacement" is > viable.But debugging your own build is more common and ideallyno special > setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > I'd vote for full absolute path. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 31 17:06:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 31 Aug 2015 10:06:12 -0500 Subject: [M3devel] Oops, belated commit of llvm3.6.1 Message-ID: <55E46D64.1020806@lcwb.coop> I apparently forgot, a while back, to commit new package llvm3.6.1, which I have been writing about. It is now in the Github repo. It compiles, but segfaults, most likely because of type problems in llvmbindings. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 20:02:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 18:02:07 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: , <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com>, Message-ID: And in compiler/warnings errors? i.e. do we need to preserve the "short" path in some cases? I vote for full path always. We can also provide options to trim specified prefixes, so builds at different paths get identical output. - Jay Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: Re: [M3devel] paths too truncated in assertion failures From: lists at darko.org To: wagner at elegosoft.com CC: jay.krell at cornell.edu; m3devel at elegosoft.com Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 20:08:44 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 20:08:44 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: <20150831200844.5b6c4c09e86238534fd1f659@elegosoft.com> On Mon, 31 Aug 2015 18:02:07 +0000 Jay K wrote: > And in compiler/warnings errors? > > i.e. do we need to preserve the "short" path in some cases? > > I vote for full path always. I second that. > > We can also provide options to trim specified prefixes, so > builds at different paths get identical output. > > - Jay -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Sun Aug 2 02:43:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 01 Aug 2015 19:43:43 -0500 Subject: [M3devel] better typing on SUBARRAY temporaries? In-Reply-To: References: <55B03594.20800@lcwb.coop> <55B25261.5010106@lcwb.coop> <55B3AA70.5050906@lcwb.coop> <55B3B8CF.1060208@lcwb.coop> Message-ID: <55BD67BF.4040606@lcwb.coop> On 07/26/2015 02:23 AM, Peter McKinna wrote: > So far not doing anything with free_temp, declare_temp just does an alloca but does it at the end of the first basic block to avoid dominate all uses problems. ( I have since discovered that there is a flag -alloca-hoisting which does this for you). The language ref hardly mentions temps except that they are referred to as unnamed values ie %1 %2 etc whether the optimiser does anything with these I dont know. I think I'm giving the temps names like %tmp.1 etc > The workaround I have at the moment is that if in the store, (which I think is the first place one of these temps is referenced) the var is a temp ie having NoID and is not declared in the current procedure then declare it ie alloca it. Seems to work but seems pretty kludgy. > My first thought when I struck this problem was that I should put all temps on the static link (since there is no parameter in declare_temp to say its uplevel and hence say which could be nested proc material) but that didnt work and really it was a pretty dumb idea. I cant see why the front end couldnt just declare a new temp in the finally proc. > Commit 496e9be1dcdcf87bda8e72239fc90132591b4cf4 fixes this, for this test case. > Regards Peter > > On Sun, Jul 26, 2015 at 2:26 AM, Rodney M. Bates > wrote: > > More on this: > > I appears that, in the CM3 IR, "variables" declared by declare_local and one > declared by declare_temp (and a few others, too), while declared with different > operators, are accessible interchangeably. In this example, the FINALLY procedure > accesses both s and the temporary nonlocally, in the same way. The difference > is, the temp, unlike a local variable, has the possibility that its space in > the activation record is freed prior to the end of the corresponding code. > > What llvm IR are you translating declare_temp and free_temp into? Llvm might > have different rules for temps. > > > > > On 07/25/2015 10:25 AM, Rodney M. Bates wrote: > > I compiled this and looked at the cm3 IR for it. At first glance, it looks like > a bug in the front end. The FINALLY code is translated as a nested procedure, > inside the one containing the entire TRY..FINALLY statement. The temp is created by > a declare_temp in the outer procedure, freed by a free_temp in the inner one, > and used in both, the 2 uses being unrelated. > > In m3-sys/m3middle/src/M3CG_Ops.i3, I see: > ----------------------------------------------------------------------------- > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > free_temp (v: Var); > (* releases the space occupied by temp 'v' so that it may be reused by > other new temporaries. *) > ----------------------------------------------------------------------------- > > And it also seems strange for the temp to be freed inside the nested procedure, > although this does not violate what the comment says. The fact that every > temp has a unique variable number would take care of matching the free_temp to the > right declare_temp, and maybe the code generators can handle freeing a nonlocal > temp. > > Apparently, this has caused no problems with preexisting code generators. > But it certainly looks wrong, and clearly violates the comments. > > I recall that the gcc-derived code generator and the integrated x86 code generator > both unnest nested procedures, in opposite ways (one pulls them out in front, the > other in back), which might have something to do with how they handle this. > > What happens in the llvm back end for a programmer-declared nested procedure > making a nonlocal reference to a programmer-declared local variable of the > containing procedure? If you can handle this latter case, can you handle the > failing one the same way? Maybe this is what is happening in the other code > generators, and the comment is just too strict. > > > > > On 07/24/2015 08:13 PM, Peter McKinna wrote: > > On the subject of temporaries can I get your thoughts on the following? > > TRY > s := s & "a"; > FINALLY > s := s & "b"; > END; > > The front end declares a temp in the try block as part of its concat and then refers to the same temp in the finally block. The trouble is that the finally code is generated in a separate procedure and references a temp declared in the proc of the try. In llvm the first you know of the problem is a store to the temp which has not been declared. > Just wondering whether the front end should redeclare this temp? Also is the front end generating similar temps for other runtime operations? > > Regards Peter > > On Sat, Jul 25, 2015 at 12:57 AM, Rodney M. Bates >> wrote: > > > > On 07/24/2015 03:57 AM, Jay K wrote: > > I model this in C like: > > > M3C.m3: > print(self, "/*declare_open_array*/typedef struct {"); > > > Presumably, this comment means you have some way of knowing, in M3C, that this is an open array? > > print(self, element_type.text); > print(self, "* _elts; CARDINAL _size"); > IF bit_size > Target.Integer.size * 2 THEN > print(self, "s["); > print(self, IntToDec((bit_size - Target.Integer.size) DIV Target.Integer.size)); > print(self, "]"); > END; > > > that is..and this i stinky that I get the struct "size", > > > size == 2 * sizeof(integer): > > > struct { > T* elements > size_t size; > } > > > else: > N = size - sizeof(INTEGER) / sizeof(INTEGER) > T ** elements; // where the number of star is N > > > I would not do pointer to pointer to pointer ... here. Just "T* elements", regardless > of the number of open dimensions. As I understand them, C's language-supported > multidimensional arrays are like Modula3 multidimensional _fixed_ arrays, i.e., > the language does the multi-subscript address arithmetic, which means the type > system needs to provide a static element size of each dimension. And that, > except for the innermost, depends on the element count of the next inner > dimension, which is not static here. > > So, the multiple dimensions are essentially flattened into one, and access to A[I,J,K] > is lowered by the front end into explicit address arithmetic into the flattened > array, using the values in the shape. Something like A[I*Shape[0]+J*Shape[1]+K] > > size_t sizes[N] > > > It is kind of lame that the frontend just gives the overall size > and the backend is just left to assume the layout like that. > > > If you know in M3C that it's an open array, you can infer what the layout is. > But yes, it's kind of lame. This is just another of the several places we have > seen that the front end has lowered things too far, and you have to > You know it's generated by code in OpenArray.m3, and you know the general dope layout, > and open dimension count, so you can generate an appropriate type. > > > > Really, the frontend should declare a type "pointer to pointer to pointer" with > the right "depth", and then a record with that pointer and a size or fixed size array of sizes. > > > Again, really ugly how it works now where backend is just given a size and can only > assume the layout. > > I don't know what a "dope vector" is. A value used as an initializer? > > > This is a very old and rather uninformative term for any block of stuff stored at runtime > that describes some source programmer's real data and how to access it. The only alternative > term I can think of would be "metadata", although that is rather overgeneral, and is usually > used with quite different specific meanings. But it is data describing data. > > > It is even worse for subarray. In this case we aren't even told it is an open array, just > some random struct with a size. That is what I first want to fix. It should declare an open array, > assuming they do have the same layout, which I think they do. > > > What you really need to know is that it's an open array, of which subarray is a subcategory. > In our implementation, all open array values have the same dope. E.g., look for the case where > a fixed array actual parameter is passed to an open array formal. > > > subarray temporaries and jmpbufs are I believe the only place the frontend passes so little > type information. > > > For jmpbufs I'm hoping to notice their name, and, unfortunately expensive, replace them > with #include and jmpbuf, instead of just a struct with an array of bytes. > > > > > - Jay > > > > Date: Wed, 22 Jul 2015 19:30:12 -0500 > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] better typing on SUBARRAY temporaries? > > > > I'm not exactly sure what you are asking, but here is some light on what > > you are seeing. These temporaries are exactly the dope the compiler uses > > to represent all open array values. First a pointer to the zeroth > > array element, then the "shape", as defined in M3 definition, 2.2.3, i.e. > > an array of element counts for each open subscript. For an open array > > parameter, this would be the machine representation of the parameter > > itself, authored in M3. (but passed by reference.) For a heap object, > > it is stored right before the elements themselves. For a SUBARRAY > > expression, it has to be a temporary. It also has to be constructed > > at the call site, as an anonymous temporary, when passing an entire fixed > > array to an open array parameter > > > > So, a good type for it might look like: > > > > > > RECORD > > Elements : REF ARRAY [0..Max, ... ,0..Max] OF ElementType > > ; Shape : ARRAY [0..OpenDepth-1] of CARDINAL > > END > > > > Max will be like the notorious TextLiteral.MaxBytes, i.e., we don't want any > > static limit here in the type of Elements, as it will be enforced dynamically, > > using Shape. But we don't want to just say REF ARRAY OF ElementType either, > > as this would mean another open array inside the dope, resulting in infinite > > recursion. > > > > On 07/22/2015 12:42 AM, Jay K wrote: > > > In the C backend I have a notion of "weak struct types" and "strong struct types". > > > > > > > > > "Strong" types have fields with types and names corresponding to the original Modula-3. i.e. they debug well. > > > > > > > > > "Weak" types have just arrays of characters (in a struct), sized/aligned to what the front end asked for. i.e. they debug poorly. > > > > > > > > > > > > Originally I had only weak types. > > > Ideally I have no weak types. > > > I'm down to very few weak types now. > > > I'd like to finish eliminating weak types. > > > > > > > > > > > > A quick investigation shows weak types come from open arrays and jmpbufs. > > > Open array temporaries from SUBARRAY specifically. > > > > > > > > > > > > Can we fix this? > > > > > > > > > > > > We have: > > > m3front/src/types/OpenArrayType.m3: > > > > > > PROCEDURE DeclareTemp (t: Type.T): CG.Var = > > > VAR > > > p := Reduce (t); > > > size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > RETURN CG.Declare_temp (size, Target.Address.align, > > > CG.Type.Struct, in_memory := TRUE); > > > END DeclareTemp; > > > > > > > > > PROCEDURE Compiler (p: P) = > > > VAR size := Target.Address.pack + OpenDepth (p) * Target.Integer.pack; > > > BEGIN > > > Type.Compile (p.element); > > > CG.Declare_open_array (Type.GlobalUID(p), Type.GlobalUID(p.element), size); > > > END Compiler; > > > > > > > > > DeclareTemp is used in SUBARRAY expressions -- truly temporaries, > > > not variables authored by anyone in Modula-3. > > > > > > > > > Can this be easily fixed? > > > > > > > > > Thanks, > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > Rodney Bates > rodney.m.bates at acm.org > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 2 05:36:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 03:36:05 +0000 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? Message-ID: I propose the following: - A flag to m3front indicating if the backend "knows" what a jmpbuf is. - extend M3CG.Type = { int8, uint8, ... addr ... struct } by adding a jmpbuf element - if the backend "knows" what a jmpbuf is, then, vs. the current code, the frontend shall just declare a jmpbuf per try, leaving the backend to size it, and w/o using the upcoming alloca approach. - if the backend does not "know" what a jmpbuf is, then, well, there is the current approach of making a sized struct, or my approach that is almost working where we use alloca. The frontend will not output cgtype.jmpbuf unless the backend "knows" what it is. ok? The information as to the backend's knowledge of a jmpbuf shall beyet another internal m3front flag, like what guides nested functionoutput order. I was going to propose something slightly different, likem3front always use this new jmpbuf type, and existing backendscan just change it "back" to struct. The problem with that approach..it works easily with the current code, but does not addressmy upcoming alloca(sizeof_jmpbuf) change. In this way, what the C backend will do is: #include jmp_buf buf1; jmp_buf buf2; etc. leaving the size of the jmpbuf completely absent in the generated code,and allocating in the slightly more efficient way than alloca. Exception handling is special, so accomodating this seems reasonable. Yes, it might be temporary, replaced by libunwind stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 2 10:44:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 2 Aug 2015 08:44:22 +0000 Subject: [M3devel] m3core.h broken In-Reply-To: <55BBDD0F.4080206@lcwb.coop> References: <55BBDD0F.4080206@lcwb.coop> Message-ID: Sorry, yes, my mistake.I should have used the UINT64/INT64 typedefs from nearby earlier.and my change wasn't well motivated anyway. - Jay > Date: Fri, 31 Jul 2015 15:39:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: [M3devel] m3core.h broken > > On AMD64_LINUX, with gcc 4.8.1, m3-libs/src/m3core.h suffers many compile errors, > when compiling on m3core and cm3. reverting line 363 of m3core.h makes them go away. > This is probably not the right way to fix this. > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 2 22:18:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 02 Aug 2015 15:18:58 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail>, Message-ID: <55BE7B32.7000502@lcwb.coop> On 08/02/2015 04:23 AM, Jay K wrote: > Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. > > so I have to move shared code to something called...ProcedureOrFinallyProcedure? > that otherwise was just in Procedure. I'm not sure what the breakage is, but for finally, the custom procedure generation consists only of: CG.Begin_procedure (p.handler.cg_proc); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc); in two places in TryFinStmt.m3. I would expect what you need to do to happen somewhere inside Stmt.Compile. Only there would it be known whether any TRYs are inside the FINALLY procedure. I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no doubt be compiler internally generated procedures. I would expect only those emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements inside. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3commit at elegosoft.com > Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > Date: Sun, 2 Aug 2015 02:20:51 +0000 > > Rodney, Do you have a test case that was broken before and fixed afterward? > > If possible, though not required, check something into m3-sys/m3tests? > > I don't mean to imply the code was correct before. I don't know, at all. > > Thank you, > - Jay > > > > Date: Sat, 1 Aug 2015 17:36:54 -0700 > From: rodney.m.bates at acm.org > To: m3commit at elegosoft.com > Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... > > Branch: refs/heads/master > Home:https://github.com/modula3/cm3 > Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 > https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 > Author: Rodney Bates > Date: 2015-08-01 (Sat, 01 Aug 2015) > > Changed paths: > M m3-sys/m3front/src/misc/CG.m3 > > Log Message: > ----------- > Fix compiler-generated FINALLY procedure using parent procedure's temporary. > > Changes to be committed: > > modified: m3-sys/m3front/src/misc/CG.m3 > > CG was not stacking global variables used for keeping track of temporaries > when entering nested procedures. In the case of a compiler-generated > nested procedure for the finally part of a TRY--FINALLY block, this > meant the finally code could use a temporary allocated by the parent, > though not in use at the time. This violated the comment in M3CG_Ops.i3: > > declare_temp (s: ByteSize; a: Alignment; t: Type; > in_memory: BOOLEAN): Var; > (* declares an anonymous local variable. Temps are declared > and freed between their containing procedure's begin_procedure and > end_procedure calls. Temps are never referenced by nested procedures. *) > > In cases of nested procedures explicitly declared in source code, this > didn't matter because they are located ahead of any of the parent > procedure's executable code, when it has no allocated temps, and they > free their own temps at their end, restoring the previous state. > > > > > _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > _______________________________________________ > M3commit mailing list > M3commit at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 3 09:05:17 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 00:05:17 -0700 Subject: [M3devel] [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... In-Reply-To: <55BE7B32.7000502@lcwb.coop> References: <55bd6626bc310_52df3f9e055cb2b8121d8@hookshot-fe2-cp1-prd.iad.github.net.mail> <55BE7B32.7000502@lcwb.coop> Message-ID: <13B01241-502D-415A-A98E-F818DD8C77CA@gmail.com> Thank you for the obvious reminder I needed -- look at all begin_procedure calls. I briefly mulled over that my changes belong in Stmt or elsewhere, but, the thing is..maybe my understanding isn't right, Stmt occurs in many places, you can say "begin" anywhere. My changes are very related to procedures -- counting TRY per procedure, calling alloca once per procedure, near the start -- before any branches or TRYs. The breakage is that I was considering finally procs to be part of the enclosing function. Very wrong. I put the alloca & setting a local pointer per TRY circa Procedure.GenBody, which isn't for the "synthetic" procedures.. I'll poke around more. Thanks, - Jay On Aug 2, 2015, at 1:18 PM, "Rodney M. Bates" wrote: > > > On 08/02/2015 04:23 AM, Jay K wrote: >> Eh, these finally procedures cause other problems..I have code to alloca a jmpbuf per try in a procedure..and it is broken by finally generating procedures in a custom way.. >> >> so I have to move shared code to something called...ProcedureOrFinallyProcedure? >> that otherwise was just in Procedure. > > I'm not sure what the breakage is, but for finally, the custom procedure generation consists > only of: > > CG.Begin_procedure (p.handler.cg_proc); > xc := Stmt.Compile (p.finally); > CG.Exit_proc (CG.Type.Void); > CG.End_procedure (p.handler.cg_proc); > > in two places in TryFinStmt.m3. I would expect what you need to do to happen > somewhere inside Stmt.Compile. Only there would it be known whether any TRYs > are inside the FINALLY procedure. > > I find Begin_procedure called in Procedure.m3, Exceptionz.m3, ObjectType.m3(2), > RefType.m3, TryFinStmt.m3(2). and Module.m3. All but Procedure.m3 would no > doubt be compiler internally generated procedures. I would expect only those > emitted by Procedure.m3, Module.m3, and TryFinStmt could have any TRY statements > inside. > > > >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3commit at elegosoft.com >> Subject: RE: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> Date: Sun, 2 Aug 2015 02:20:51 +0000 >> >> Rodney, Do you have a test case that was broken before and fixed afterward? >> >> If possible, though not required, check something into m3-sys/m3tests? >> >> I don't mean to imply the code was correct before. I don't know, at all. >> >> Thank you, >> - Jay >> >> >> >> Date: Sat, 1 Aug 2015 17:36:54 -0700 >> From: rodney.m.bates at acm.org >> To: m3commit at elegosoft.com >> Subject: [M3commit] [modula3/cm3] 496e9b: Fix compiler-generated FINALLY procedure using par... >> >> Branch: refs/heads/master >> Home:https://github.com/modula3/cm3 >> Commit: 496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> https://github.com/modula3/cm3/commit/496e9be1dcdcf87bda8e72239fc90132591b4cf4 >> Author: Rodney Bates >> Date: 2015-08-01 (Sat, 01 Aug 2015) >> >> Changed paths: >> M m3-sys/m3front/src/misc/CG.m3 >> >> Log Message: >> ----------- >> Fix compiler-generated FINALLY procedure using parent procedure's temporary. >> >> Changes to be committed: >> >> modified: m3-sys/m3front/src/misc/CG.m3 >> >> CG was not stacking global variables used for keeping track of temporaries >> when entering nested procedures. In the case of a compiler-generated >> nested procedure for the finally part of a TRY--FINALLY block, this >> meant the finally code could use a temporary allocated by the parent, >> though not in use at the time. This violated the comment in M3CG_Ops.i3: >> >> declare_temp (s: ByteSize; a: Alignment; t: Type; >> in_memory: BOOLEAN): Var; >> (* declares an anonymous local variable. Temps are declared >> and freed between their containing procedure's begin_procedure and >> end_procedure calls. Temps are never referenced by nested procedures. *) >> >> In cases of nested procedures explicitly declared in source code, this >> didn't matter because they are located ahead of any of the parent >> procedure's executable code, when it has no allocated temps, and they >> free their own temps at their end, restoring the previous state. >> >> >> >> >> _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit >> >> >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 04:35:10 2015 From: jay.krell at cornell.edu (Jay) Date: Mon, 3 Aug 2015 19:35:10 -0700 Subject: [M3devel] proposal for m3cgtype.jmpbuf etc.? In-Reply-To: References: Message-ID: <542D0438-1C2D-463A-96BC-E7CA6D0F2122@gmail.com> The alloca approach is fairly sound, w/o this change. However in the "cake and eat it too" dept..the C backend could instead say: #include setjmp.h . . . jmp_buf x; thereby still not knowing the size of a jmp_buf but generating correct portable slightly more efficient code. Also doing a better job of telling the C compiler that setjmp is being used -- rather than e.g. declaring setjmp ourselves, which I think is somewhat dubious, though faster to compile... I admit that the efficiency still stinks & we have much greater efficiency goals here. - Jay On Aug 3, 2015, at 7:05 PM, Antony Hosking wrote: > Remind me again what the issue is with the alloca approach? > >> On Aug 2, 2015, at 1:36 PM, Jay K wrote: >> >> I propose the following: >> >> >> - A flag to m3front indicating if the backend "knows" what a jmpbuf is. >> >> - extend M3CG.Type = { int8, uint8, ... addr ... struct } >> by adding a jmpbuf element >> >> >> - if the backend "knows" what a jmpbuf is, then, vs. the current code, >> the frontend shall just declare a jmpbuf per try, leaving the backend >> to size it, and w/o using the upcoming alloca approach. >> >> >> - if the backend does not "know" what a jmpbuf is, then, well, there >> is the current approach of making a sized struct, or my approach >> that is almost working where we use alloca. The frontend >> will not output cgtype.jmpbuf unless the backend "knows" what it is. >> >> >> ok? >> >> >> The information as to the backend's knowledge of a jmpbuf shall be >> yet another internal m3front flag, like what guides nested function >> output order. >> >> >> I was going to propose something slightly different, like >> m3front always use this new jmpbuf type, and existing backends >> can just change it "back" to struct. The problem with that approach.. >> it works easily with the current code, but does not address >> my upcoming alloca(sizeof_jmpbuf) change. >> >> >> In this way, what the C backend will do is: >> #include >> jmp_buf buf1; >> jmp_buf buf2; >> etc. >> >> >> leaving the size of the jmpbuf completely absent in the generated code, >> and allocating in the slightly more efficient way than alloca. >> >> >> Exception handling is special, so accomodating this seems reasonable. >> >> >> Yes, it might be temporary, replaced by libunwind stuff. >> >> >> Thank you, >> - Jay >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 09:36:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 07:36:21 +0000 Subject: [M3devel] Modula-2 parsing Message-ID: Procedure.m3: Do we need this?I know it isn't much, but remove it? ELSIF (cur.token = TK.tSEMI) THEN t.body := NEW (Body, self := t); ProcBody.Push (t.body); (* try accepting the Modula-2 syntax *) Error.ID (id, "expecting \'=\' before procedure body"); GetToken (); (* ; *) t.syms := Scope.PushNew (TRUE, id); t.block := BlockStmt.Parse (FALSE); t.fails := BlockStmt.ExtractFails (t.block); t.end_origin := Scanner.offset; final_id := MatchID (); IF (final_id # id) THEN Error.ID (id, "Initial name doesn\'t match final name"); END; Scope.PopNew (); ProcBody.Pop (); ELSE It always errors? But I guess it errors and checks nicer than it might otherwise? Why am I looking here? Tangential: I'm looking for where/how to model counting TRYs per "procedure"and doing the alloca at the start of a "procedure". "procedure"'s definition for my context is being worked on. It appears to be three things: 1) Things called "PROCEDURE". 2) FINALLY blocks, sometimes 3) "Module main" I was looking for what they have in common already, and it looks like maybe "ProcBody.Push". So I was looking all of them. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 4 12:02:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 4 Aug 2015 10:02:03 +0000 Subject: [M3devel] declaring/initializing variables "anywhere" Message-ID: I had: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count;BEGIN which is fairly nice and elegant.I like the static type inference. but p could be NIL so I need like: PROCEDURE CompileProcAllocateJmpbufs (p: Proc) =VAR module: Module.T; alloca: CG.Proc; size: CG.Var; try_count: INTEGER;BEGIN IF p = NIL THEN RETURN END; module := Module.Current (); alloca := Module.GetAlloca (module); size := Module.GetJmpbufSize (module); try_count := p.try_count; double the lines, and a need to state types that I didn't have to state before, OR I can use WITH to get the "best" of both, but implied extra indentation. Pretty much all other mainstream languages in use today,except C89, are "better" than this, they all let youboth delay local variable declaration until arbitrarilyinto a function, but with implying that you should indent further,or close an extra block. Most mainstream languages also share Modula-3's type inferenceand let you omit the type, except C. Can we update Modula-3 here somehow? Or just use "with"? Proposals for a syntax that works well with the existing language and compiler? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 4 18:42:23 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 11:42:23 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: References: Message-ID: <55C0EB6F.7050704@lcwb.coop> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually involves finding one's way around in hundreds of thousands of lines of unfamiliar code. If you're exceptionally lucky, an eventual one-line change can be figured out after vetting a mere several tens of lines. Very often, it is more like a few hundreds, and not infrequently, even more. The average line of actively maintained code is written once and read something like seven times. (I don't remember where that number came from.) Speaking for myself, even most brand new code gets read at least seven times before it gets past my own testing, before it gets passed to anybody else to either use or further test. Moreover, the initial writer already has lots of context in his head that a maintainer has to ferret out. What I am leading to is that things should be *locally explicit*, far more than we usually see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches who have to come along and figure it out later to constantly locate, open, scan another source file, every fourth token they read, is penny wise and pound foolish. It's classic case of instant-gratification immaturity. Ensuring a ton of aggravation later, to save an ounce of effort now. It's just plain laziness. (more below:) On 08/04/2015 05:02 AM, Jay K wrote: > I had: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); So, coded like this, the poor schmuck who has to come along and figure this out will have to find Module.i3 and search it for declarations of Current, GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly more steps, as with alloca, whose type, as we will find out, requires going to yet another source file CG.i3. It can be even worse, e.g.: VAR V := Ptr^.Field[I].method(X); You have to track down the type of Ptr, to find Field, to find the type of its elements, to find method, to find its result type. Sometimes it's obvious from identifier names, but more often, it is not, or at least only too vaguely for serious software work. Is a module denoted by an abstract type? An integer numbering of modules? something else? Very often, there are two or three different types that denote a module or whatever. (An aside: In C++, things often get dramatically worse when coders, overly enamored with cleverness for its own sake, define implicit type conversions among these, so the poor maintainer has even less idea what is going on. Moreover, the conversions could be declared anywhere in the tens of transitively #included files, and there is no path leading to them, only exhaustive search through all the files. And this possibility has to be considered for every single assignment and actual parameter.) Instead, I always code cases like this by putting both the type and the initializer in the declaration: VAR module : Module.T := Module.Current (); I even often do it this way when the initializer is a literal. For one thing, I may want a subrange for the type of the variable. The exception is when the initializer itself also names the type. It's shorter, but at no loss of explicitness. VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > try_count := p.try_count; > BEGIN > > > which is fairly nice and elegant. > I like the static type inference. > > > but p could be NIL so I need like: > > > PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = > VAR module: Module.T; > alloca: CG.Proc; > size: CG.Var; > try_count: INTEGER; > BEGIN > IF p = NIL THEN RETURN END; > module := Module.Current (); > alloca := Module.GetAlloca (module); > size := Module.GetJmpbufSize (module); > try_count := p.try_count; > > > double the lines, and a need to state types that I didn't have to state before, > This will save wasted computation calling the functions, if it turns out p=NIL. Depending on the purpose of the code. In some code, I care about efficiency at this level. > > OR I can use WITH to get the "best" of both, but implied extra indentation. > If I were to propose any language change at all, it would be to allow a WITH binding to optionally specify the type too, for exactly the same reasons. WITH w : T = Var.method(x) DO ... Many a WITH statement has been far too hard to read, requiring the side trips of checking other source files to see what this value really is. > > > Pretty much all other mainstream languages in use today, > except C89, are "better" than this, they all let you > both delay local variable declaration until arbitrarily > into a function, but with implying that you should indent further, > or close an extra block. > To be clear, I definitely do not advocate allowing declarations and statements to be arbitrarily mixed in a block, BUT... if we did it, at least the decls would be syntactically explicit with a VAR, and we would not have to "define" our language with words like "if it looks like a declaration...". > > Most mainstream languages also share Modula-3's type inference > and let you omit the type, except C. > > > Can we update Modula-3 here somehow? > > > Or just use "with"? > > > Proposals for a syntax that works well with the existing language and compiler? > > > Thank you, > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 20:07:54 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 11:07:54 -0700 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <55C0EB6F.7050704@lcwb.coop> References: <55C0EB6F.7050704@lcwb.coop> Message-ID: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> But...Modula-3 already has this in VAR and WITH. Every language has this for temporaries: F1(F2() + F3()) Are these bad? Too much explicit detail can actually get in the way of reading the code. People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. - Jay On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually > involves finding one's way around in hundreds of thousands of lines of unfamiliar code. > If you're exceptionally lucky, an eventual one-line change can be figured out after vetting > a mere several tens of lines. Very often, it is more like a few hundreds, and not > infrequently, even more. > > The average line of actively maintained code is written once and read something like > seven times. (I don't remember where that number came from.) Speaking for myself, > even most brand new code gets read at least seven times before it gets past my own testing, > before it gets passed to anybody else to either use or further test. Moreover, the initial > writer already has lots of context in his head that a maintainer has to ferret out. > > What I am leading to is that things should be *locally explicit*, far more than we usually > see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches > who have to come along and figure it out later to constantly locate, open, scan another > source file, every fourth token they read, is penny wise and pound foolish. It's classic > case of instant-gratification immaturity. Ensuring a ton of aggravation later, to > save an ounce of effort now. It's just plain laziness. > > (more below:) > > On 08/04/2015 05:02 AM, Jay K wrote: >> I had: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); > > So, coded like this, the poor schmuck who has to come along and figure this > out will have to find Module.i3 and search it for declarations of Current, > GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly > more steps, as with alloca, whose type, as we will find out, requires > going to yet another source file CG.i3. It can be even worse, e.g.: > > VAR V := Ptr^.Field[I].method(X); > > You have to track down the type of Ptr, to find Field, to find the type of > its elements, to find method, to find its result type. > > Sometimes it's obvious from identifier names, but more often, it is not, or at > least only too vaguely for serious software work. Is a module denoted by an > abstract type? An integer numbering of modules? something else? Very often, > there are two or three different types that denote a module or whatever. > > (An aside: In C++, things often get dramatically worse when coders, overly > enamored with cleverness for its own sake, define implicit type conversions > among these, so the poor maintainer has even less idea what is going on. > Moreover, the conversions could be declared anywhere in the tens of transitively > #included files, and there is no path leading to them, only exhaustive search > through all the files. And this possibility has to be considered for every > single assignment and actual parameter.) > > Instead, I always code cases like this by putting both the type and the initializer > in the declaration: > VAR module : Module.T := Module.Current (); > > I even often do it this way when the initializer is a literal. For one thing, > I may want a subrange for the type of the variable. > > The exception is when the initializer itself also names the type. It's shorter, > but at no loss of explicitness. > > VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; > >> try_count := p.try_count; >> BEGIN >> >> >> which is fairly nice and elegant. >> I like the static type inference. >> >> >> but p could be NIL so I need like: >> >> >> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >> VAR module: Module.T; >> alloca: CG.Proc; >> size: CG.Var; >> try_count: INTEGER; >> BEGIN >> IF p = NIL THEN RETURN END; >> module := Module.Current (); >> alloca := Module.GetAlloca (module); >> size := Module.GetJmpbufSize (module); >> try_count := p.try_count; >> >> >> double the lines, and a need to state types that I didn't have to state before, > > This will save wasted computation calling the functions, if it turns out p=NIL. > Depending on the purpose of the code. In some code, I care about efficiency > at this level. > >> >> OR I can use WITH to get the "best" of both, but implied extra indentation. > > If I were to propose any language change at all, it would be to allow a WITH > binding to optionally specify the type too, for exactly the same reasons. > > WITH w : T = Var.method(x) DO ... > > Many a WITH statement has been far too hard to read, requiring the side trips of > checking other source files to see what this value really is. > > > >> >> >> Pretty much all other mainstream languages in use today, >> except C89, are "better" than this, they all let you >> both delay local variable declaration until arbitrarily >> into a function, but with implying that you should indent further, >> or close an extra block. > > To be clear, I definitely do not advocate allowing declarations and statements > to be arbitrarily mixed in a block, BUT... if we did it, at least the decls > would be syntactically explicit with a VAR, and we would not have to "define" > our language with words like "if it looks like a declaration...". > >> >> Most mainstream languages also share Modula-3's type inference >> and let you omit the type, except C. >> >> >> Can we update Modula-3 here somehow? >> >> >> Or just use "with"? >> >> >> Proposals for a syntax that works well with the existing language and compiler? >> >> >> Thank you, >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:18:15 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:18:15 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C101E7.60106@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 20:30:47 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 13:30:47 -0500 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: Message-ID: <55C104D7.4040601@lcwb.coop> On 08/04/2015 02:36 AM, Jay K wrote: > Procedure.m3: > > Do we need this? > I know it isn't much, but remove it? > It looks to me like the comment "accepting" is misleading, since it produces an error message. My guess is that this is a specialized syntax error recovery technique, designed to do a better job of continuing when this particular error occurs, probably based on experience. Modula-3 evolved from Modula-2+, which evolved from Modula-2, and I'll bet there was a lot of code converted from one language to another. This would have been a common leftover from that process. I'd change the comment, but otherwise leave it. > > ELSIF (cur.token = TK.tSEMI) THEN > t.body := NEW (Body, self := t); > ProcBody.Push (t.body); > (* try accepting the Modula-2 syntax *) > Error.ID (id, "expecting \'=\' before procedure body"); > GetToken (); (* ; *) > t.syms := Scope.PushNew (TRUE, id); > t.block := BlockStmt.Parse (FALSE); > t.fails := BlockStmt.ExtractFails (t.block); > t.end_origin := Scanner.offset; > final_id := MatchID (); > IF (final_id # id) THEN > Error.ID (id, "Initial name doesn\'t match final name"); > END; > Scope.PopNew (); > ProcBody.Pop (); > ELSE > > > It always errors? > > But I guess it errors and checks nicer than it might otherwise? > > Why am I looking here? > > Tangential: > > I'm looking for where/how to model counting TRYs per "procedure" > and doing the alloca at the start of a "procedure". > Are you counting TRYs in the front end, or on the back side of the cm3 IR? If the former, try looking at the way "fails" is collected from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, to a field of Procedure.T. But it has to come out of TryStmt.Parse, to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not a list? If the latter, aren't you already doing two passes over the IR? > > "procedure"'s definition for my context is being worked on. > It appears to be three things: > 1) Things called "PROCEDURE". > 2) FINALLY blocks, sometimes > 3) "Module main" > > I was looking for what they have in common already, > and it looks like maybe "ProcBody.Push". > > So I was looking all of them. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Aug 4 21:02:53 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 04 Aug 2015 14:02:53 -0500 Subject: [M3devel] declaring/initializing variables "anywhere" In-Reply-To: <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> References: <55C0EB6F.7050704@lcwb.coop> <847137D1-3D18-4AD8-9223-BCEC3954F5CB@gmail.com> Message-ID: <55C10C5D.40402@lcwb.coop> On 08/04/2015 01:07 PM, Jay wrote: > But...Modula-3 already has this in VAR and WITH. Huh? WithSt = WITH Binding {"," Binding} DO S END. Binding = Id "=" Expr. ^ no explicit type allowed here. > > Every language has this for temporaries: > > F1(F2() + F3()) > > Are these bad? > > Too much explicit detail can actually get in the way of reading the code. Yes, it is a balance. But much of the world is waaaay to far on the implicit side. Explicitly typing proper subexpressions would be far too tedious. But that is exactly why the rules for inferring should not be highly complex, multi-pronged, or depend on a huge amount of context. > > > People also claim "just hover over the variable in IDE or debugger" which I don't buy. The tools always lag the language -- even if some don't, old tools remain in use. > Yes, if such tools were ubiquitous and dependable, it would go a long way to help. > > "instant gratification" is also spun as "productivity" and "automatic maintenance" -- don't need to adjust redundant type declarations when things change, assuming compiler gets it right. Some people trust the deduction more than human, some don't. > Yes, there is a tension in making things self-adapt to one-place changes. But if it's as complicated as is often the case, the time saved editing will be overwhelmed by the time figuring out what to change. And trusting a language/compiler to make the deduction that works right while the human can't understand it, or just doesn't have the time and gumption, is insane. > > - Jay > > On Aug 4, 2015, at 9:42 AM, "Rodney M. Bates" wrote: > >> You're poking at a really big peeve of mine. I do a lot of maintenance work. It usually >> involves finding one's way around in hundreds of thousands of lines of unfamiliar code. >> If you're exceptionally lucky, an eventual one-line change can be figured out after vetting >> a mere several tens of lines. Very often, it is more like a few hundreds, and not >> infrequently, even more. >> >> The average line of actively maintained code is written once and read something like >> seven times. (I don't remember where that number came from.) Speaking for myself, >> even most brand new code gets read at least seven times before it gets past my own testing, >> before it gets passed to anybody else to either use or further test. Moreover, the initial >> writer already has lots of context in his head that a maintainer has to ferret out. >> >> What I am leading to is that things should be *locally explicit*, far more than we usually >> see. Saving keystrokes once at initial writing time, at the cost of forcing the poor wretches >> who have to come along and figure it out later to constantly locate, open, scan another >> source file, every fourth token they read, is penny wise and pound foolish. It's classic >> case of instant-gratification immaturity. Ensuring a ton of aggravation later, to >> save an ounce of effort now. It's just plain laziness. >> >> (more below:) >> >> On 08/04/2015 05:02 AM, Jay K wrote: >>> I had: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >> >> So, coded like this, the poor schmuck who has to come along and figure this >> out will have to find Module.i3 and search it for declarations of Current, >> GetAlloca, and GetJmpbufSize, just to know what types we have. And possibly >> more steps, as with alloca, whose type, as we will find out, requires >> going to yet another source file CG.i3. It can be even worse, e.g.: >> >> VAR V := Ptr^.Field[I].method(X); >> >> You have to track down the type of Ptr, to find Field, to find the type of >> its elements, to find method, to find its result type. >> >> Sometimes it's obvious from identifier names, but more often, it is not, or at >> least only too vaguely for serious software work. Is a module denoted by an >> abstract type? An integer numbering of modules? something else? Very often, >> there are two or three different types that denote a module or whatever. >> >> (An aside: In C++, things often get dramatically worse when coders, overly >> enamored with cleverness for its own sake, define implicit type conversions >> among these, so the poor maintainer has even less idea what is going on. >> Moreover, the conversions could be declared anywhere in the tens of transitively >> #included files, and there is no path leading to them, only exhaustive search >> through all the files. And this possibility has to be considered for every >> single assignment and actual parameter.) >> >> Instead, I always code cases like this by putting both the type and the initializer >> in the declaration: >> VAR module : Module.T := Module.Current (); >> >> I even often do it this way when the initializer is a literal. For one thing, >> I may want a subrange for the type of the variable. >> >> The exception is when the initializer itself also names the type. It's shorter, >> but at no loss of explicitness. >> >> VAR rec := SomeInterface.SomeType { 0 , NIL , FALSE }; >> >>> try_count := p.try_count; >>> BEGIN >>> >>> >>> which is fairly nice and elegant. >>> I like the static type inference. >>> >>> >>> but p could be NIL so I need like: >>> >>> >>> PROCEDURE CompileProcAllocateJmpbufs (p: Proc) = >>> VAR module: Module.T; >>> alloca: CG.Proc; >>> size: CG.Var; >>> try_count: INTEGER; >>> BEGIN >>> IF p = NIL THEN RETURN END; >>> module := Module.Current (); >>> alloca := Module.GetAlloca (module); >>> size := Module.GetJmpbufSize (module); >>> try_count := p.try_count; >>> >>> >>> double the lines, and a need to state types that I didn't have to state before, >> >> This will save wasted computation calling the functions, if it turns out p=NIL. >> Depending on the purpose of the code. In some code, I care about efficiency >> at this level. >> >>> >>> OR I can use WITH to get the "best" of both, but implied extra indentation. >> >> If I were to propose any language change at all, it would be to allow a WITH >> binding to optionally specify the type too, for exactly the same reasons. >> >> WITH w : T = Var.method(x) DO ... >> >> Many a WITH statement has been far too hard to read, requiring the side trips of >> checking other source files to see what this value really is. >> >> >> >>> >>> >>> Pretty much all other mainstream languages in use today, >>> except C89, are "better" than this, they all let you >>> both delay local variable declaration until arbitrarily >>> into a function, but with implying that you should indent further, >>> or close an extra block. >> >> To be clear, I definitely do not advocate allowing declarations and statements >> to be arbitrarily mixed in a block, BUT... if we did it, at least the decls >> would be syntactically explicit with a VAR, and we would not have to "define" >> our language with words like "if it looks like a declaration...". >> >>> >>> Most mainstream languages also share Modula-3's type inference >>> and let you omit the type, except C. >>> >>> >>> Can we update Modula-3 here somehow? >>> >>> >>> Or just use "with"? >>> >>> >>> Proposals for a syntax that works well with the existing language and compiler? >>> >>> >>> Thank you, >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:47:16 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:47:16 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: <55C104D7.4040601@lcwb.coop> References: <55C104D7.4040601@lcwb.coop> Message-ID: Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. This change is for all backends. No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. If a backend wants to know about jmp_buf, that ability will be added soon thereafter. I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? There are obvious counter points but most seem moot these days. Yes, it means a larger working set and more likely-traced allocations. Ignoring the tracing though, this is how cm3cg has always worked. And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. At least one of the transforms M3C does could be moved to m3front -- removing unused locals. And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. Given the M3C code, this is easy at this point. Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. I'll send diffs later. I think I'm quite close. Thoughts? - Jay On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > On 08/04/2015 02:36 AM, Jay K wrote: >> Procedure.m3: >> >> Do we need this? >> I know it isn't much, but remove it? > > It looks to me like the comment "accepting" is misleading, since it produces > an error message. My guess is that this is a specialized syntax error recovery > technique, designed to do a better job of continuing when this particular > error occurs, probably based on experience. Modula-3 evolved from Modula-2+, > which evolved from Modula-2, and I'll bet there was a lot of code converted > from one language to another. This would have been a common leftover from > that process. > > I'd change the comment, but otherwise leave it. > >> >> ELSIF (cur.token = TK.tSEMI) THEN >> t.body := NEW (Body, self := t); >> ProcBody.Push (t.body); >> (* try accepting the Modula-2 syntax *) >> Error.ID (id, "expecting \'=\' before procedure body"); >> GetToken (); (* ; *) >> t.syms := Scope.PushNew (TRUE, id); >> t.block := BlockStmt.Parse (FALSE); >> t.fails := BlockStmt.ExtractFails (t.block); >> t.end_origin := Scanner.offset; >> final_id := MatchID (); >> IF (final_id # id) THEN >> Error.ID (id, "Initial name doesn\'t match final name"); >> END; >> Scope.PopNew (); >> ProcBody.Pop (); >> ELSE >> >> >> It always errors? >> >> But I guess it errors and checks nicer than it might otherwise? >> >> Why am I looking here? >> >> Tangential: >> >> I'm looking for where/how to model counting TRYs per "procedure" >> and doing the alloca at the start of a "procedure". > > Are you counting TRYs in the front end, or on the back side of the > cm3 IR? If the former, try looking at the way "fails" is collected > from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, > to a field of Procedure.T. But it has to come out of TryStmt.Parse, > to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not > a list? > > If the latter, aren't you already doing two passes over the IR? > >> >> "procedure"'s definition for my context is being worked on. >> It appears to be three things: >> 1) Things called "PROCEDURE". >> 2) FINALLY blocks, sometimes >> 3) "Module main" >> >> I was looking for what they have in common already, >> and it looks like maybe "ProcBody.Push". >> >> So I was looking all of them. >> >> - Jay >> >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 4 21:56:44 2015 From: jay.krell at cornell.edu (Jay) Date: Tue, 4 Aug 2015 12:56:44 -0700 Subject: [M3devel] Modula-2 parsing In-Reply-To: References: <55C104D7.4040601@lcwb.coop> Message-ID: Ps: yes just a count. A list could be useful but I achieved similar end another way. Each try records the current count as its index before incrementing it. Then we allocate an array of that many vars, which are initialized (perhaps too preemptively) to point into the bulk alloca. Then each TRY uses the corresponding variable. With a list I could poke each var into each Try*.T -- introducing a common base. The difference is negligible -- in m3front an extra array per procedure and extra indirection when compiling TRY, vs. walking the list. The generated code would be the same either way. The "spreading out of the division of labor" also similar. I introduced module Jmpbufs in m3front to hold most of this. I consider Marker, even existing functions in it, but it didn't seem to fit well. - Jay On Aug 4, 2015, at 12:47 PM, Jay wrote: > Agreed the recovery might be worthwhile. Maybe the code can be refactored -- I'll look later. > > > "Counting TRY" is in m3front. No full IR -- though I agree that isn't a bad idea.. > > > This change is for all backends. > > No front end or backend will need to know the size of a jmp_buf and a chunk of target-specific code will be gone. > > If a backend wants to know about jmp_buf, that ability will be added soon thereafter. > > > I have it "almost working" and might send diffs soon. Specifically, "upgrade", which does raise exceptions, e.g. file not found, works. Compilation fails though in jvideo. I have to debug that and write more tests, esp. for TRY in finally and module main -- the cases of "procedures" I had originally missed (and still wish were modeled using Procedure.T). > > > So..indeed...what about this idea: form up the entire IR in memory in m3front, allow some passes over it, and then onto backend? > > > There are obvious counter points but most seem moot these days. > > > Yes, it means a larger working set and more likely-traced allocations. > > > Ignoring the tracing though, this is how cm3cg has always worked. > > > And much of the world has moved past even this -- attempting "whole program compilation" -- with imho mixed results -- the resulting compilation speed and scaling and memory use and loss of inceementality have been problems. But the results are also good. My favorite example is cross-module inlining. > > > At least one of the transforms M3C does could be moved to m3front -- removing unused locals. > > > And maybe unused imports -- that is an extremely small problem though -- OpenBSD C compiler warns/errors for strcpy/strcat being declared. > > > Heck -- there is also an implied better in-memory IR that could be used -- i.e. the whole thing per module instead of just a function call at a time. > > > Given the M3C code, this is easy at this point. > > Isn't BlockStmt introducable almost anywhere? How do I know start of a procedure? Well, by looking for cg.begin_procedure. > > I'll send diffs later. I think I'm quite close. > > Thoughts? > > - Jay > > On Aug 4, 2015, at 11:30 AM, "Rodney M. Bates" wrote: > >> On 08/04/2015 02:36 AM, Jay K wrote: >>> Procedure.m3: >>> >>> Do we need this? >>> I know it isn't much, but remove it? >> >> It looks to me like the comment "accepting" is misleading, since it produces >> an error message. My guess is that this is a specialized syntax error recovery >> technique, designed to do a better job of continuing when this particular >> error occurs, probably based on experience. Modula-3 evolved from Modula-2+, >> which evolved from Modula-2, and I'll bet there was a lot of code converted >> from one language to another. This would have been a common leftover from >> that process. >> >> I'd change the comment, but otherwise leave it. >> >>> >>> ELSIF (cur.token = TK.tSEMI) THEN >>> t.body := NEW (Body, self := t); >>> ProcBody.Push (t.body); >>> (* try accepting the Modula-2 syntax *) >>> Error.ID (id, "expecting \'=\' before procedure body"); >>> GetToken (); (* ; *) >>> t.syms := Scope.PushNew (TRUE, id); >>> t.block := BlockStmt.Parse (FALSE); >>> t.fails := BlockStmt.ExtractFails (t.block); >>> t.end_origin := Scanner.offset; >>> final_id := MatchID (); >>> IF (final_id # id) THEN >>> Error.ID (id, "Initial name doesn\'t match final name"); >>> END; >>> Scope.PopNew (); >>> ProcBody.Pop (); >>> ELSE >>> >>> >>> It always errors? >>> >>> But I guess it errors and checks nicer than it might otherwise? >>> >>> Why am I looking here? >>> >>> Tangential: >>> >>> I'm looking for where/how to model counting TRYs per "procedure" >>> and doing the alloca at the start of a "procedure". >> >> Are you counting TRYs in the front end, or on the back side of the >> cm3 IR? If the former, try looking at the way "fails" is collected >> from Decl.Parse, to BlockStmt.ExtractFails, to Procedure.ParseDecl, >> to a field of Procedure.T. But it has to come out of TryStmt.Parse, >> to Stmt.Parse, to BlockStmt.Parse. And you only need a count, not >> a list? >> >> If the latter, aren't you already doing two passes over the IR? >> >>> >>> "procedure"'s definition for my context is being worked on. >>> It appears to be three things: >>> 1) Things called "PROCEDURE". >>> 2) FINALLY blocks, sometimes >>> 3) "Module main" >>> >>> I was looking for what they have in common already, >>> and it looks like maybe "ProcBody.Push". >>> >>> So I was looking all of them. >>> >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 5 09:17:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 07:17:42 +0000 Subject: [M3devel] names in (hopefully) upcoming changes Message-ID: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 11:15:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:15:22 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review Message-ID: Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 22.txt URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.i3 Type: application/octet-stream Size: 1138 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Jmpbufs.m3 Type: application/octet-stream Size: 5020 bytes Desc: not available URL: From jay.krell at cornell.edu Wed Aug 5 11:49:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 09:49:15 +0000 Subject: [M3devel] setjmp/jmpbuf declaration vs. #include setjmp.h Message-ID: fyi...having updated tools, I now get: cm3.d/Main.m3.c:757:1: warning: declaration of built-in function '_setjmp' requires inclusion of the header [-Wbuiltin-requires-header] which I think helps vindicate the upcoming addition of CGType.Jmpbuf. That is, we shouldn't declare setjmp ourselves.And therefore, we also can't pretend jmpbuf is just a struct with a size, but must use whatever setjmp.h says.I hope to fix this after the first round of "alloca(jmpbuf)". Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Aug 5 17:54:48 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 5 Aug 2015 15:54:48 +0000 (UTC) Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: Message-ID: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Hi all:long time ago, a proposal for supporting more higher level R like Aimer Diwan et al, would benefit, support C -> Fortran compilation to further parallelize in big machines (Virtual Speculative Parallel Machine), see [1]:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear Anyway, hearing of what you are trying to do especially support OPenVMS, etc, is nice Thanks in advance [1]R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of Language Transcription?, en Languages and Compilers for Parallel Computing, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. El Mi?rcoles 5 de agosto de 2015 2:17, Jay K escribi?: I need some names. C has a type "jmp_buf" -- with no "u" and with an underscore. I need a module in m3front for counting tries and managing jmp_bufs.I call it Jmpbufs.It could be Jumpbufs.Or JumpBuffers.Perhaps that is too direct.Or sjljeh (setjmp/longjmp exception handling)Or worked into the existing Marker. A "constant variable" in C to hold sizeof(jmp_buf).So far this was called?Csetjmp__Jumpbuf_size.However this "constant variable" should never be used from Modula-3.Putting in a Modula-3 interface (Csetjmp), with the two level naming and a double underscore is unnecessary and I suggest should not be done. This name is an "interface" between m3front and m3core, i.e. m3core/src/unix/Common/Uconstants.c.It is always statically linked. Potential names here are: m3_jmpbuf_size my favorate at the momentjmpbuf_sizeCsetjmp_Jumpbuf_size already present by this name, but can be changed? "alloca" This is (will be) part of the interface between m3front and every backend.While it is *almost* pass-through, it isn't.The C backend targeting non-Win32, non-VMS, can just pass through the name.But every other case in every other backend must treat the name specially.cm3cg has to convert it to "builtin_alloca".LLVM probably is like cm3cg.Win32 non-C has to change it to __chkstk or perhaps _chkstk.Win32 C has to change it to _alloca.OpenVMS has to change it to __ALLOCA or such.That is, the symbol "alloca" is almost but not quite portable in C. Again the function is never called from Modula-3 and need not be in an interface. Potential names here are: "alloca" what I'm using currently and remains my favorate."m3_alloca""m3_allocate_jmpbufs" So far I'm calling it "alloca". Thoughts/suggestions/criticisms? Just wait for the larger diff? As well, some of this could be more than "just random string function names".We could make them CONSTs in M3CG_Ops.i3. We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" and leave the "lowering" always to the backend -- this isn't likely, as many backends can share part of the lowering. But adding alloca to M3CG_Ops.i3 might be reasonable. Thanks,?- Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 5 20:53:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 5 Aug 2015 18:53:32 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 Message-ID: Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 6 01:45:02 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> References: <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com> Message-ID: Just wondering if the OpenVMS port is being used or if indeed its been tested. I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter On Thu, Aug 6, 2015 at 1:54 AM, Daniel Alejandro Benavides D. < dabenavidesd at yahoo.es> wrote: > Hi all: > long time ago, a proposal for supporting more higher level R like Aimer > Diwan et al, would benefit, support C -> Fortran compilation to further > parallelize in big machines (Virtual Speculative Parallel Machine), see [1]: > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.3982&rep=rep1&type=pdfear > > > > > Anyway, hearing of what you are trying to do especially support OPenVMS, > etc, is nice > > Thanks in advance > > [1] > R. L. Kennel y R. Eigenmann, ?Automatic Parallelization of C by Means of > Language Transcription?, en *Languages and Compilers for Parallel > Computing*, S. Chatterjee, J. F. Prins, L. Carter, J. Ferrante, Z. Li, D. > Sehr, y P.-C. Yew, Eds. Springer Berlin Heidelberg, 1999, pp. 166-180. > > > > El Mi?rcoles 5 de agosto de 2015 2:17, Jay K > escribi?: > > > I need some names. > > C has a type "jmp_buf" -- with no "u" and with an underscore. > > > I need a module in m3front for counting tries and managing jmp_bufs. > I call it Jmpbufs. > It could be Jumpbufs. > Or JumpBuffers. > Perhaps that is too direct. > Or sjljeh (setjmp/longjmp exception handling) > Or worked into the existing Marker. > > > A "constant variable" in C to hold sizeof(jmp_buf). > So far this was called Csetjmp__Jumpbuf_size. > However this "constant variable" should never be used from Modula-3. > Putting in a Modula-3 interface (Csetjmp), with the two level naming and a > double underscore is unnecessary and I suggest should not be done. > > > This name is an "interface" between m3front and m3core, i.e. > m3core/src/unix/Common/Uconstants.c. > It is always statically linked. > > > Potential names here are: > > > m3_jmpbuf_size my favorate at the moment > jmpbuf_size > Csetjmp_Jumpbuf_size already present by this name, but can be changed > > > > "alloca" > > > This is (will be) part of the interface between m3front and every backend. > While it is *almost* pass-through, it isn't. > The C backend targeting non-Win32, non-VMS, can just pass through the name. > But every other case in every other backend must treat the name specially. > cm3cg has to convert it to "builtin_alloca". > LLVM probably is like cm3cg. > Win32 non-C has to change it to __chkstk or perhaps _chkstk. > Win32 C has to change it to _alloca. > OpenVMS has to change it to __ALLOCA or such. > That is, the symbol "alloca" is almost but not quite portable in C. > > > Again the function is never called from Modula-3 and need not be in an > interface. > > > Potential names here are: > > "alloca" what I'm using currently and remains my favorate. > "m3_alloca" > "m3_allocate_jmpbufs" > > > So far I'm calling it "alloca". > > > Thoughts/suggestions/criticisms? > > > Just wait for the larger diff? > > > As well, some of this could be more than "just random string function > names". > We could make them CONSTs in M3CG_Ops.i3. > > > We could make separate functions in M3CG_Ops.i3, like "allocate_jmpbufs" > and leave the "lowering" always to the backend -- this isn't likely, as > many backends can share part of the lowering. > > > But adding alloca to M3CG_Ops.i3 might be reasonable. > > > Thanks, > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:06:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:06:02 +0000 Subject: [M3devel] cm3 on Mac OSX 10.10.4 In-Reply-To: References: , Message-ID: So far the problems aren't in anything you did, and likely won't be. They are all subtle minor things. I gave a detailed commit. I can repeat it if needed. - Jay Subject: Re: [M3devel] cm3 on Mac OSX 10.10.4 From: hosking at purdue.edu Date: Thu, 6 Aug 2015 09:51:54 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Keep me posted on the problems. I did the original OS X port. On Aug 6, 2015, at 4:53 AM, Jay K wrote:Fyi, I finally updated from 10.5.8 to 10.10.4.There are several problems. I'm working through them gradually. Bootstrapping via the C backend from 10.5.8 worked at least.(with the alloca(setjmp) change even) - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 02:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 00:12:18 +0000 Subject: [M3devel] names in (hopefully) upcoming changes In-Reply-To: References: , <1631927461.481488.1438790088232.JavaMail.yahoo@mail.yahoo.com>, Message-ID: I doubt the OpenVMS port is being used.It may or may not be complete. I managed to construct a Mac-hosted OpenVMS-targeted gcc.I built a native gcc with it. I didn't set the environment variables on the OpenVMS machinefor it to work. The biggest problem in all that is building the "sysroot".i.e. the headers and libraries. I built a bootstrap archive with cm3cg.I went to the trouble of building a ".com" file (makefile/scripty thing).So I might have gotten a "cm3" out of it. I don't remember. If someone has an OpenVMS machine available via ssh I might give it a try.I was working on an Alpha. I can see if it is still available. Also of note -- IA64 would need a little work, for all operating systems -- OpenVMS, HP-UX, Linux, FreeBSD, NT -- to scan the upper half of the stack. Thanks, - Jay From: peter.mckinna at gmail.com Date: Thu, 6 Aug 2015 09:45:02 +1000 Subject: Re: [M3devel] names in (hopefully) upcoming changes CC: jay.krell at cornell.edu; m3devel at elegosoft.com Just wondering if the OpenVMS port is being used or if indeed its been tested.I used VMS years ago but thought it went the way of the dodo when DEC bit the dust. Regards Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 6 10:46:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 6 Aug 2015 08:46:23 +0000 Subject: [M3devel] alloca(sizeof(jmp_buf)) changes for review In-Reply-To: References: Message-ID: Any objections? criticisms? suggestions?I'd like to commit this. My bootstrapping procedure (boot1.py) precludes cm3 incompatible with m3core/libm3 -- so this is holding up moving to 10.10.4, unless I do some git gyrations. Bootstrapping really needs to produce the matching m3core and libm3, so target can have older/newer source. Thank you, - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: alloca(sizeof(jmp_buf)) changes for review Date: Wed, 5 Aug 2015 09:15:22 +0000 Here are my changes for "alloca(sizeof(jmp_buf))". NOTE: I updated 3 of the 4 backends, and tested one of them so far.I can test the other two. NOTE: The update procedure is the same as usual, and very important, given the change to M3RT/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3. The compiler output is closedly tied to m3core. NOTE: This uses alloca even for the C backend. I expect to do better later. NOTE: The real value here is the deletion of all the target-dependent linesin m3-sys/m3middle/src/Target.m3. diff --git a/m3-libs/m3core/src/C/Common/Csetjmp.i3 b/m3-libs/m3core/src/C/Common/Csetjmp.i3index bd75beb..df2f139 100755--- a/m3-libs/m3core/src/C/Common/Csetjmp.i3+++ b/m3-libs/m3core/src/C/Common/Csetjmp.i3@@ -5,11 +5,22 @@ UNSAFE INTERFACE Csetjmp; FROM Ctypes IMPORT int; -(* jmp_buf is allocated with alloca(Csetjmp__Jumpbuf_size)+(* TODO? Move this to C?+ "u" in "ulongjmp" is probably for "underscore". This variant of longjmp never restores the signal mask.+ Because we believe we never change it?+ And restoring it is less efficient? (Requires possible kernel+ interaction?)++ If the platform only has "regular" longjmp and no signal mask,+ e.g. Win32, then this is resolved to that.+ + This function does not return in the usual sense.+ This is used to raise an exception.+ This is subject to be removed, either by using C, or "libunwind", or+ Win32 exceptions, or C++ exceptions. *)-<*EXTERNAL "Csetjmp__Jumpbuf_size" *> VAR Jumpbuf_size: INTEGER; <*EXTERNAL "Csetjmp__ulongjmp" *> PROCEDURE ulongjmp (env: ADDRESS; val: int); END Csetjmp.diff --git a/m3-libs/m3core/src/m3core.h b/m3-libs/m3core/src/m3core.hindex 37a9862..4be9ac3 100644--- a/m3-libs/m3core/src/m3core.h+++ b/m3-libs/m3core/src/m3core.h@@ -358,7 +358,9 @@ extern "C" { /* WORD_T/INTEGER are always exactly the same size as a pointer. * VMS sometimes has 32bit size_t/ptrdiff_t but 64bit pointers. */-#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)+/* commented out is correct, but so is the #else */+/*#if defined(_WIN64) || __INITIAL_POINTER_SIZE == 64 || defined(__LP64__) || defined(_LP64)*/+#if __INITIAL_POINTER_SIZE == 64 typedef INT64 INTEGER; typedef UINT64 WORD_T; #elsediff --git a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3index 6ec33fc..423a835 100644--- a/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3+++ b/m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3@@ -36,7 +36,7 @@ TYPE (* Except, ExceptElse, Finally *) class : INTEGER; (* ORD(ScopeKind) *) handles : ExceptionList; (* NIL-terminated list of exceptions handled *) info : RT0.RaiseActivation; (* current exception being dispatched *)- jmpbuf : ARRAY [0..16_FFFF] OF LONGREAL; (* gigantic, size is not used *)+ jmpbuf : ADDRESS; (* allocated with alloca *) END; TYPE (* FinallyProc *)@@ -172,7 +172,7 @@ PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *)- Csetjmp.ulongjmp (ADR(p.jmpbuf), 1); (* and jump... *)+ Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; diff --git a/m3-libs/m3core/src/unix/Common/Uconstants.c b/m3-libs/m3core/src/unix/Common/Uconstants.cindex 99b1adf..84095c1 100644--- a/m3-libs/m3core/src/unix/Common/Uconstants.c+++ b/m3-libs/m3core/src/unix/Common/Uconstants.c@@ -46,7 +46,7 @@ typedef int CheckMax[248 - sizeof(CheckMax_t)]; #include "UerrorX.h" /* This needs to be aligned to 16, at least for Win32. */-EXTERN_CONST INTEGER Csetjmp__Jumpbuf_size = ((sizeof(jmp_buf) + 15) & ~15);+EXTERN_CONST INTEGER m3_jmpbuf_size = ((sizeof(jmp_buf) + 15) & ~15); #ifndef _WIN32 diff --git a/m3-sys/cminstall/src/config-no-install/Darwin.common b/m3-sys/cminstall/src/config-no-install/Darwin.commonindex 0bf3e7e..7043383 100644 diff --git a/m3-sys/m3back/src/M3C.m3 b/m3-sys/m3back/src/M3C.m3index 82285e9..ab858b0 100644--- a/m3-sys/m3back/src/M3C.m3+++ b/m3-sys/m3back/src/M3C.m3@@ -66,7 +66,7 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT Err : ErrorHandler := DefaultErrorHandler; anonymousCounter := -1; c : Wr.T := NIL;- debug := 0; (* or 0, 1, 2, 3, 4 *)+ debug := 1; (* 1-4 *) stack : RefSeq.T := NIL; params : TextSeq.T := NIL; op_index := 0;@@ -80,6 +80,10 @@ T = M3CG_DoNothing.T BRANDED "M3C.T" OBJECT RTHooks_Raise_id: M3ID.T := 0; RTHooks_ReportFault_id: M3ID.T := 0; RTHooks_ReportFault_imported_or_declared := FALSE;+ alloca_id : M3ID.T := 0;+ setjmp_id : M3ID.T := 0;+ u_setjmp_id : M3ID.T := 0;+ longjmp_id : M3ID.T := 0; (* labels *) labels_min := FIRST(Label);@@ -1679,6 +1683,7 @@ TYPE Proc_t = M3CG.Proc OBJECT uplevels := FALSE; is_exception_handler := FALSE; is_RTHooks_Raise := FALSE;+ omit_prototype := FALSE; is_RTException_Raise := FALSE; no_return := FALSE; exit_proc_skipped := 0;@@ -1752,23 +1757,45 @@ BEGIN END IsNameExceptionHandler; PROCEDURE Proc_Init(proc: Proc_t; self: T): Proc_t =-VAR is_common := (proc.parent = NIL- AND (proc.exported = TRUE OR proc.imported = TRUE)- AND proc.level = 0- AND proc.return_type = CGType.Void);- is_RTHooks_ReportFault := (is_common- AND proc.name = self.RTHooks_ReportFault_id- AND proc.parameter_count = 2);- is_RTHooks_AssertFailed := (is_common- AND proc.name = self.RTHooks_AssertFailed_id- AND proc.parameter_count = 3);-BEGIN- proc.is_RTHooks_Raise := (is_common- AND proc.name = self.RTHooks_Raise_id- AND proc.parameter_count = 4);- proc.is_RTException_Raise := (is_common- AND proc.name = self.RTException_Raise_id- AND proc.parameter_count = 1);+VAR name := proc.name;+ parameter_count := proc.parameter_count;+ is_common := proc.parent = NIL+ AND (proc.exported = TRUE OR proc.imported = TRUE)+ AND proc.level = 0;+ is_common_void := is_common AND proc.return_type = CGType.Void;+ is_RTHooks_ReportFault := is_common_void+ AND name = self.RTHooks_ReportFault_id+ AND parameter_count = 2;+ is_RTHooks_AssertFailed := is_common_void+ AND name = self.RTHooks_AssertFailed_id+ AND parameter_count = 3;+BEGIN+ (* Omit a few prototypes that the frontend might have slightly wrong,+ e.g. alloca(unsigned int vs. unsigned long vs. unsigned long long)+ vs. not a function.+ e.g. setjmp(void* ) vs. setjmp(array)+ *)+ proc.omit_prototype := is_common+ AND parameter_count = 1 (* TODO 2 for longjmp *)+ AND (name = self.alloca_id+ (* TODO+ - add CGType.Jmpbuf+ - #include if there are any+ calls to setjmp/_setjmp/longjmp+ or instances of CGType.Jmpbuf+ - render CGType.Jmpbuf as "jmp_buf"+ - omit setjmp/_setjmp/longjmp prototypes+ OR name = self.setjmp_id+ OR name = self.u_setjmp_id+ OR name = self.longjmp_id+ *)+ );+ proc.is_RTHooks_Raise := is_common_void+ AND name = self.RTHooks_Raise_id+ AND parameter_count = 4;+ proc.is_RTException_Raise := is_common_void+ AND name = self.RTException_Raise_id+ AND parameter_count = 1; IF is_RTHooks_ReportFault THEN self.RTHooks_ReportFault_imported_or_declared := TRUE; END;@@ -1777,9 +1804,10 @@ BEGIN no_return(self); END; proc.self := self;- proc.name := Proc_FixName(proc.self, proc.name);- proc.is_exception_handler := proc.level > 0 AND proc.parameter_count = 1 AND IsNameExceptionHandler(self, NameT(proc.name));- proc.parameter_count_without_static_link := proc.parameter_count;+ proc.name := Proc_FixName(proc.self, name);+ name := proc.name;+ proc.is_exception_handler := proc.level > 0 AND parameter_count = 1 AND IsNameExceptionHandler(self, NameT(name));+ proc.parameter_count_without_static_link := parameter_count; proc.add_static_link := proc.level > 0; INC(proc.parameter_count, ORD(proc.add_static_link)); proc.locals := NEW(RefSeq.T).init();@@ -1865,8 +1893,10 @@ CONST Prefix = ARRAY OF TEXT { "#pragma warning(disable:4255) /* () change to (void) */", "#pragma warning(disable:4668) /* #if of undefined symbol */", "#endif",-"typedef char* ADDRESS;", (* TODO remove this when we finish strong typing *)-"typedef char* STRUCT;", (* TODO remove this when we finish strong typing *)+(* TODO ideally these are char* for K&R or ideally absent when strong+ typing and setjmp work done *)+"typedef char* ADDRESS;",+"typedef char* STRUCT;", "typedef signed char INT8;", "typedef unsigned char UINT8;", "typedef short INT16;",@@ -1907,6 +1937,8 @@ CONST Prefix = ARRAY OF TEXT { "#include ", (* try to remove this, it is slow -- need size_t *) "#endif", +(* "#include ", TODO do not always #include *)+ "/* http://c.knowcoding.com/view/23699-portable-alloca.html */", "/* Find a good version of alloca. */", "#ifndef alloca",@@ -1946,7 +1978,7 @@ CONST Prefix = ARRAY OF TEXT { "#define STRUCT1(n) typedef struct { volatile char a[n]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT2(n) typedef struct { volatile short a[n/2]; } STRUCT(n);", (* TODO prune if not used *) "#define STRUCT4(n) typedef struct { volatile int a[n/4]; } STRUCT(n);", (* TODO prune if not used *)-"#define STRUCT8(n) typedef struct { volatile double a[n/8]; } STRUCT(n);", (* TODO prune if not used *)+"#define STRUCT8(n) typedef struct { volatile UINT64 a[n/8]; } STRUCT(n);", (* TODO prune if not used *) "#ifdef __cplusplus", "#define DOTDOTDOT ...", "#else",@@ -2285,6 +2317,10 @@ BEGIN self.comment("M3_TARGET = ", Target.System_name); self.comment("M3_WORDSIZE = ", IntToDec(Target.Word.size)); self.static_link_id := M3ID.Add("_static_link");+ self.alloca_id := M3ID.Add("alloca");+ self.setjmp_id := M3ID.Add("setjmp");+ self.u_setjmp_id := M3ID.Add("_setjmp"); (* "u" is for underscore *)+ self.longjmp_id := M3ID.Add("longjmp"); self.RTHooks_ReportFault_id := M3ID.Add("RTHooks__ReportFault"); self.RTHooks_Raise_id := M3ID.Add("RTHooks__Raise"); self.RTException_Raise_id := M3ID.Add("RTException__Raise");@@ -3555,6 +3591,7 @@ BEGIN HelperFunctions_helper_with_type_and_array(self, op, type, types, ARRAY OF TEXT{first}); END HelperFunctions_helper_with_type; +(* TODO give up and #include ? *) PROCEDURE HelperFunctions_memset(self: HelperFunctions_t) = CONST text = "void* __cdecl memset(void*, int, size_t); /* string.h */"; BEGIN@@ -4270,6 +4307,9 @@ VAR params := proc.params; define_kr := NOT ansi AND kind = FunctionPrototype_t.Define; kr_part2 := ""; BEGIN+ IF proc.omit_prototype THEN+ RETURN "";+ END; IF NUMBER (params^) = 0 THEN text := text & "(void)"; ELSIF NOT ansi AND NOT define_kr THENdiff --git a/m3-sys/m3back/src/M3x86.m3 b/m3-sys/m3back/src/M3x86.m3index 206f9fb..2fff809 100644--- a/m3-sys/m3back/src/M3x86.m3+++ b/m3-sys/m3back/src/M3x86.m3@@ -3283,7 +3283,7 @@ CONST BP { "m3_rotate_right64",3, Type.Word64, Target.STDCALL }, BP { "m3_rotate64", 3, Type.Word64, Target.STDCALL }, - BP { "m3_alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX }+ BP { "alloca", 0, Type.Addr, Target.CDECL, "__chkstk", EAX } }; PROCEDURE start_int_proc (u: U; b: Builtin) =diff --git a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c b/m3-sys/m3cc/gcc/gcc/m3cg/parse.cindex d965c8a..fdbbdf4 100644--- a/m3-sys/m3cc/gcc/gcc/m3cg/parse.c+++ b/m3-sys/m3cc/gcc/gcc/m3cg/parse.c@@ -643,7 +643,7 @@ static GTY (()) tree t_int; #define t_void void_type_node static GTY (()) tree t_set; -static tree m3_alloca;+static tree alloca_tree; static const struct { UINT32 type_id; tree* t; } builtin_uids[] = { { UID_INTEGER, &t_int },@@ -1914,7 +1914,7 @@ m3_init_decl_processing (void) bits_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER); bytes_per_integer_tree = build_int_cst (t_word, BITS_PER_INTEGER / BITS_PER_UNIT); tree stdcall = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("stdcall"));- m3_alloca = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("m3_alloca"));+ alloca_tree = get_identifier_with_length (CONSTANT_STRING_AND_LENGTH ("alloca")); stdcall_list = build_tree_list (stdcall, NULL); t_set = m3_build_pointer_type (t_word); @@ -3076,7 +3076,7 @@ fix_name (PCSTR name, size_t length, UINT32 type_id) } else if (type_id == NO_UID) {- buf = (PSTR)alloca (sizet_add(length, 1));+ buf = (PSTR)alloca (sizet_add (length, 1)); buf[0] = 'M'; memcpy (&buf[1], name, length); length += 1;@@ -3153,7 +3153,8 @@ m3_pop_param (tree type) static tree m3_convert_function_to_builtin (tree p) {- if (DECL_NAME (p) == m3_alloca)+ tree name = DECL_NAME (p);+ if (name == alloca_tree) p = builtin_decl_explicit (BUILT_IN_ALLOCA); return p; }diff --git a/m3-sys/m3front/src/misc/M3.i3 b/m3-sys/m3front/src/misc/M3.i3index ec7f972..4a3793e 100644--- a/m3-sys/m3front/src/misc/M3.i3+++ b/m3-sys/m3front/src/misc/M3.i3@@ -11,7 +11,7 @@ INTERFACE M3; -IMPORT M3ID, M3Buf;+IMPORT M3ID, M3Buf, Jmpbufs; TYPE Flag = BITS 1 FOR BOOLEAN;@@ -33,6 +33,7 @@ TYPE ExSet <: Node; (* == ESet.T *) ExSetList <: REFANY; (* == list of ExSet *) EqAssumption <: ADDRESS; (* == Type.Assumption *)+ Procedure <: Value; (* == Procedure.T *) (*--------------------------------------------------------- type checking ---*) @@ -41,13 +42,15 @@ TYPE (* the "global state" that is passed around during type checking *) raises_others : BOOLEAN; ok_to_raise : ExSetList; no_error : ExSetList;+ jmpbufs := Jmpbufs.CheckState { }; END; CONST OuterCheckState = CheckState { raises_others := FALSE, ok_to_raise := NIL,- no_error := NIL+ no_error := NIL,+ jmpbufs := Jmpbufs.CheckState { } }; (*-------------------------------------------------------- fingerprinting ---*)diff --git a/m3-sys/m3front/src/misc/Marker.i3 b/m3-sys/m3front/src/misc/Marker.i3index db880a0..8d8f044 100644--- a/m3-sys/m3front/src/misc/Marker.i3+++ b/m3-sys/m3front/src/misc/Marker.i3@@ -64,8 +64,9 @@ PROCEDURE PopFrame (frame: CG.Var); PROCEDURE SetLock (acquire: BOOLEAN; var: CG.Var; offset: INTEGER); (* generate the call to acquire or release a mutex *) -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label);-(* call 'setjmp' on 'frame's jmpbuf and branch to 'handler' on re-returns. *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label);+(* frame.jmpbuf = jmpbuf+ if (setjmp(jmpbuf)) goto handler *) PROCEDURE Reset (); diff --git a/m3-sys/m3front/src/misc/Marker.m3 b/m3-sys/m3front/src/misc/Marker.m3index 6512fbb..9421b7c 100644--- a/m3-sys/m3front/src/misc/Marker.m3+++ b/m3-sys/m3front/src/misc/Marker.m3@@ -53,9 +53,6 @@ VAR all_frames : FramePtr := NIL; n_frames : INTEGER := 0; save_depth : INTEGER := 0;- setjmp : CG.Proc := NIL;- alloca : CG.Proc := NIL;- Jumpbuf_size : CG.Var := NIL; tos : INTEGER := 0; stack : ARRAY [0..50] OF Frame; @@ -231,68 +228,13 @@ PROCEDURE CallFinallyHandler (info: CG.Var; END; END CallFinallyHandler; -PROCEDURE CaptureState (frame: CG.Var; handler: CG.Label) =- CONST Alloca_jmpbuf = FALSE;- VAR new: BOOLEAN;- label_already_allocated: CG.Label;- BEGIN- (* int setjmp(void* ); *)- IF setjmp = NIL THEN- setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,- Target.Integer.cg_type,- Target.DefaultCall, new);- IF (new) THEN- EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,- Target.Address.align, CG.Type.Addr, 0,- in_memory := FALSE, up_level := FALSE,- f := CG.Never);- END;- END;- - IF Alloca_jmpbuf THEN- label_already_allocated := CG.Next_label ();-- (* void* _alloca(size_t); *)- IF alloca = NIL THEN- alloca := CG.Import_procedure (M3ID.Add ("m3_alloca"), 1, CG.Type.Addr,- Target.DefaultCall, new);- IF new THEN- EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0, in_memory := FALSE,- up_level := FALSE, f := CG.Never);- END;- END;- (* extern /*const*/ size_t Csetjmp__Jumpbuf_size/* = sizeof(jmp_buf)*/; *)- IF Jumpbuf_size = NIL THEN- Jumpbuf_size := CG.Import_global (M3ID.Add ("Csetjmp__Jumpbuf_size"),- Target.Word.size, Target.Word.align,- Target.Word.cg_type, 0);- END;- - (* if (!frame.jmpbuf)- frame.jmpbuf = alloca(Csetjmp__Jumpbuf_size);- *)- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- CG.Load_nil ();- CG.If_compare (Target.Address.cg_type, CG.Cmp.NE, label_already_allocated, CG.Likely);-- CG.Start_call_direct (alloca, 0, Target.Address.cg_type);- CG.Load_int (Target.Word.cg_type, Jumpbuf_size);- CG.Pop_param (Target.Word.cg_type);- CG.Call_direct (alloca, Target.Address.cg_type);- CG.Check_nil (CG.RuntimeError.BadMemoryReference);- CG.Store_addr (frame, M3RT.EF1_jmpbuf);-- CG.Set_label (label_already_allocated);- END;-- (* setjmp(frame.jmpbuf) or setjmp(&frame.jmpbuf) *)+PROCEDURE CaptureState (frame: CG.Var; jmpbuf: CG.Var; handler: CG.Label) =+ VAR setjmp := Module.GetSetjmp (Module.Current ());+ BEGIN+ CG.Load_addr (jmpbuf);+ CG.Store_addr (frame, M3RT.EF1_jmpbuf); CG.Start_call_direct (setjmp, 0, Target.Integer.cg_type);- IF Alloca_jmpbuf THEN- CG.Load_addr (frame, M3RT.EF1_jmpbuf);- ELSE- CG.Load_addr_of (frame, M3RT.EF1_jmpbuf, 128);- END;+ CG.Load_addr (jmpbuf); CG.Pop_param (CG.Type.Addr); CG.Call_direct (setjmp, Target.Integer.cg_type); CG.If_true (handler, CG.Never);@@ -820,9 +762,6 @@ PROCEDURE Reset () = all_frames := NIL; n_frames := 0; save_depth := 0;- setjmp := NIL;- alloca := NIL;- Jumpbuf_size := NIL; tos := 0; END Reset; diff --git a/m3-sys/m3front/src/misc/m3makefile b/m3-sys/m3front/src/misc/m3makefileindex 721cc27..7d5fba3 100644--- a/m3-sys/m3front/src/misc/m3makefile+++ b/m3-sys/m3front/src/misc/m3makefile@@ -16,6 +16,7 @@ module ("M3WString") module ("Token") module ("Error") module ("Host")+module ("Jmpbufs") module ("Marker") module ("Scanner") module ("Scope")diff --git a/m3-sys/m3front/src/stmts/TryFinStmt.m3 b/m3-sys/m3front/src/stmts/TryFinStmt.m3index 89233c5..e0d9b33 100644--- a/m3-sys/m3front/src/stmts/TryFinStmt.m3+++ b/m3-sys/m3front/src/stmts/TryFinStmt.m3@@ -10,6 +10,7 @@ MODULE TryFinStmt; IMPORT M3ID, CG, Token, Scanner, Stmt, StmtRep, Marker, Target, Type, Addr; IMPORT RunTyme, Procedure, ProcBody, M3RT, Scope, Fmt, Host, TryStmt, Module;+IMPORT Jmpbufs; FROM Stmt IMPORT Outcome; TYPE@@ -20,6 +21,7 @@ TYPE viaProc : BOOLEAN; scope : Scope.T; handler : HandlerProc;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -30,6 +32,7 @@ TYPE HandlerProc = ProcBody.T OBJECT self: P; activation: CG.Var;+ jmpbufs : Jmpbufs.Proc; OVERRIDES gen_decl := EmitDecl; gen_body := EmitBody;@@ -66,6 +69,7 @@ PROCEDURE Parse (body: Stmt.T; ): Stmt.T = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR zz: Scope.T; oc: Stmt.Outcomes; name: INTEGER; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); Marker.PushFinally (CG.No_label, CG.No_label, NIL); Stmt.TypeCheck (p.body, cs); Marker.Pop ();@@ -89,8 +93,11 @@ PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = next_uid := 0; END; zz := Scope.Push (p.scope);+ p.handler.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs,+ M3ID.Add (p.handler.name)); Scope.TypeCheck (p.scope, cs); Stmt.TypeCheck (p.finally, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.handler.jmpbufs); Scope.Pop (zz); END; END;@@ -226,6 +233,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = CG.Gen_location (p.forigin); IF (Host.inline_nested_procs) THEN CG.Begin_procedure (p.handler.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (p.handler.jmpbufs); xc := Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (p.handler.cg_proc);@@ -272,6 +280,7 @@ PROCEDURE EmitBody (x: HandlerProc) = Scanner.offset := p.forigin; CG.Gen_location (p.forigin); CG.Begin_procedure (x.cg_proc);+ Jmpbufs.CompileProcAllocateJmpbufs (x.jmpbufs); EVAL Stmt.Compile (p.finally); CG.Exit_proc (CG.Type.Void); CG.End_procedure (x.cg_proc);@@ -302,7 +311,7 @@ PROCEDURE Compile3 (p: P): Stmt.Outcomes = l := CG.Next_label (3); CG.Set_label (l, barrier := TRUE); Marker.PushFrame (frame, M3RT.HandlerClass.Finally);- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) Marker.PushFinally (l, l+1, frame);diff --git a/m3-sys/m3front/src/stmts/TryStmt.m3 b/m3-sys/m3front/src/stmts/TryStmt.m3index 8e7a308..a81e0da 100644--- a/m3-sys/m3front/src/stmts/TryStmt.m3+++ b/m3-sys/m3front/src/stmts/TryStmt.m3@@ -10,7 +10,7 @@ MODULE TryStmt; IMPORT M3, M3ID, CG, Variable, Scope, Exceptionz, Value, Error, Marker; IMPORT Type, Stmt, StmtRep, TryFinStmt, Token;-IMPORT Scanner, ESet, Target, M3RT, Tracer;+IMPORT Scanner, ESet, Target, M3RT, Tracer, Jmpbufs; FROM Scanner IMPORT Match, MatchID, GetToken, Fail, cur; TYPE@@ -22,6 +22,7 @@ TYPE hasElse : BOOLEAN; elseBody : Stmt.T; handled : ESet.T;+ jmpbufs : Jmpbufs.Try; OVERRIDES check := Check; compile := Compile;@@ -162,6 +163,7 @@ PROCEDURE ReverseHandlers (p: P) = PROCEDURE Check (p: P; VAR cs: Stmt.CheckState) = VAR h: Handler; handled: ESet.T; BEGIN+ Jmpbufs.CheckTry (cs.jmpbufs, p.jmpbufs); h := p.handles; WHILE (h # NIL) DO CheckLabels (h, p.scope, cs); h := h.next; END; @@ -429,7 +431,7 @@ PROCEDURE Compile2 (p: P): Stmt.Outcomes = END; (* capture the machine state *)- Marker.CaptureState (frame, l+1);+ Marker.CaptureState (frame, Jmpbufs.CompileTryGetJmpbuf (p.jmpbufs), l+1); (* compile the body *) oc := Stmt.Compile (p.body);diff --git a/m3-sys/m3front/src/values/Module.i3 b/m3-sys/m3front/src/values/Module.i3index 58fa9bd..3935126 100644--- a/m3-sys/m3front/src/values/Module.i3+++ b/m3-sys/m3front/src/values/Module.i3@@ -72,4 +72,13 @@ PROCEDURE Reset (); PROCEDURE MakeCurrent (t: T); (* refresh 't' and its imports for the current compilation *) +(* Exception handling support, using setjmp/longjmp, w/o front/middle-end+ knowing jmpbuf size -- use alloca at runtime getting size from+ constant initialized in C; these functions are in Module+ to accomodate hypothetical multi-threaded m3front,+ i.e. instead of having globals initialized on-demand in Jmpbufs. *)+PROCEDURE GetAlloca (t: T) : CG.Proc;+PROCEDURE GetJmpbufSize (t: T): CG.Var;+PROCEDURE GetSetjmp (t: T) : CG.Proc;+ END Module.diff --git a/m3-sys/m3front/src/values/Module.m3 b/m3-sys/m3front/src/values/Module.m3index 336b0f2..b587639 100644--- a/m3-sys/m3front/src/values/Module.m3+++ b/m3-sys/m3front/src/values/Module.m3@@ -12,7 +12,7 @@ IMPORT M3, M3ID, CG, Value, ValueRep, Scope, Stmt, Error, ESet, External; IMPORT Variable, Type, Procedure, Ident, M3Buf, BlockStmt, Int; IMPORT Host, Token, Revelation, Coverage, Decl, Scanner, WebInfo; IMPORT ProcBody, Target, M3RT, Marker, File, Tracer, Wr;-IMPORT WCharr; +IMPORT WCharr, Jmpbufs; FROM Scanner IMPORT GetToken, Fail, Match, MatchID, cur; @@ -50,6 +50,10 @@ REVEAL value_info : Value.T; lazyAligned : BOOLEAN; containsLazyAlignments: BOOLEAN;+ jmpbuf_size : CG.Var := NIL;+ alloca : CG.Proc := NIL;+ setjmp : CG.Proc := NIL;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := TypeCheckMethod; set_globals := ValueRep.NoInit;@@ -109,6 +113,61 @@ PROCEDURE Reset () = INC (compile_age); END Reset; +PROCEDURE GetAlloca (t: T) : CG.Proc =+VAR new := FALSE;+BEGIN+ (* alloca must be special cased by backends to mean+ alloca, _alloca, chkstk, etc. *)+ IF t.alloca = NIL THEN+ t.alloca := CG.Import_procedure (M3ID.Add ("alloca"), 1, CG.Type.Addr,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.NoID, Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0, in_memory := FALSE,+ up_level := FALSE, f := CG.Never);+ END;+ END;+ RETURN t.alloca;+END GetAlloca;+++PROCEDURE GetJmpbufSize (t: T): CG.Var =+BEGIN+ (* m3_jmpbuf_size is a "constant variable" initialized in + C via:+ #include + extern const m3_jmpbuf_size = sizeof(jmp_buf);+ As an optimization, and to avoid any matters involving dynamically+ importing data on Win32, Uconstants is always statically linked.+ + This isolates the front/middle end from the target.+ *)+ IF t.jmpbuf_size = NIL THEN+ t.jmpbuf_size := CG.Import_global (M3ID.Add ("m3_jmpbuf_size"),+ Target.Word.size, Target.Word.align,+ Target.Word.cg_type, 0);+ END;+ RETURN t.jmpbuf_size;+END GetJmpbufSize;++PROCEDURE GetSetjmp (t: T): CG.Proc =+VAR new := FALSE;+BEGIN+ (* int setjmp(void* ); *)+ IF t.setjmp = NIL THEN+ t.setjmp := CG.Import_procedure (M3ID.Add (Target.Setjmp), 1,+ Target.Integer.cg_type,+ Target.DefaultCall, new);+ IF new THEN+ EVAL CG.Declare_param (M3ID.Add ("jmpbuf"), Target.Address.size,+ Target.Address.align, CG.Type.Addr, 0,+ in_memory := FALSE, up_level := FALSE,+ f := CG.Never);+ END;+ END;+ RETURN t.setjmp;+END GetSetjmp;+ PROCEDURE Create (name: M3ID.T): T = VAR t: T; BEGIN@@ -592,8 +651,10 @@ PROCEDURE TypeCheck (t: T; main: BOOLEAN; VAR cs: Value.CheckState) = Revelation.TypeCheck (t.revelations); Scope.TypeCheck (t.localScope, cs); IF (NOT t.interface) THEN+ t.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, 0); BlockStmt.CheckTrace (t.trace, cs); Stmt.TypeCheck (t.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, t.jmpbufs); END; ESet.Pop (cs, NIL, t.fails, stop := TRUE);@@ -1041,6 +1102,7 @@ PROCEDURE EmitBody (x: InitBody) = (* perform the main body *) Tracer.Push (t.trace);+ Jmpbufs.CompileProcAllocateJmpbufs (t.jmpbufs); EVAL Stmt.Compile (t.block); Tracer.Pop (t.trace); diff --git a/m3-sys/m3front/src/values/Procedure.i3 b/m3-sys/m3front/src/values/Procedure.i3index dddb1f3..114a3ec 100644--- a/m3-sys/m3front/src/values/Procedure.i3+++ b/m3-sys/m3front/src/values/Procedure.i3@@ -8,9 +8,9 @@ INTERFACE Procedure; -IMPORT CG, Value, Type, CallExpr, Decl;+IMPORT CG, Value, Type, CallExpr, Decl, M3; -TYPE T <: Value.T;+TYPE T = M3.Procedure; (* <: Value.T *) PROCEDURE ParseDecl (READONLY att: Decl.Attributes; headerOnly: BOOLEAN := FALSE);diff --git a/m3-sys/m3front/src/values/Procedure.m3 b/m3-sys/m3front/src/values/Procedure.m3index ad0d39c..39df85b 100644--- a/m3-sys/m3front/src/values/Procedure.m3+++ b/m3-sys/m3front/src/values/Procedure.m3@@ -12,7 +12,7 @@ MODULE Procedure; IMPORT M3, M3ID, CG, Value, ValueRep, Type, Scope, Error, Host; IMPORT ProcType, Stmt, BlockStmt, Marker, Coverage, M3RT; IMPORT CallExpr, Token, Variable, ProcExpr, Tracer;-IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal;+IMPORT Scanner, Decl, ESet, ProcBody, Target, Expr, Formal, Jmpbufs; FROM Scanner IMPORT GetToken, Match, MatchID, cur; REVEAL@@ -33,6 +33,7 @@ REVEAL cg_proc : CG.Proc; next_cg_proc : T; end_origin : INTEGER;+ jmpbufs : Jmpbufs.Proc; OVERRIDES typeCheck := Check; set_globals := ValueRep.NoInit;@@ -307,7 +308,9 @@ PROCEDURE CheckBody (p: T; VAR cs: Value.CheckState) = INC (Type.recursionDepth); Scope.TypeCheck (p.syms, cs); Marker.PushProcedure (result, p.result, cconv);+ p.jmpbufs := Jmpbufs.CheckProcPush (cs.jmpbufs, p.name); Stmt.TypeCheck (p.block, cs);+ Jmpbufs.CheckProcPop (cs.jmpbufs, p.jmpbufs); Marker.Pop (); Scope.WarnUnused (p.syms); DEC (Type.recursionDepth);@@ -574,6 +577,7 @@ PROCEDURE GenBody (p: T) = Scope.InitValues (p.syms); Scanner.offset := BlockStmt.BodyOffset (p.block); Coverage.CountProcedure (p);+ Jmpbufs.CompileProcAllocateJmpbufs (p.jmpbufs); oc := Stmt.Compile (p.block); fallThru := (Stmt.Outcome.FallThrough IN oc); EndRaises (p, l, frame, fallThru);diff --git a/m3-sys/m3middle/src/M3RT.m3 b/m3-sys/m3middle/src/M3RT.m3index 058cea3..463860c 100644--- a/m3-sys/m3middle/src/M3RT.m3+++ b/m3-sys/m3middle/src/M3RT.m3@@ -10,9 +10,9 @@ IMPORT Target; PROCEDURE Init () = VAR- IP := Target.Integer.pack;- AP := Target.Address.pack;- CP := Target.Char.pack;+ IP := Target.Integer.pack; (* 32 or 64, same as Target.Address.pack *)+ AP := Target.Address.pack; (* 32 or 64, same as Target.Integer.pack *)+ CP := Target.Char.pack; (* 8 *) BEGIN (* closure offsets *) CL_marker := 0; (* : INTEGER *)@@ -57,8 +57,8 @@ PROCEDURE Init () = (* Except, ExceptElse, and Finally frames *) EF1_handles := EF_SIZE; (* : ADDRESS *) EF1_info := EF1_handles + AP; (* : RTException.Activation *)- EF1_jmpbuf := RoundUp (EF1_info + EA_SIZE, 128); (* : jmp_buf *)- EF1_SIZE := EF1_jmpbuf + Target.Jumpbuf_size;+ EF1_jmpbuf := EF1_info + EA_SIZE; (* : jmp_buf *)+ EF1_SIZE := EF1_jmpbuf + AP; (* FinallyProc frames *) EF2_handler := EF_SIZE; (* : ADDRESS (PROC) *)@@ -149,10 +149,5 @@ PROCEDURE Init () = MUTEX_release := 1 * AP; (*: PROC() *) END Init; -PROCEDURE RoundUp (a, b: INTEGER): INTEGER =- BEGIN- RETURN (a + b - 1) DIV b * b;- END RoundUp;- BEGIN END M3RT.diff --git a/m3-sys/m3middle/src/Target.i3 b/m3-sys/m3middle/src/Target.i3index c0e5c4c..bd27a07 100644--- a/m3-sys/m3middle/src/Target.i3+++ b/m3-sys/m3middle/src/Target.i3@@ -386,9 +386,6 @@ VAR (*CONST*) will cause an address faults. Hence, no explicit NIL checks are needed for dereferencing with offsets in this range. *) - (* Thread stacks *)- Jumpbuf_size : CARDINAL; (* size of a "jmp_buf" *)- (* floating point values *) All_floats_legal : BOOLEAN; (* If all bit patterns are "legal" floating point values (i.e. they candiff --git a/m3-sys/m3middle/src/Target.m3 b/m3-sys/m3middle/src/Target.m3index 514a75c..475422f 100644--- a/m3-sys/m3middle/src/Target.m3+++ b/m3-sys/m3middle/src/Target.m3@@ -193,132 +193,11 @@ PROCEDURE Init (system: TEXT; in_OS_name: TEXT; backend_mode: M3BackendMode_t): END; CASE System OF- - | Systems.ALPHA_LINUX => Jumpbuf_size := 34 * Address.size;- | Systems.ALPHA_OPENBSD => Jumpbuf_size := 81 * Address.size;- | Systems.ALPHA_OSF => Jumpbuf_size := 84 * Address.size;-- | Systems.I386_FREEBSD, Systems.FreeBSD4 =>- Jumpbuf_size := 11 * Address.size;-- | Systems.AMD64_NETBSD,- Systems.AMD64_OPENBSD,- Systems.AMD64_FREEBSD =>- Jumpbuf_size := 12 * Address.size;-- | Systems.ARM_LINUX,- Systems.ARMEL_LINUX =>- Jumpbuf_size := 64 * Int64.size; (* 392 bytes = 49 * Int64.size on Raspberry Pi *)-- | Systems.PA32_HPUX =>- (* 200 bytes with 8 byte alignment *)- Jumpbuf_size := 50 * Address.size;-- | Systems.PA64_HPUX =>- (* 640 bytes with 16 byte alignment *)- Jumpbuf_size := 80 * Address.size;-- | Systems.MIPS64_OPENBSD,- Systems.MIPS64EL_OPENBSD =>- Jumpbuf_size := 16_53 * Address.size;-- | Systems.I386_INTERIX =>-- (* Visual C++'s 16 plus 2 ints: is sigmask saved, its value. *)-- Jumpbuf_size := 18 * Address.size;-- | Systems.NT386, Systems.I386_NT, Systems.I386_CYGWIN, Systems.I386_MINGW =>-- (* Cygwin: 13, Visual C++: 16, Interix: 18.- Use 18 for interop.- Cygwin's setjmp.h is wrong by a factor of 4.- Cygwin provides setjmp and _setjmp that resolve the same.- Visual C++ provides only _setjmp.- Visual C++ also has _setjmp3 that the compiler generates- a call to. In fact _setjmp appears to only use 8 ints- and _setjmp3 appears to use more. Consider using _setjmp3.- *)- Jumpbuf_size := 18 * Address.size; | Systems.AMD64_NT =>- (* 256 bytes with 16 byte alignment *)- Jumpbuf_size := 32 * Int64.size; Setjmp := "setjmp"; - | Systems.IA64_FREEBSD, Systems.IA64_HPUX,- Systems.IA64_LINUX, Systems.IA64_NETBSD, Systems.IA64_NT,- Systems.IA64_OPENBSD, Systems.IA64_VMS =>- (* random guess: 1K *)- Jumpbuf_size := 128 * Address.size;-- | Systems.SPARC32_SOLARIS, Systems.SOLgnu, Systems.SOLsun =>- (* 76 bytes with 4 byte alignment *)- Jumpbuf_size := 19 * Address.size;-- | Systems.SPARC32_LINUX =>- Jumpbuf_size := 16_90 * Char.size;-- | Systems.SPARC64_OPENBSD =>- Jumpbuf_size := 14 * Address.size;-- | Systems.SPARC64_LINUX =>- Jumpbuf_size := 16_280 * Char.size;-- | Systems.SPARC64_SOLARIS =>- (* 96 bytes with 8 byte alignment *)- Jumpbuf_size := 12 * Address.size;-- | Systems.I386_SOLARIS =>- (* 40 bytes with 4 byte alignment *)- Jumpbuf_size := 10 * Address.size;-- | Systems.AMD64_SOLARIS =>- (* 64 bytes with 8 byte alignment *)- Jumpbuf_size := 8 * Address.size;-- | Systems.I386_LINUX, Systems.LINUXLIBC6 =>- Jumpbuf_size := 39 * Address.size;-- | Systems.AMD64_LINUX =>- Jumpbuf_size := 25 * Address.size;-- | Systems.I386_DARWIN =>- Jumpbuf_size := 18 * Address.size;-- | Systems.AMD64_DARWIN =>- Jumpbuf_size := ((9 * 2) + 3 + 16) * Int32.size;-- | Systems.ARM_DARWIN =>- Jumpbuf_size := 28 * Address.size;-- | Systems.PPC_DARWIN =>- Jumpbuf_size := 768 * Word8.size;-- | Systems.PPC64_DARWIN =>- Jumpbuf_size := 872 * Word8.size;-- | Systems.PPC_LINUX => - Jumpbuf_size := 74 * Int64.size;- (* ideal alignment is 16 bytes, but 4 is ok *)-- | Systems.PPC32_OPENBSD => - Jumpbuf_size := 100 * Address.size;-- | Systems.I386_NETBSD =>- Jumpbuf_size := 14 * Address.size; (* 13? *)- - | Systems.ALPHA32_VMS,- Systems.ALPHA64_VMS =>- Jumpbuf_size := 68 * Word64.size;--(* | Systems.I386_MSDOS =>- Jumpbuf_size := 172 * Char.size; TBD *)-- | Systems.I386_OPENBSD =>- Jumpbuf_size := 10 * Address.size;-- ELSE RETURN FALSE;+ ELSE END; InitCallingConventions (backend_mode,diff --git a/m3-sys/m3tests/src/p2/p251/Main.m3 b/m3-sys/m3tests/src/p2/p251/Main.m3index 998415e..b6d2d30 100644--- a/m3-sys/m3tests/src/p2/p251/Main.m3+++ b/m3-sys/m3tests/src/p2/p251/Main.m3@@ -172,6 +172,70 @@ BEGIN TRY F6(); EXCEPT END; END Main; +PROCEDURE Finally () =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();+ FINALLY+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ END;+END Finally;++PROCEDURE NestedFinally() =+BEGIN+ (* same thing but in FINALLY, and nested FINALLY *)+ (* NOTE: This testing is haphazard as I don't+ understand exception handling enough to aim for coverage. *)+ TRY+ top_of_stack := GetStack();+ F0();++ FINALLY+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END; TRY TRY F1(); FINALLY F0(); END; EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ + TRY+ TRY+ F2();+ EXCEPT+ ELSE+ Put("exception " & Int(Line())); NL();+ END;+ FINALLY+ F0();+ END; + END;++ TRY top_of_stack := GetStack(); TRY F0();+ FINALLY TRY F0(); FINALLY F0(); END; END; FINALLY TRY F0(); FINALLY F0(); END; END;+ +END NestedFinally;+ BEGIN Main();++ (* same thing but in Module main *)++ top_of_stack := GetStack();+ F0();+ TRY F1(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F2(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F3(); EXCEPT ELSE Put("exception " & Int(Line())); NL(); END;+ TRY F4(); EXCEPT END;+ TRY F5(); EXCEPT END;+ TRY F6(); EXCEPT END;+ + Finally();+ NestedFinally();+ END Main. ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:31:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:31:14 +0000 Subject: [M3devel] alloca(jmpbuf_size) is in In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, Message-ID: I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 03:33:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 01:33:00 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? Message-ID: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:01:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:01:30 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , Message-ID: And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:02:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:02:42 +0000 Subject: [M3devel] updating m3cg? Message-ID: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 05:05:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 03:05:27 +0000 Subject: [M3devel] multi-threaded m3front? Message-ID: Is anyone interested in updating m3front to be multi-threaded? I haven't seen a single core system in a while. Surely each module can be compiled separately, possibly with some serialization around compiling interfaces? Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 18:18:59 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 16:18:59 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu> Message-ID: https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Aug 9 18:58:41 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Sun, 09 Aug 2015 09:58:41 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150809165841.BA38C1A2062@async.async.caltech.edu> I think this is a great idea. I made the back-end parallel already (at least for the version of the compiler that calls the gcc-based backend). To enable that you have to set "M3_PARALLEL_BACK" to the number of threads you want to spawn in your cm3.cfg . Mika Jay K writes: >--===============7278135725256123965== >Content-Type: multipart/alternative; > boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > >Is anyone interested in updating m3front to be multi-threaded? >I haven't seen a single core system in a while. >Surely each module can be compiled separately=2C possibly with some seriali= >zation around compiling interfaces? >Thanks=2C - Jay > > = > >--_9e876ef9-e673-486b-befa-abff52e19a99_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > >
Is anyone interested in updating= > m3front to be multi-threaded?

I haven't seen a single c= >ore system in a while.

Surely each module can be c= >ompiled separately=2C possibly with some serialization around compiling int= >erfaces?

Thanks=2C
 =3B- Jay

= >
>= > >--_9e876ef9-e673-486b-befa-abff52e19a99_-- > >--===============7278135725256123965== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============7278135725256123965==-- From rodney_bates at lcwb.coop Sun Aug 9 21:05:57 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:05:57 -0500 Subject: [M3devel] updating m3cg? In-Reply-To: References: Message-ID: <55C7A495.6090409@lcwb.coop> Not high on my priority list, but I wouldn't object. On 08/08/2015 10:02 PM, Jay K wrote: > Is there any interesting in updating the gcc backend from 4.7, like to 5.2? > I'm guessing not really. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 9 21:14:48 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 09 Aug 2015 14:14:48 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150809165841.BA38C1A2062@async.async.caltech.edu> References: <20150809165841.BA38C1A2062@async.async.caltech.edu> Message-ID: <55C7A6A8.2050805@lcwb.coop> On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > I think this is a great idea. I made the back-end parallel already (at > least for the version of the compiler that calls the gcc-based backend). I believe this works for any executable(s) separate from cm3 itself. > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > you want to spawn in your cm3.cfg . > > Mika > > Jay K writes: >> --===============7278135725256123965== >> Content-Type: multipart/alternative; >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> Is anyone interested in updating m3front to be multi-threaded? >> I haven't seen a single core system in a while. >> Surely each module can be compiled separately=2C possibly with some seriali= >> zation around compiling interfaces? >> Thanks=2C - Jay >> >> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >>
Is anyone interested in updating= >> m3front to be multi-threaded?

I haven't seen a single c= >> ore system in a while.

Surely each module can be c= >> ompiled separately=2C possibly with some serialization around compiling int= >> erfaces?

Thanks=2C
 =3B- Jay

= >>
>> = >> >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- >> >> --===============7278135725256123965== >> Content-Type: text/plain; charset="us-ascii" >> MIME-Version: 1.0 >> Content-Transfer-Encoding: 7bit >> Content-Disposition: inline >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> --===============7278135725256123965==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 9 21:18:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:18:12 +0000 Subject: [M3devel] updating m3cg? In-Reply-To: <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> References: , <316CB5DF-CAFF-4310-8D95-3D9C7BD56F28@purdue.edu> Message-ID: The level of change of supported targets would likely be small either way.Maybe a little lost, maybe a little gained.The C or C++ backend to me has the most reach. - Jay Subject: Re: [M3devel] updating m3cg? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:05:02 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu If it gets us running on newer targets then yes. Sent from my iPad On 9 Aug 2015, at 1:02 pm, Jay K wrote: Is there any interesting in updating the gcc backend from 4.7, like to 5.2?I'm guessing not really. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 9 21:20:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 9 Aug 2015 19:20:01 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <55C7A6A8.2050805@lcwb.coop> References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: Right, agreed -- this should work for LLVM, and the C compiler.LLVM should be multi-threadable in-proc too. Fixing m3front itself -- we'd have to identify and deal with all the globals.For the alloca/setjmp work I put the "globals" in Module.T. - Jay > Date: Sun, 9 Aug 2015 14:14:48 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > I think this is a great idea. I made the back-end parallel already (at > > least for the version of the compiler that calls the gcc-based backend). > > I believe this works for any executable(s) separate from cm3 itself. > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > you want to spawn in your cm3.cfg . > > > > Mika > > > > Jay K writes: > >> --===============7278135725256123965== > >> Content-Type: multipart/alternative; > >> boundary="_9e876ef9-e673-486b-befa-abff52e19a99_" > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> Is anyone interested in updating m3front to be multi-threaded? > >> I haven't seen a single core system in a while. > >> Surely each module can be compiled separately=2C possibly with some seriali= > >> zation around compiling interfaces? > >> Thanks=2C - Jay > >> > >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >>
Is anyone interested in updating= > >> m3front to be multi-threaded?

I haven't seen a single c= > >> ore system in a while.

Surely each module can be c= > >> ompiled separately=2C possibly with some serialization around compiling int= > >> erfaces?

Thanks=2C
 =3B- Jay

= > >>
> >> = > >> > >> --_9e876ef9-e673-486b-befa-abff52e19a99_-- > >> > >> --===============7278135725256123965== > >> Content-Type: text/plain; charset="us-ascii" > >> MIME-Version: 1.0 > >> Content-Transfer-Encoding: 7bit > >> Content-Disposition: inline > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> --===============7278135725256123965==-- > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:19:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:19:27 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, Message-ID: The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 06:23:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 04:23:05 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? Message-ID: I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 07:02:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 05:02:57 +0000 Subject: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? In-Reply-To: References: , <06A5766A-C8F4-4143-9F1C-736738D8FCD4@purdue.edu>, , , Message-ID: Right -- I would have just said, add a name or such parameter to load/store, but I'm aware of LLVM GetElementPointer so came up with a name like it.But, I'm not sure we want a separate call -- GetElementPointer, followed by a load/store of whatever is stacked -- or build it into the load/store. I've always found the presence of the offset parameter in load/store "surprising".I'd'a'naively'thunk that'd a be a separate add operation.But I also suspect the original authors knew what they were doing and had possiblydone it several times already, and this parameter's presence was an easy lesson they learnedfrom experience. Heck, perhaps they had my current desires in mind? Then again, I was also /initially/ surprised by both the low level nature of the IR, all the layout done already, and that I was able to get it to work generating C. But ultimately it makes a lot of sense in the current form. If you look at the "level" and sophistication of M3x86 -- it doesn't do layout. Even M3C doesn't really do layout -- it echoes/renders the information in another form, leaving the C compiler to do the layout. > something that suits LLVM and C And presumably for the time being, gcc and M3x86? :) My actual extremely long term goal that I might never get to is that once we have something very portable -- C. And I guess something everyone likes -- LLVM.Perhaps try to write more of our backends. Like starting with M3x86.m3.I know I'll never achieve the reach of C/gcc/LLVM though. Merely targeting AMD64 across NT/Linux/Mac will be difficult -- three file formats and at least two ABIs. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:31:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Ideally, we would also converge on something that suits both LLVM and C.Your notion of get_member_reference is similar to LLVM getelementptr, which allows the backend to do layout for structs.It would be very nice if the front-end used this level of abstraction.I assume that is what you are talking about. On Aug 10, 2015, at 2:19 PM, Jay K wrote:The more I think about this, the more I want to do it the other way,at least for the C backend, but only after another round of change. For now, I prefer how it is. The other change I want is "get_member_reference", or for load/store to indicatefield names or constant array indices, along with the "resolved" offset. That is, I want the backend to be able to do layout, and I want the generatedC to be independent of word size. Including expressions involvingBITSIZE(INTEGER) to be rendered as #include // or stdint.h (or autoconf) typedef ptrdiff_t INTEGER; // or intptr_t (or autoconf) #define BITSIZE(x) sizeof(x) * CHAR_BIT BITSIZE(INTEGER) instead of the current 32 or 64. Currently I have two pointers per jmpbuf. One in the frame, one in another local variable. Perhaps this can be converged to have just the one in the frame. Independently of any other change. Once we have get_member_reference, then for C thiswould just be a nothing-special struct/record with a jmpbuf value at the end,referenced by name. For the other backends, perhaps we could make it a record,of unspecified size, have the runtime compute the size by addingthe size of the prefix to the size of a jmpbuf and doing alloca. I'd like C and non-C to be ABI compatible though.They have other differences already, in particular around the frame pointerfor nested functions. This bothers me. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu Date: Sun, 9 Aug 2015 16:18:59 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? https://github.com/modula3/cm3/commit/5b04afe9134e5807196ba6af13fa4cd28401f054 jmpbuf in frame changed from value to pointer. - Jay Subject: Re: [M3devel] alloca(jmpbuf) vs. alloca(jmpbuf + frame)? From: hosking at purdue.edu Date: Sun, 9 Aug 2015 20:09:36 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the m3core change you refer to? Sent from my iPad On 9 Aug 2015, at 11:33 am, Jay K wrote: I considered that I could lump the jmpbuf alloca in with its enclosing frame -- one slightly larger alloca,and the same amount of overall stack, minus the pointer or two per jmpbuf in the current scheme. That would preclude the need for m3core change, which I'm sure will cause trouble,but is meant to be an ok thing to do. However, that isn't perhaps as amenable to my next desired step in the C backend wrt jmp_buf.Though, I guess, I could have it both ways -- the compiler and runtime could share a flagand the frame be altered. Thoughts? - Jay_______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 08:55:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 06:55:01 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , Message-ID: Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? > BYTESIZE(INTEGER) I know this is trickier but I consider it almost the same problem. Imagine I have this C code: typedef struct T1 { size_t a,b; } T1;T1* pt1; pt1->b (*size_t*)(((char*)pt1) + sizeof(size_t)) while the second is not idiomatic, and may skirt alignment/padding rules (imagine if a was a char and b a size_t), it is just about as portable. And you can write Modula-3 that way, with LOOPHOLE, ADR, BITESIZE, etc. I think "expressions" or "constant expressions" need to be maintained in "string" or "structural" form and passed to the backend..."string" would be, like, the original Modula-3."structural" would be parse them and make an "expression tree" with nodes like plus, minus, multiplication, division. I think, perhaps, the expression tree would be optional. Backends that ignore them could say so and save that cost. I believe most/all Modula-3 constant expressions render trivially in C. Really, like,RTIO.PutInt(BYTESIZE(INTEGER)); ideally translates to C involving the string/type "INTEGER", and not INT32 or INT64. I have to go. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 14:37:03 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 2:23 PM, Jay K wrote:I want load/store of field names, or of constant array indices, to include a string or M3.ID indicating the field name. A cleaner solution is to use types and request the nth field of the type, etc. That way we don?t need symbolic names for the fields. This way, the C backend can avoid hardcoding the offsets. A step toward target-independent generated C. Ok? Similarly, I want expressions involving BYTESIZE(INTEGER) or BITSIZE(INTEGER) or BYTESIZE(pointer) or BYTESIZE(RECORD or ARRAY involving INTEGER or pointer) etc. to be in a state that the C backend can render them in a target-independent way. Thoughts? That is a bit trickier if not impossible. Modula-3 inherently exposes these as compile-time integer constants. Basically, I want to generate one version of C that works on all targets.Currently we are far from that, and it isn't easy.The jmpbuf change was a small step in that direction. - Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 10 19:34:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 10 Aug 2015 17:34:34 +0000 Subject: [M3devel] "get member reference" /load/store(field name)? In-Reply-To: References: , , , Message-ID: > BYTESIZE(INTEGER) I have other ideas here. 1) Give up. Be like the 3.6 release and have target-specific boot archives, and target specific intermediate C, like current. 2) kinda of compile everything twice. The generated C would be: #ifdef _LP64_ // or such ... #else ... almost but not quite the same .. #endif To cut down cost somewhat, rereading dead code, it could be: foo32.c foo64.c foo.c #ifdef _LP64_ #include "foo64.c" #else #include "foo32.c" #endif 3) Sort of compile as always/only 64bit, when using C backend. Specifically, in m3middle/Target.m3, when using C backend, setup 64bit types. And in M3C output something like: #ifdef _LP64_ /* or such -- not quite*/ #define PAD64(x) /* nothing */ #else #define PAD64(x) int x##pad; #endif RECORD T1 = a: REF INTEGER; END; => struct T1 { INTEGER* a; PAD64(a); /* after any pointer field in a record */ }; This is an efficiency loss on 32bit platforms but maybe ok. It would gain a certain compatibility, i.e. in persistance formats. Local variables not in a record..would possibly be put into such a record, depending on if we could guarantee the entire pointer doesn't get overwritten..consider unusual valid Modula-3 such as: Cstring.memset(ADR(ptr), 0, BYTESIZE(ptr)); so we would have to pad out locals also. Possibly subject to #ifdef _LP64_. Big problem here would be interfacing with actual C code. For example Win32 and X Windows. If not for that, I was thinking this is a very viable approach. Given that, I'm not sure. Maybe as I was saying really kind of preserve all constant expressions symbolically and let the C backend rerender them.including stuff like: a: ARRAY [0.. BYTESIZE(INTEGER)] OF CHAR. or a: ARRAY [0.. BYTESIZE(INTEGER) * 2] OF CHAR. or a: ARRAY [0.. (BYTESIZE(INTEGER) = 32) * 4 + (BYTESIZE(INTEGER) = 64) * 8] OF CHAR. would have to be certain basically what is constant in Modula-3 is constant in C. - Jay Subject: Re: [M3devel] "get member reference" /load/store(field name)? From: hosking at purdue.edu Date: Mon, 10 Aug 2015 17:02:37 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 10, 2015, at 4:55 PM, Jay K wrote:Interesting. I hadn't thought of that. We do "need" symbolic names. Well..they could be omitted, but: - Having symbolic names is great for debugging, vs. making up names. The names are in the type. - This assumes between C and Modula-3 the rules for the naming are close enough. I already handle things like if you have a record field named "int", I change it. For every reserved word as far as I knew (something I needs to be checked against K&R, ANSI, and C++latest). This idea of "nth" then unifies constant array indices with record fields? Yes. I would have likely had two parameters, a string/id and an integer.Or two different functions. Oh..I guess this isn't to avoid the strings, but pass them when defining the type,and not repeatedly for access? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 11 00:13:00 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 10 Aug 2015 17:13:00 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , <20150809165841.BA38C1A2062@async.async.caltech.edu>, <55C7A6A8.2050805@lcwb.coop> Message-ID: <55C921EC.7070303@lcwb.coop> On 08/09/2015 02:20 PM, Jay K wrote: > Right, agreed -- this should work for LLVM, and the C compiler. > LLVM should be multi-threadable in-proc too. > > Fixing m3front itself -- we'd have to identify and deal with all the globals. > For the alloca/setjmp work I put the "globals" in Module.T. > There are also sequence constraints on order of compilation that the back ends don't have. At a minimum, a unit can't be front-ended (yes, I am told that any word can be verbed, although I try to resist unless it has a really useful conciseness benefit. ;-|) until all its imported and exported interfaces have been front-ended, particularly, their .M3EXPORTS files created. > - Jay > > > > Date: Sun, 9 Aug 2015 14:14:48 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > > > > > On 08/09/2015 11:58 AM, mika at async.caltech.edu wrote: > > > > > > I think this is a great idea. I made the back-end parallel already (at > > > least for the version of the compiler that calls the gcc-based backend). > > > > I believe this works for any executable(s) separate from cm3 itself. > > > > > To enable that you have to set "M3_PARALLEL_BACK" to the number of threads > > > you want to spawn in your cm3.cfg . > > > > > > Mika > > > ..... -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Tue Aug 11 06:01:50 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:50 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 06:01:51 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 21:01:51 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: The most speedup you'll get is where there's least dependencies (imports or exports), with zero being the optimal number (a "root" file). If there are more than zero dependencies for a given file then the total time it will take to finish the file will be its own compile time plus the total time of its slowest dependency, that is, the sum of the compile times of the slowest compiling files between it and some root file. The slowest of those is the "critical path." Of course that assumes that you have an infinite number of processors. Because you don't you then have to ensure that the critical path gets priority in processing, since delaying it for any other work would be counterproductive. Realistically the critical path and that scheduling can only be guessed before the first compile, unless file size is an accurate predictor of compile time, or some sort of prior analysis is done. The critical path represents the serial portion of the compilation so if you figure out that as the percentage of the total compilation you can get an idea of the maximum possible speedup using Amdahl's law. Here is a rather depressing graph that illustrates it: https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png Also while compiling the entire repository in parallel may yield a significant speedup, most people compile incrementally, involving relatively few files. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 07:00:17 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 10 Aug 2015 22:00:17 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: A more fruitful approach might be pipelining the compiler: - files enter the pipeline in reverse dependency order - have a thread (stage) to read those files into memory - have a thread to tokenize and parse the files and form the data structure - have a thread to do intermediary processing - have a thread to generate the output representation for the back end You'll only get a maximum 4x speedup and you'll only be as fast as the slowest thread, but you can reliably tune the divisions based on simple benchmarking of each thread. You're very likely to get somewhat close to the 4x speedup. This works best when there is a 1:1 between pipeline stages and cores. If you were ambitious you could attempt an 8 stage pipeline for 8 core processors. On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > Is anyone interested in updating m3front to be multi-threaded? > > I haven't seen a single core system in a while. > > Surely each module can be compiled separately, possibly with some > serialization around compiling interfaces? > > Thanks, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Aug 11 09:13:25 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 11 Aug 2015 09:13:25 +0200 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: Message-ID: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Aug 11 12:47:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , Message-ID: I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 13:44:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , , Message-ID: Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 19:20:26 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 10:20:26 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 20:46:35 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 14:46:35 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811184635.GA3909@topoi.pooq.com> On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. I'm even pleased how *fast* the compiler is. If I could make it generate portable C, it would satisy most of the uses I can imagine for it, not just the uses I actually have. -- hendrik From jay.krell at cornell.edu Tue Aug 11 20:56:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 18:56:03 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, Message-ID: The problem with cross compiling is you need: a working cross-compiler for C/C++ a working cross-assembler a working cross-linker If you have those, it is fairly easy. * Getting those is often not easy. Esp. the C/C+ compiler as you need headers/libraries/startup code. "sysroot" or "buildroot" it is often called. * fairly easy: for NT386 -- nothing extra to do, the backend is in cm3 There are endianness bugs in mklib, but it works on the majority of systems -- little endian hosts. I can fix it. For the C backend -- nothing extra to do, the backend is in cm3. For the gcc-based backend -- fairly easy -- you need to rebuild a different cm3cg and point the config at it. It kind of is already setup to work. For the LLVM backend, probably also easy. I think on Debian and Gentoo, cross C compilers at least to every Linux architecture are readily available. But I don't know about otherwise. In as much as the compiler is gcc, the assembler GNU as, and the linker GNU ld, that gets you far, but you still need the headers/libraries/startup code. If your C compiler is LLVM, that might also help for compiler/assembler/linker. But again, the headers/libraries/startup code. There are various systems out there that try to automate this but I don't have much experience with them, and they often are slightly different than what we want. Otherwise, what we have setup is we cross compile to assembly files, copy them to the target, which typically does have the compiler/assembler/linker, and finish there. What remains there is that I only have this automated to build cm3. It should also at least also build m3core and libm3 statically. And therefore reach approximately parity with the 3.6 "boot" archives and install process. Has anyone here installed 3.6? And consider there method a decent goal and stopping point? Today is slightly different since quake has been rewritten from C to Modula-3 in the 4.0 timeframe. And *possibly* boot archives should contain everything -- going beyond what 3.6 did. This is just a matter of work in scripts/python/pylib.py. Copying stuff into sub directories and generating a hierarhical recursive or not make system in there. My make skills aren't great, and I've been hung up on details like depending on GNU make or trying to be more portable. On the threading area, I believe there is a simple almost ideal design. - parse and mostly "execute" all the quake code almost unchanged, single thread - difference starts at about the last line of quake where it says library or program - possibly though queueing every file to a "prefetcher" thread, it just reads every file into a reused buffer and throws out the data, populating the operating system's file system cache; or possibly mmaping every file and keeping it mapped, and touching every page; doing this in a somewhat thread aware/safe way for the upcoming actual accesses - once quake is done, one of two choices: simple but not optimal: do a single threaded parse of every interface less simple: parsing here can be in parallel, depending on the dependency graph; you'd just start up a thread per interface, but block on parsing its dependencies; as you get away from the root (RT0.i3), the tree should get ever wider You would either manually throttle the number of threads, or rely on an underlying threadpool. NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be simplar. Maybe that wasn't good. Simpler as in, including, thread id is now just the underlying Win32 thread id. It wasn't before. Win32 has had a thread pool since circa Windows 2000. That might be profitable to use. - once parsing interfaces is done, I believe codegen of every interface, and parsing and codegen of every module can proceed with arbitrary parallelism Fetching interface/module contents might synchronize with the prefetcher depending on which of the two approaches -- if the prefetcher is just prefetch and discard, populating the file system cache, then later compilation just refetches oblibviously/unchanged. If the prefetcher thread mmaps and keeps everything, then serialize with it. Also, you might have multiple prefetcher threads. What you really want is..difficult. You want a prefetcher thread per spindle. How to count them up? If you have an SSD, then prefetching might not buy much and just forget about it. The thing is, if you have spindles, this might be the most gain, and if you have an SSD, the cost might be negligible or slightly beneficial. I/O is so often the bottleneck, at least in the days of rotating storage. In the event that the file system cache is small, or the source for a package really large, prefetching can be counterproductive -- moving stuff through the cache only to flush it before it is used and fetching it a second time. I expect on most systems this is not a concern though. One of the pieces of work here, I believe, is to move most globals in m3front to be in Module.T. This would actually make things cleaner imho. Yes, it consolidates knowledge that you might want distributed. And accessing context'ed data is slightly slower than globals. But imho globals are bad and everything should be placed in *some* context other than the process or thread. I would not advocate compiling functions within a module in parallel, only modules. Similarly, code generators must have no globals, and put all the state in the cg object. - Jay Date: Tue, 11 Aug 2015 10:20:26 -0700 From: lists at darko.org To: wagner at elegosoft.com CC: m3devel at elegosoft.com; jay.krell at cornell.edu Subject: Re: [M3devel] multi-threaded m3front? I couldn't agree with you more. I think being able to compile or cross compile the system without spending hours (or days) hacking scripts/environments would be a huge step forward for the project. Or having compiler binaries for more than two platforms (four if you count different word sizes). Meanwhile I've never heard anyone complain that the compiler is too slow. But of course everyone has different priorities, different interests and different opinions in a volunteer project so any discussion on this subject will inevitably boil down to "he who writes the code determines the priorities." On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner wrote: On Mon, 10 Aug 2015 22:00:17 -0700 Darko Volaric wrote: > A more fruitful approach might be pipelining the compiler: > > - files enter the pipeline in reverse dependency order > - have a thread (stage) to read those files into memory > - have a thread to tokenize and parse the files and form the data structure > - have a thread to do intermediary processing > - have a thread to generate the output representation for the back end > > You'll only get a maximum 4x speedup and you'll only be as fast as the > slowest thread, but you can reliably tune the divisions based on simple > benchmarking of each thread. You're very likely to get somewhat close to > the 4x speedup. This works best when there is a 1:1 between pipeline stages > and cores. If you were ambitious you could attempt an 8 stage pipeline for > 8 core processors. > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > Is anyone interested in updating m3front to be multi-threaded? > > > > I haven't seen a single core system in a while. > > > > Surely each module can be compiled separately, possibly with some > > serialization around compiling interfaces? While this is all correct, I'd like to remark that in my expecience optimizing for performance has always got in the way of clear, understandable and maintainable code. So while there are semantic refactorings and feature additions in the pipeline I would not like to see anybody rework the whole compiler with the intention of making it faster. I would rather see the few active programmers interested in M3 concentrate on more backends, better intermediate code, better code optimization, better maintainability etc. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 11 21:04:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 19:04:23 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811184635.GA3909@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: There are clearly three speeds to the compiler in my informal observation. The NT386 backend is fastest. The m3cg backend is medium. The C backed is a little slower. But if this can all be divided by number of cores, and the I/O overlapped,they might all be fast. :) I would like to write more integrated backends, but the C backend needs a bit more work,and we want to remove setjmp/longjmp for most targets still,and the existing M3x86.m3 has a terrible debugging story -- only line numbers. - Jay > Date: Tue, 11 Aug 2015 14:46:35 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > I couldn't agree with you more. I think being able to compile or cross > > compile the system without spending hours (or days) hacking > > scripts/environments would be a huge step forward for the project. Or > > having compiler binaries for more than two platforms (four if you count > > different word sizes). Meanwhile I've never heard anyone complain that the > > compiler is too slow. > > I'm even pleased how *fast* the compiler is. > > If I could make it generate portable C, it would satisy most of the > uses I can imagine for it, not just the uses I actually have. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 11 21:32:37 2015 From: lists at darko.org (Darko Volaric) Date: Tue, 11 Aug 2015 12:32:37 -0700 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: Jay I can see the complexity of the task but can we have an option for users who cannot invest time in this? What would be great is if you could set up the scripts in a static environment available to all that builds various platforms and back ends that are requested or that you feel like maintaining. Having it so it could auto build based on a particular version from github would be optimal. Surely this would help with developing and debugging your scripts and backend and would solve the problem of other people getting the environment right. I have dozens of servers (x86 and ARM) with ample memory and storage plus PDUs that allow powering the servers up and down over the net. If you want to set up the environments and scripts I can setup the access and install Mac OS and whatever Unixes on some of them. You can have carte blanche as to what you do with them, I don't care as long as working compilers pop out somewhere. Maybe others are interested in helping with this too. On Tue, Aug 11, 2015 at 11:56 AM, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ > a working cross-assembler > a working cross-linker > > > If you have those, it is fairly easy. * > Getting those is often not easy. > Esp. the C/C+ compiler as you need headers/libraries/startup code. > "sysroot" or "buildroot" > it is often called. > > > * fairly easy: > for NT386 -- nothing extra to do, the backend is in cm3 > There are endianness bugs in mklib, but it works on the majority of > systems -- little endian hosts. I can fix it. > > For the C backend -- nothing extra to do, the backend is in cm3. > > For the gcc-based backend -- fairly easy -- you need to rebuild a > different cm3cg > and point the config at it. It kind of is already setup to work. > > > For the LLVM backend, probably also easy. > > > I think on Debian and Gentoo, cross C compilers at least to every Linux > architecture are readily available. > But I don't know about otherwise. > > > In as much as the compiler is gcc, the assembler GNU as, and the linker > GNU ld, that gets you far, > but you still need the headers/libraries/startup code. > > > If your C compiler is LLVM, that might also help for > compiler/assembler/linker. > But again, the headers/libraries/startup code. > > > There are various systems out there that try to automate this but I > don't have much experience > with them, and they often are slightly different than what we want. > > > Otherwise, what we have setup is we cross compile to assembly files, copy > them to the target, > which typically does have the compiler/assembler/linker, and finish there. > > > What remains there is that I only have this automated to build cm3. > It should also at least also build m3core and libm3 statically. And > therefore reach > approximately parity with the 3.6 "boot" archives and install process. > > > Has anyone here installed 3.6? And consider there method a decent goal > and stopping point? > Today is slightly different since quake has been rewritten from C to > Modula-3 in the 4.0 timeframe. > > > And *possibly* boot archives should contain everything -- going beyond > what 3.6 did. > > > > This is just a matter of work in scripts/python/pylib.py. > Copying stuff into sub directories and generating a hierarhical recursive > or not make system in there. > My make skills aren't great, and I've been hung up on details like > depending on GNU make or trying to > be more portable. > > > On the threading area, I believe there is a simple almost ideal design. > > - parse and mostly "execute" all the quake code almost unchanged, single > thread > > - difference starts at about the last line of quake where it says library > or program > > - possibly though queueing every file to a "prefetcher" thread, it just > reads > every file into a reused buffer and throws out the data, populating the > operating > system's file system cache; or possibly mmaping every file and keeping > it mapped, > and touching every page; doing this in a somewhat thread aware/safe way > for the > upcoming actual accesses > > - once quake is done, one of two choices: > > simple but not optimal: do a single threaded parse of every interface > less simple: parsing here can be in parallel, depending on the dependency > graph; > you'd just start up a thread per interface, but block on parsing its > dependencies; > as you get away from the root (RT0.i3), the tree should get ever wider > > > You would either manually throttle the number of threads, or rely on an > underlying threadpool. > > > NOTE: Modula-3 used to have a thread pool on Win32 but I removed it to be > simplar. > Maybe that wasn't good. Simpler as in, including, thread id is now just > the underlying > Win32 thread id. It wasn't before. > > Win32 has had a thread pool since circa Windows 2000. That might be > profitable to use. > > - once parsing interfaces is done, I believe codegen of every interface, > and parsing and codegen > of every module can proceed with arbitrary parallelism > > > Fetching interface/module contents might synchronize with the prefetcher > depending > on which of the two approaches -- if the prefetcher is just prefetch and > discard, > populating the file system cache, then later compilation just refetches > oblibviously/unchanged. > If the prefetcher thread mmaps and keeps everything, then serialize with > it. > > > Also, you might have multiple prefetcher threads. What you really want > is..difficult. > You want a prefetcher thread per spindle. How to count them up? > If you have an SSD, then prefetching might not buy much and just forget > about it. > The thing is, if you have spindles, this might be the most gain, and if > you have an SSD, > the cost might be negligible or slightly beneficial. > > > I/O is so often the bottleneck, at least in the days of rotating storage. > > > In the event that the file system cache is small, or the source for a > package > really large, prefetching can be counterproductive -- moving stuff > through the cache > only to flush it before it is used and fetching it a second time. > > > I expect on most systems this is not a concern though. > > > One of the pieces of work here, I believe, is to move most globals in > m3front > to be in Module.T. This would actually make things cleaner imho. > Yes, it consolidates knowledge that you might want distributed. > And accessing context'ed data is slightly slower than globals. > But imho globals are bad and everything should be placed in *some* context > other than the process or thread. > > > I would not advocate compiling functions within a module in parallel, > only modules. > > > Similarly, code generators must have no globals, and put all the state in > the cg object. > > > - Jay > > > > > ------------------------------ > Date: Tue, 11 Aug 2015 10:20:26 -0700 > From: lists at darko.org > To: wagner at elegosoft.com > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] multi-threaded m3front? > > I couldn't agree with you more. I think being able to compile or cross > compile the system without spending hours (or days) hacking > scripts/environments would be a huge step forward for the project. Or > having compiler binaries for more than two platforms (four if you count > different word sizes). Meanwhile I've never heard anyone complain that the > compiler is too slow. > > But of course everyone has different priorities, different interests and > different opinions in a volunteer project so any discussion on this subject > will inevitably boil down to "he who writes the code determines the > priorities." > > > > > On Tue, Aug 11, 2015 at 12:13 AM, Olaf Wagner > wrote: > > On Mon, 10 Aug 2015 22:00:17 -0700 > Darko Volaric wrote: > > > A more fruitful approach might be pipelining the compiler: > > > > - files enter the pipeline in reverse dependency order > > - have a thread (stage) to read those files into memory > > - have a thread to tokenize and parse the files and form the data > structure > > - have a thread to do intermediary processing > > - have a thread to generate the output representation for the back end > > > > You'll only get a maximum 4x speedup and you'll only be as fast as the > > slowest thread, but you can reliably tune the divisions based on simple > > benchmarking of each thread. You're very likely to get somewhat close to > > the 4x speedup. This works best when there is a 1:1 between pipeline > stages > > and cores. If you were ambitious you could attempt an 8 stage pipeline > for > > 8 core processors. > > > > > > On Sat, Aug 8, 2015 at 8:05 PM, Jay K wrote: > > > > > Is anyone interested in updating m3front to be multi-threaded? > > > > > > I haven't seen a single core system in a while. > > > > > > Surely each module can be compiled separately, possibly with some > > > serialization around compiling interfaces? > > While this is all correct, I'd like to remark that in my expecience > optimizing for performance has always got in the way of clear, > understandable and maintainable code. > > So while there are semantic refactorings and feature additions in the > pipeline I would not like to see anybody rework the whole compiler > with the intention of making it faster. I would rather see the few > active programmers interested in M3 concentrate on more backends, better > intermediate code, better code optimization, better maintainability > etc. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 11 22:57:21 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 11 Aug 2015 16:57:21 -0400 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com> Message-ID: <20150811205721.GA9712@topoi.pooq.com> On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > The problem with cross compiling is you need: > a working cross-compiler for C/C++ Slightly clumsier is just haveing a C/C++ compiler on the target system. But it's often easier to get that than a cross-compiler. -- hendrik From rodney_bates at lcwb.coop Tue Aug 11 23:01:40 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 11 Aug 2015 16:01:40 -0500 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , <20150811184635.GA3909@topoi.pooq.com> Message-ID: <55CA62B4.5000205@lcwb.coop> In my informal observation, it is additionally true that: On 08/11/2015 02:04 PM, Jay K wrote: > There are clearly three speeds to the compiler in my informal observation. > > The NT386 backend is fastest. > The m3cg backend is medium. > The C backed is a little slower. > gcc compiling m3cc is quite a bit slower than cm3 with m3cc backend. gcc compiling llvm/clang is a lot slower than that. clang compiling individual C++ units is noticeably slower than gcc on the same units. All of which is leading to my feeling that compiling Modula-3 code is already so much faster than compiling C or C++, that it's a pretty minor need to speed it up even more. > > But if this can all be divided by number of cores, and the I/O overlapped, > they might all be fast. :) > > > I would like to write more integrated backends, but the C backend needs a bit more work, > and we want to remove setjmp/longjmp for most targets still, > and the existing M3x86.m3 has a terrible debugging story -- only line numbers. > > > - Jay > > > > > Date: Tue, 11 Aug 2015 14:46:35 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] multi-threaded m3front? > > > > On Tue, Aug 11, 2015 at 10:20:26AM -0700, Darko Volaric wrote: > > > I couldn't agree with you more. I think being able to compile or cross > > > compile the system without spending hours (or days) hacking > > > scripts/environments would be a huge step forward for the project. Or > > > having compiler binaries for more than two platforms (four if you count > > > different word sizes). Meanwhile I've never heard anyone complain that the > > > compiler is too slow. > > > > I'm even pleased how *fast* the compiler is. > > > > If I could make it generate portable C, it would satisy most of the > > uses I can imagine for it, not just the uses I actually have. > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 12 00:33:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: <20150811205721.GA9712@topoi.pooq.com> References: , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , <20150811205721.GA9712@topoi.pooq.com> Message-ID: Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:37:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:37:10 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , , , Message-ID: It should be ok now. Have you been using Modula-3 on it? How?I can imagine a preexisting path of low resistance was to use gcc everywhere and reject Apple's tools.gcc as the "assembler driver" would have worked as it'd pick the working assembler instead of the LLVM assembler.As well as using (/usr/bin/)as directly, which is what I do now (it actually calls out to yet another as, but I didn'twant to hardcode that path -- /applications/xcode and all that .. which reminds me I need to uninstall xcodeand see what things do -- as there is an option to install command line tools w/o xcode).. But Apple's tools should work now. - Jay > Subject: Re: [M3devel] Mac 10.10? > From: hosking at purdue.edu > Date: Wed, 12 Aug 2015 07:49:58 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I am and I suspect it'll be a popular target! > > Sent from my iPad > > > On 11 Aug 2015, at 9:44 pm, Jay K wrote: > > > > Anyone else running 10.10? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 00:44:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 11 Aug 2015 22:44:19 +0000 Subject: [M3devel] multi-threaded m3front? In-Reply-To: References: , , , , <20150811091325.8ff0f34d4191abab26782c63@elegosoft.com>, , , , , , <20150811205721.GA9712@topoi.pooq.com>, Message-ID: I suspect using Debian or Ubuntu is the quickest route to a suite of cross tools. https://wiki.debian.org/CrossToolchains at least to Linux, not necessarily to BSD/NT/Solaris/Mac. We could add "support" to that -- having the default config noticethat host = is debian or ubuntu and host != target and target is linux and invoke those toolsand tell you to apt-get install whatever if they aren't present.. But I still think the "fair" / "flat" / "consistent" story is to make boot archives something like 3.6and user then completes the build on the target with native tools. We'd distribute like three archives per target: boot -- no binaries, all assembly/C source, to just cm3/libm3/m3core (not just cm3 like current) binmin -- binaries -- cm3, cm3cg (except for NT386), m3core (maybe just static), libm3 (ditto) binall -- all binaries, nothing to build The C source might be somewhat target-independent, but getting it to be fully target-independentI can't yet commit to. binmin has a pretty narrow use case, maybe strike it - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 11 Aug 2015 22:33:22 +0000 Subject: Re: [M3devel] multi-threaded m3front? Hendrik: exactly My longer blurbe from earlier. > https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/buildmany.sh builds cm3cg for every target. Seems like overkill, but which ones? Do we make a bunch of single file downloads? Notice how buildmany.sh is trivial. m3cc/src/m3makefile does the "real" work, and it isn't complicated. And, btw, I didn't write it. It's been there "forever". https://github.com/modula3/cm3/blob/master/m3-sys/cminstall/src/config-no-install/cm3cfg.common look for the lines: "for cross builds" knows about the *unshipped* structure that https://github.com/modula3/cm3/blob/master/m3-sys/m3cc/src/m3makefile creates for cross cm3cg -- instead of host-target/cm3cg. We could consider: always buildmany (all) and ship them and adapt cm3cg.common to look in the shipped location instead, like bin/target/cm3cg. But is this useful though? i.e. w/o the cross-c-compiler/linker/assembler? There are larger problems. "Many things are configurable". More than people often consider. "Linux" could be using glibc, or eglibc, or newlib, etc. For the most part, they are probably ABI-compatible. But are they completely? Is struct stat always the same? Do all the libcs even have pthreads? (I doubt lacking pthreads is a useful system though.) This is why, btw, I had proposed, like m3core/src/thread/linux. to replace, on Linux, m3core/src/PTHREAD, to skip through the varying libc's and go to the less varying kernel. The futexes and clone and all that. Maybe research the packages that Debian and Gentoo have? And just recommend whatever apt-get, and invoke gcc/ld/as by the names we know they use? They likely have mingw in there, so you can cross from Linux to NT. I believe Debian has a general story btw, of using cross building compilers to bootstrap, but then giving up on cross building otherwise. Too many autoconf's depend on being native builds. And using qemu if it is faster than the native hardware. Really, I think the problem is cross compilation of C. Solve/automate that, and finishing the Modula-3 work shouldn't be difficult. You can go the other way, but I don't want to: - eliminate all C (undoing everything I did in m3core/src/unix) - write all integrated backends - write our own linker - including writing our own startup code and pthreads (the futex/clone proposal) But if you look into this, esp. the startup code and syscall stubs, you realize it is a very large taskfor very little payoff. - Jay > Date: Tue, 11 Aug 2015 16:57:21 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] multi-threaded m3front? > > On Tue, Aug 11, 2015 at 06:56:03PM +0000, Jay K wrote: > > The problem with cross compiling is you need: > > a working cross-compiler for C/C++ > > Slightly clumsier is just haveing a C/C++ compiler on the target system. > But it's often easier to get that than a cross-compiler. > > -- hendrik > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 08:14:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 06:14:02 +0000 Subject: [M3devel] cm3cg coverage/profiling features? Message-ID: Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elego.de Wed Aug 12 08:40:40 2015 From: wagner at elego.de (Olaf Wagner) Date: Wed, 12 Aug 2015 08:40:40 +0200 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812084040.665db643b5756d863fb2eb2a@elego.de> On Wed, 12 Aug 2015 06:14:02 +0000 Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. I'm not sure what exactly you're going to remove, but keeping coverage and profiling support would be great. Though it's been years since I used them... Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From hendrik at topoi.pooq.com Wed Aug 12 14:09:19 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 12 Aug 2015 08:09:19 -0400 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812120918.GA578@topoi.pooq.com> On Wed, Aug 12, 2015 at 06:14:02AM +0000, Jay K wrote: > Does anyone use the coverage or profiling or optimization-via-profiling-feedback features of cm3cg?I'm removing dead stuff. > Thank you, - Jay Didn't know they were there. How to use them? -- hendrik From lists at darko.org Wed Aug 12 17:17:54 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 08:17:54 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: Why does it have to be removed? Is there some pressing reason that justifies removing functionality? How does it improve the compiler? Also, how does asking in the mailing list justify its removal? Not all users follow the mailing list, and future users do not get a say. On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > Does anyone use the coverage or profiling or > optimization-via-profiling-feedback features of cm3cg? > I'm removing dead stuff. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Aug 12 17:21:55 2015 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Wed, 12 Aug 2015 08:21:55 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: Message-ID: <20150812152155.6D9891A2062@async.async.caltech.edu> Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- From jay.krell at cornell.edu Wed Aug 12 17:33:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 15:33:24 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: , , <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: gcc is so big, I like to trim it down. For example, I removed the C preprocessor, libgcc (adding back a subsetted alternative when using Sun CC, in another place), the driver, most of the frontends, LTO which I assume nobody uses, all the dependency on gmp/mpfr/mpc, lots of "builtins" that we don't expose so you can't use in Modula-3, etc. This is stuff in gcc, not the stuff in cm3/m3front. But arguably we could (have) left it all alone, just apply a minimal patch. Note that anything here is going to be somewhat specific to this backend.Though LLVM may very well have similar functionality.The integrated backend does not -- though for coverage there exists post-link instrumentation options at least on Windows in Visual Studio.The C backend could get it via the underlying C compiler, e.g. gcc. Someone should try them out and report.The config files have long offered profile/-pg.I'll leave it for now. Thanks, - Jay > To: lists at darko.org > Date: Wed, 12 Aug 2015 08:21:55 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing rea= > >son that justifies removing functionality? How does it improve the compiler= > >?

Also, how does asking in the mailing list justify its = > >removal? Not all users follow the mailing list, and future users do not get= > > a say.

>">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= > >t; wrote:
> .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= > >uff.

Thank you,
=C2=A0- Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > >=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 18:11:10 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 09:11:10 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <20150812152155.6D9891A2062@async.async.caltech.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 19:41:26 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 17:41:26 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, Message-ID: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 20:21:42 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 11:21:42 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top; TYPE T <: ROOT; END Top. INTERFACE Pub; IMPORT Top; TYPE TPub = OBJECT [...] END; REVEAL Top.T <: TPub; END Pub. INTERFACE Pvt; IMPORT Top, Pub; TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt; END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 12 20:56:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 12 Aug 2015 18:56:12 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , , Message-ID: > Error recovery is hard but the original authors laboured to implement it and it worked Within a module it is unbounded problem and such labor will only slightly work. That said, I know of no changes within modules or across modules wrt error recovery.We can/should investigate and fix. I don't think a compiler switch is a good idea here, as we should have a smaller matrixof behaviors and the behavior you desire is probably the one and only way it should be. Making switches for too many things is its own madness. Nobody runs the same thingand the test matrix explodes. Across modules we should probably just proceed, just being sure to error at the end (i.e. so we don't ship, and maybe so scripts don't proceed, thoughactually, scripts probably should proceed, but again error at the end -- builds tend to contain highly independent graph pieces, even if thereare some heavily shared nodes at the root like m3core) There are at least two schools of thought in software in general. There is "fail fast" and "muddle along". Generally older thinking is "muddle along", make a best effort attempt to work in the face of errors. Newer thinking is "fail fast", and generally acknowledging that once there is an error, some?many?most?all "bets"are off and best to stop, vs. doing the wrong thing. But there is a middle ground, which is to try to think about dependencies and if errors in one placereally affect another. Often times, for a simple example, iterations of a loop are independent and failurewithin one shouldn't stop the loop. Imagine you specify to copy many files, but a few fail. Should you stop the copy at the first failure?Should you rollback the earlier successful copies? Should you have some level of "transactionality" so that no intermediate state is visible unlessthe entire operation is guanteed to succeed? Anyway, compilers, even within a module, seem popular to place in the middle and muddle along.Their output is eventuall mission critical, but how they handle errors is not.People like to get as many errors as possible, fix them all, try again.Instead of the compiler stopping at the first. You know, to try to divide the quadratic natureof fixing errors down to almost linear. But it depends on the nature of the error. "breaking" a declaration is presumably removing it or changing it to a different valid declaration.Changing it to a declaration that doesn't parse is risky -- you have to assume error recovery works a certain way. I also discovered an incrementality problem -- if you move a module across packages.That is probably not a new regression though. I don't think we are losing functionality and all changes still come through m3commit. Imho review can occur before or after change, but I know, not everyone in the world agrees.Many development environments require review before commit. There is the matter of if rate of change exceeds rate of review and needs to be throttled down,but rate of change is really quite low (even if rate of review is even lower). - Jay Date: Wed, 12 Aug 2015 11:21:42 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? The larger point is that is that changes are being made to the compiler ad-hoc. There seems to be no consideration of the consequence of those changes and how it affects users. These changes are being made on the trunk instead of a branch and it may be hard to untangle them. If we're losing functionality accidently then something is wrong with the development process. On this issue, ignoring cascading errors is perfectly legitimate but so is not ignoring them. I (used to) purposely break a declaration so see the consequences of it throughout the code and to reveal where the declaration was used. It's also very slow having to recompile after fixing a change in each client module to reveal the next place it needs fixing, just because it's in a different file. Error recovery is hard but the original authors laboured to implement it and it worked, it shouldn't be lost. At the very least a compiler switch is in order here. Either way this seems to be a bug because the compiler is also failing to track dependencies in some causes causing files not to be recompiled. For example it seems to be happening somewhere around using this sort of idiom: INTERFACE Top;TYPE T <: ROOT;END Top. INTERFACE Pub;IMPORT Top;TYPE TPub = OBJECT [...] END;REVEAL Top.T <: TPub;END Pub. INTERFACE Pvt;IMPORT Top, Pub;TYPE TPvt = Pub.TPub OBJECT [...] END; REVEAL Top.T <: TPvt;END Pvt. MODULE Top; IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *)REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END;BEGIN END Top. I suspect the problem has something to do with the "injection" of the partial revelations through importing interfaces. On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: I understand your description of the changed behavior but I hadn't noticed either way.Perhaps it was changed by accident and likely it should be unconditionally restored.It depends somewhat. Error recovery isn't always easy.Within a module, recovery is likely problematic.If all interfaces compile successfully, then I believe all modules are independent andfailure in any one module should not stop compilation in another module. If any interfaces fail to compile, then there could be quite a cascade across modules.However the way to handle that is probably to ignore it. Attempt to compile all modules."Summarize" the error at the end.i.e. the overall package build will fail.Developer fixes it, rebuilds, possily only a small amount will rebuild. In terms of the scripts, well, not clear. They are potentially iterating over a lot of code.There is somewhat a dilemna of "interactive or not". If I'm there watching and it stops, I might fix it fast, and resume, to success. If I'm not watching, and the problem is small, it might be profitable to continue,do the most possible, and then when I come back, I'll fix it up. If the problem is large, it matters less. Continuing on might go fast if everything fails.It is a somewhat interesting experiment -- put an error in RT0.i3 and see how long it takesto attempt to compile every module in every package. err..wait..you can't ship with an error.Continuing through packages after an error will build against older packages. Consistency, well..partly it is based on what I notice and what I see.What I see cluttering up where I need to debug and change.True consistency might require knowing all and having infinite time. - Jay Date: Wed, 12 Aug 2015 09:11:10 -0700 From: lists at darko.org To: mika at async.caltech.edu; jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? OK, but can someone explain to me why other, less obscure features have been removed? For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? But more generally: even if the functionality is broken, if it's still there there is a chance someone will fix it, if we remove it we're drawing a line underneath it and ending it. The repository is replete with stuff that isn't used or is broken, so removing this one function seems arbitrary. Should we have some sort of consistency? On Wed, Aug 12, 2015 at 8:21 AM, wrote: Don't forget the as-yet-unborn users! In all seriousness... the compiler includes a lot of cool stuff, and I wish it worked. I used some of these features that Jay is talking about when SRC M3 was all there was, and I remember it did work. I think the profiling (and probably other stuff) has been broken since today's junior high school students were born though. Mika Darko Volaric writes: >--===============9067918515746620203== >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/plain; charset=UTF-8 > >Why does it have to be removed? Is there some pressing reason that >justifies removing functionality? How does it improve the compiler? > >Also, how does asking in the mailing list justify its removal? Not all >users follow the mailing list, and future users do not get a say. > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > >> Does anyone use the coverage or profiling or >> optimization-via-profiling-feedback features of cm3cg? >> I'm removing dead stuff. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > >--bcaec548634a0fec67051d1eb98e >Content-Type: text/html; charset=UTF-8 >Content-Transfer-Encoding: quoted-printable > >
Why does it have to be removed? Is there some pressing rea= >son that justifies removing functionality? How does it improve the compiler= >?

Also, how does asking in the mailing list justify its = >removal? Not all users follow the mailing list, and future users do not get= > a say.

">On Tue, Aug 11, 2015 at 11:14 PM, Jay K <mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu&g= >t; wrote:
.8ex;border-left:1px #ccc solid;padding-left:1ex"> > > >
Does anyone use the coverage or profiling or optimiza= >tion-via-profiling-feedback features of cm3cg?
I'm removing dead st= >uff.

Thank you,
=C2=A0- Jay


<= >/div>
>
_______________________________________________
>M3devel mailing list
>M3devel at elegosoft.com
>=3D"noreferrer" target=3D"_blank">https://mail.elegosoft.com/cgi-bin/mailma= >n/listinfo/m3devel
>

> >--bcaec548634a0fec67051d1eb98e-- > >--===============9067918515746620203== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >--===============9067918515746620203==-- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Aug 12 21:41:43 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 12:41:43 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> Message-ID: The error recovery doesn't matter. For instance I just rename a declaration so it no longer exists. The compiler will then produce an error for each usage of that symbol, saying it's no longer found, and it will report it at least once for every module where it used (I've noticed that the compiler will often only report one error if it is repeated multiple times in a module). I'm not sure the rate of change is that low. I think the best idea would be put changes in branches so each set can be reviewed and tested before being merged. This would also allow people to choose what they want in their build. With the new GitHub repository this should be a lot easier than with CVS. On Wed, Aug 12, 2015 at 11:56 AM, Jay K wrote: > > Error recovery is hard but the original authors laboured to implement > it and it worked > > > Within a module it is unbounded problem and such labor will only slightly > work. > > > That said, I know of no changes within modules or across modules wrt error > recovery. > We can/should investigate and fix. > > I don't think a compiler switch is a good idea here, as we should have a > smaller matrix > of behaviors and the behavior you desire is probably the one and only way > it should be. > > Making switches for too many things is its own madness. Nobody runs the > same thing > and the test matrix explodes. > > > Across modules we should probably just proceed, just being sure to error > at the end (i.e. so we don't ship, and maybe so scripts don't proceed, > though > actually, scripts probably should proceed, but again error at the end -- > builds tend to contain *highly* *independent* graph pieces, even if there > are some heavily shared nodes at the root like m3core) > > > There are at least two schools of thought in software in general. > > There is "fail fast" and "muddle along". > > > Generally older thinking is "muddle along", make a best effort attempt to > work in the face of errors. > > > Newer thinking is "fail fast", and generally acknowledging that once there > is an error, some?many?most?all "bets" > are off and best to stop, vs. doing the wrong thing. > > > But there is a middle ground, which is to try to think about dependencies > and if errors in one place > really affect another. Often times, for a simple example, iterations of a > loop are independent and failure > within one shouldn't stop the loop. > > > Imagine you specify to copy many files, but a few fail. Should you stop > the copy at the first failure? > Should you rollback the earlier successful copies? > > Should you have some level of "transactionality" so that no intermediate > state is visible unless > the entire operation is guanteed to succeed? > > > Anyway, compilers, even within a module, seem popular to place in the > middle and muddle along. > Their output is eventuall mission critical, but how they handle errors is > not. > People like to get as many errors as possible, fix them all, try again. > Instead of the compiler stopping at the first. You know, to try to divide > the quadratic nature > of fixing errors down to almost linear. But it depends on the nature of > the error. > > > "breaking" a declaration is presumably removing it or changing it to a > different valid declaration. > Changing it to a declaration that doesn't parse is risky -- you have to > assume error recovery works a certain way. > > > I also discovered an incrementality problem -- if you move a module across > packages. > That is probably not a new regression though. > > > I don't think we are losing functionality and all changes still come > through m3commit. > > Imho review can occur before or after change, but I know, not everyone in > the world agrees. > Many development environments require review before commit. > > > There is the matter of if rate of change exceeds rate of review and needs > to be throttled down, > but rate of change is *really quite low *(even if rate of review is even > lower). > > > - Jay > > > ------------------------------ > Date: Wed, 12 Aug 2015 11:21:42 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > The larger point is that is that changes are being made to the compiler > ad-hoc. There seems to be no consideration of the consequence of those > changes and how it affects users. These changes are being made on the trunk > instead of a branch and it may be hard to untangle them. If we're losing > functionality accidently then something is wrong with the development > process. > > On this issue, ignoring cascading errors is perfectly legitimate but so is > not ignoring them. I (used to) purposely break a declaration so see the > consequences of it throughout the code and to reveal where the declaration > was used. It's also very slow having to recompile after fixing a change in > each client module to reveal the next place it needs fixing, just because > it's in a different file. > > Error recovery is hard but the original authors laboured to implement it > and it worked, it shouldn't be lost. At the very least a compiler switch is > in order here. > > Either way this seems to be a bug because the compiler is also failing to > track dependencies in some causes causing files not to be recompiled. For > example it seems to be happening somewhere around using this sort of idiom: > > INTERFACE Top; > TYPE T <: ROOT; > END Top. > > INTERFACE Pub; > IMPORT Top; > TYPE TPub = OBJECT [...] END; > REVEAL Top.T <: TPub; > END Pub. > > INTERFACE Pvt; > IMPORT Top, Pub; > TYPE TPvt = Pub.TPub OBJECT [...] END; > REVEAL Top.T <: TPvt; > END Pvt. > > MODULE Top; > IMPORT Pub, Pvt; (* or equally, appearing in the EXPORTS list *) > REVEAL T = Pvt.TPvt BRANDED OBJECT [...] END; > BEGIN > END Top. > > I suspect the problem has something to do with the "injection" of the > partial revelations through importing interfaces. > > On Wed, Aug 12, 2015 at 10:41 AM, Jay K wrote: > > I understand your description of the changed behavior but I hadn't noticed > either way. > Perhaps it was changed by accident and likely it should be unconditionally > restored. > It depends somewhat. Error recovery isn't always easy. > Within a module, recovery is likely problematic. > If all interfaces compile successfully, then I believe all modules are > independent and > failure in any one module should not stop compilation in another module. > > > If any interfaces fail to compile, then there could be quite a cascade > across modules. > However the way to handle that is probably to ignore it. Attempt to > compile all modules. > "Summarize" the error at the end. > i.e. the overall package build will fail. > Developer fixes it, rebuilds, possily only a small amount will rebuild. > > In terms of the scripts, well, not clear. They are potentially iterating > over a lot of code. > There is somewhat a dilemna of "interactive or not". > If I'm there watching and it stops, I might fix it fast, and resume, to > success. > > If I'm not watching, and the problem is small, it might be profitable to > continue, > do the most possible, and then when I come back, I'll fix it up. > > > If the problem is large, it matters less. Continuing on might go fast if > everything fails. > It is a somewhat interesting experiment -- put an error in RT0.i3 and see > how long it takes > to attempt to compile every module in every package. err..wait..you can't > ship with an error. > Continuing through packages after an error will build against older > packages. > > > Consistency, well..partly it is based on what I notice and what I see. > What I see cluttering up where I need to debug and change. > True consistency might require knowing all and having infinite time. > > - Jay > > > > ------------------------------ > Date: Wed, 12 Aug 2015 09:11:10 -0700 > From: lists at darko.org > To: mika at async.caltech.edu; jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3cg coverage/profiling features? > > OK, but can someone explain to me why other, less obscure features have > been removed? > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > But more generally: even if the functionality is broken, if it's still > there there is a chance someone will fix it, if we remove it we're drawing > a line underneath it and ending it. > > The repository is replete with stuff that isn't used or is broken, so > removing this one function seems arbitrary. Should we have some sort of > consistency? > > > On Wed, Aug 12, 2015 at 8:21 AM, wrote: > > Don't forget the as-yet-unborn users! > > In all seriousness... the compiler includes a lot of cool stuff, and I > wish it worked. I used some of these features that Jay is talking about > when SRC M3 was all there was, and I remember it did work. > > I think the profiling (and probably other stuff) has been broken since > today's junior high school students were born though. > > Mika > > Darko Volaric writes: > >--===============9067918515746620203== > >Content-Type: multipart/alternative; boundary=bcaec548634a0fec67051d1eb98e > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/plain; charset=UTF-8 > > > >Why does it have to be removed? Is there some pressing reason that > >justifies removing functionality? How does it improve the compiler? > > > >Also, how does asking in the mailing list justify its removal? Not all > >users follow the mailing list, and future users do not get a say. > > > >On Tue, Aug 11, 2015 at 11:14 PM, Jay K wrote: > > > >> Does anyone use the coverage or profiling or > >> optimization-via-profiling-feedback features of cm3cg? > >> I'm removing dead stuff. > >> > >> Thank you, > >> - Jay > >> > >> > >> > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > >--bcaec548634a0fec67051d1eb98e > >Content-Type: text/html; charset=UTF-8 > >Content-Transfer-Encoding: quoted-printable > > > >
Why does it have to be removed? Is there some pressing > rea= > >son that justifies removing functionality? How does it improve the > compiler= > >?

Also, how does asking in the mailing list justify > its = > >removal? Not all users follow the mailing list, and future users do not > get= > > a say.

class=3D"gmail_quote= > >">On Tue, Aug 11, 2015 at 11:14 PM, Jay K < href=3D"= > >mailto:jay.krell at cornell.edu" target=3D"_blank">jay.krell at cornell.edu > &g= > >t; wrote:
0= > > .8ex;border-left:1px #ccc solid;padding-left:1ex"> > > > > > >
Does anyone use the coverage or profiling or > optimiza= > >tion-via-profiling-feedback features of cm3cg?
I'm removing dead > st= > >uff.

Thank you,
=C2=A0- > Jay


<= > >/div>
> >
_______________________________________________
> >M3devel mailing list
> >M3devel at elegosoft.com
> > rel= > >=3D"noreferrer" target=3D"_blank"> > https://mail.elegosoft.com/cgi-bin/mailma= > >n/listinfo/m3devel
> >

> > > >--bcaec548634a0fec67051d1eb98e-- > > > >--===============9067918515746620203== > >Content-Type: text/plain; charset="us-ascii" > >MIME-Version: 1.0 > >Content-Transfer-Encoding: 7bit > >Content-Disposition: inline > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >--===============9067918515746620203==-- > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:27:36 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:27:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: > I wasn?t aware this had changed. > When was this regression introduced? > > On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: > > For instance, cm3 used to keep on compiling files after errors were found. > Now it seems to stop after one module. That loss of functionality seriously > reduces productivity. I couldn't find any switch to reverse the change. > When was this change decided? Is there a way to restore it? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 02:29:03 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 17:29:03 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > I'm using: > > Critical Mass Modula-3 version d5.9.0 > > last updated: 2010-07-21 > > compiled: 2014-08-27 19:15:04 > > configuration: /usr/local/cm3/bin/cm3.cfg > > host: AMD64_DARWIN > > target: AMD64_DARWIN > > > > On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: > >> I wasn?t aware this had changed. >> When was this regression introduced? >> >> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >> >> For instance, cm3 used to keep on compiling files after errors were >> found. Now it seems to stop after one module. That loss of functionality >> seriously reduces productivity. I couldn't find any switch to reverse the >> change. When was this change decided? Is there a way to restore it? >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Thu Aug 13 03:00:22 2015 From: peter.mckinna at gmail.com (Peter McKinna) Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do. Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: > It's the one from the Releases section on GitHub. > > On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: > >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >> wrote: >> >>> I wasn?t aware this had changed. >>> When was this regression introduced? >>> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >>> >>> >>> >> > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 04:05:17 2015 From: lists at darko.org (Darko Volaric) Date: Wed, 12 Aug 2015 19:05:17 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> Message-ID: Sorry, I was distracted by a phone call before - that is the compiler that exhibits the behaviour. Previous to that I was using something that was compiled in 2010, so not very helpful for pinpointing. On 8/12/15, Antony Hosking wrote: > Are you saying that the behavior was still working as of d5.9.0? > Or is that the version you are currently using in which it no longer does > what you want? > We should be able to figure out what commit broke things if we can pinpoint > a time when it worked. > >> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >> >> I'm using: >> >> Critical Mass Modula-3 version d5.9.0 >> >> last updated: 2010-07-21 >> >> compiled: 2014-08-27 19:15:04 >> >> configuration: /usr/local/cm3/bin/cm3.cfg >> >> host: AMD64_DARWIN >> >> target: AMD64_DARWIN >> >> >> >> >> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > > wrote: >> I wasn?t aware this had changed. >> When was this regression introduced? >> >>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >> > wrote: >>> >>> For instance, cm3 used to keep on compiling files after errors were >>> found. Now it seems to stop after one module. That loss of functionality >>> seriously reduces productivity. I couldn't find any switch to reverse the >>> change. When was this change decided? Is there a way to restore it? >> >> > > From jay.krell at cornell.edu Thu Aug 13 05:43:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 12 Aug 2015 20:43:36 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> Message-ID: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? > >> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >> >> Sorry, I was distracted by a phone call before - that is the compiler >> that exhibits the behaviour. Previous to that I was using something >> that was compiled in 2010, so not very helpful for pinpointing. >> >> On 8/12/15, Antony Hosking wrote: >>> Are you saying that the behavior was still working as of d5.9.0? >>> Or is that the version you are currently using in which it no longer does >>> what you want? >>> We should be able to figure out what commit broke things if we can pinpoint >>> a time when it worked. >>> >>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>> >>>> I'm using: >>>> >>>> Critical Mass Modula-3 version d5.9.0 >>>> >>>> last updated: 2010-07-21 >>>> >>>> compiled: 2014-08-27 19:15:04 >>>> >>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>> >>>> host: AMD64_DARWIN >>>> >>>> target: AMD64_DARWIN >>>> >>>> >>>> >>>> >>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>> > wrote: >>>> I wasn?t aware this had changed. >>>> When was this regression introduced? >>>> >>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>> > wrote: >>>>> >>>>> For instance, cm3 used to keep on compiling files after errors were >>>>> found. Now it seems to stop after one module. That loss of functionality >>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>> change. When was this change decided? Is there a way to restore it? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 10:50:57 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 08:50:57 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , Message-ID: Note that this probably isn't the coverage/profiling I was talking about.I was talking about specifically in the gcc backend. I wonder if coverage -- a boolean per basic block; or maybe a counter,is cheap enough that we can't have it always enabled. And then some way of exposing it. I don't like having to recompile/relink everything multiple differentways to get all the features. - Jay From: peter.mckinna at gmail.com Date: Thu, 13 Aug 2015 11:00:22 +1000 Subject: Re: [M3devel] cm3cg coverage/profiling features? To: lists at darko.org CC: hosking at purdue.edu; m3devel at elegosoft.com; jay.krell at cornell.edu Hi, Thought I better upload my coverage fixes I've had sitting around for awhile. So they are in git if anyone cares to (ab)use them. I modified report_coverage to use signals as I couldnt get the old code to work. I'm not sure it will work on all platforms or indeed on 32 bit machines so if anyone with more expertise wants to fix it please do.Just to refresh the usage if you want coverage analysis compile with -Z then when you run your program you get a default coverage.out file which you run analyze_coverage -L for line number coverage (or -P for procedure coverage) see the analyze_coverage.h file for options . Should probably rewrite this stuff in m3. Regards Peter On Thu, Aug 13, 2015 at 10:29 AM, Darko Volaric wrote: It's the one from the Releases section on GitHub. On Wed, Aug 12, 2015 at 5:27 PM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking wrote: I wasn?t aware this had changed.When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 11:23:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 09:23:46 +0000 Subject: [M3devel] Mac 10.10? In-Reply-To: References: <55c64f262e181_45bc3fc0634072bc636e3@hookshot-fe6-cp1-prd.iad.github.net.mail>, , , , ,,, , , , , , Message-ID: Eh, seems like some off by one in my testing -- things work now, including X apps. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 11:44:33 +0000 Subject: Re: [M3devel] Mac 10.10? Darn, there might still be problems -- i.e. running stubgen.Though again, the C backend works to build the entire system.And I think self-built gcc 5.2.0 helped.I'd rather not depend on that though. I'm increasingly leary of the weak linkages among compiler/assembler/linker. Interesting note, if you try to build stock gcc 4.7.4 on 10.10.4 it crashes pretty early. I'll keep poking around but it could be a while.Anyone else running 10.10? cm3cg is noticeably faster to run than the C backend, even on new machine.Darn, I was hoping modern hardware would make it look better. :( - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 11 Aug 2015 10:47:57 +0000 Subject: Re: [M3devel] Mac 10.10? I figured out a big problem in 10.10.4 using the gcc backend.Probably the only problem. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67183 https://llvm.org/bugs/show_bug.cgi?id=24428 I had previously hacked cm3cg to always be 10.4-compatible.I don't like the tools to probe their host and tailor theiroutput to it, lest the output or the tools meant to be copiedto an older/newer host. The bug is demonstrated clearly in my bug reports. The bug is this: 32bit + 10.4 compatible + gcc backend + clang assembler gcc/cm3cg output "stubs' and "non-lazy pointers" in any order. clang outputs them all stubs first, then non-lazy pointers. When the order isn't like clang, the LLVM assembler doesn't match up the refs and defs correctly, leading to function calls resolving to somewhat arbitrary other functions. It is "like" a compiler bug, you get incorrect code, but it is an assembler bug. There are three classes of fix. But nothing perfect/easy. 1) Run the same assembler as gcc. On my system this is "as" in path. But I don't know if this is guaranteed to work. I currently use gcc -x assembler as the assembler. 2) Drop 10.4 support -- don't output the stubs. 3) Change cm3cg to output all stubs before all non-lazy pointers. I'd have to see if that is easy or wait for the gcc developers to offer a patch. 4) Drop 32bit. :) 64bit never has stubs. If gcc output object files directly, like most other compilers do, that'd fix it. The C backend has no problem. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: RE: Mac 10.10? Date: Sun, 9 Aug 2015 03:01:30 +0000 And now 10.10 works.I think what changed is switching to a self-built gcc 5.2.0 from the builtin "gcc" which is actually clang/llvm (frontend and backend). I'll have to try again with the builtin. Strange. It was failing with the 4.7 gcc backend. i.e. with very little C.And I believe it had failed with C backend (prior to the Builder.m3 problems). I switched to the C backend for better debugging and it went away. I had somewhat accidentally also changed C compilers -- oops, too much change. Anyway, after I rebuild the entire system instead of just the compiler, I'll go back and try some more combinations and debug.. one has to learn a new debugger on this system too -- lldb replaces m3gdb, and the Xcode gui debugger was briefly working quite nicely but stopped working. :( Is anyone else using 10.10 with the "builtin" gcc/clang?Or maybe clang toolset on another platform? Later, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com; m3devel at elegosoft.com Subject: alloca(jmpbuf_size) is in Date: Sun, 9 Aug 2015 01:31:14 +0000 I've seen it work now on AMD64_LINUX, SPARC32_SOLARIS, and I386_DARWIN 10.5.8.I think MacOSX 10.10.4 has something else wrong with it that'll take a while to figure out. So I commited it again. PLEASE NOTE that the upgrade procedure is tricky like with cm3cg, but the results of getting it wrong are worse than the recent round of cm3cg changes.You will get runtime crashes. upgrade.py and upgrade-full.sh and likely upgrade.sh all do the right thing, same as always. Not only is cm3 closely tied with cm3cg, but it also closely tied with m3core.The compiler and runtime collaborate on producing and consuming certain data structures,and these data structures have changed now. At least it should be more debuggable now with the C backend -- so I can see local variables for example.Fixing cm3cg to work with plain -g would be good. Is anyone else running 10.10? Thanks, - Jay From: jay.krell at cornell.edu To: m3commit at elegosoft.com Date: Sat, 8 Aug 2015 18:59:55 +0000 Subject: Re: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... I believe this is working ok on MacOS X 10.5.8 with cm3cg and C backend.However things do not work on 10.10.4. I'm guessing for another unknownreason but I want to debug that separately for now. If anyone else can give the previous a try on their configuration, that'd be appreciated.(In github app, you can revert my revert. I haven't found how to do w/ git command line). Thank you. Specifically on 10.0.4 I get: * thread #1: tid = 0x3a9861, 0x002b7360 cm3`RTAllocator__GetTracedObj + 36, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x24) frame #0: 0x002b7360 cm3`RTAllocator__GetTracedObj + 36 * frame #1: 0x002b6e90 cm3`RTHooks__AllocateTracedObj + 17 frame #2: 0x00281272 cm3`TextUtils_M3 + 49 frame #3: 0x002c705c cm3`RTLinker__RunMainBody + 763 frame #4: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #5: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #6: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #7: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #8: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #9: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #10: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #11: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #12: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #13: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #14: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #15: 0x002c6f2d cm3`RTLinker__RunMainBody + 460 frame #16: 0x002c64e8 cm3`RTLinker__AddUnitI + 287 frame #17: 0x002c6563 cm3`RTLinker__AddUnit + 116 frame #18: 0x00002341 cm3`main(argc=1, argv=0xbffffca8, envp=0xbffffcb0) + 97 at _m3main.c:22 frame #19: 0x9518e6d9 libdyld.dylib`start + 1 - Jay Date: Sat, 8 Aug 2015 11:49:10 -0700 From: jay.krell at cornell.edu To: m3commit at elegosoft.com Subject: [M3commit] [modula3/cm3] 57ad34: Revert "allocate jmpbufs with alloca(external vari... Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 57ad34f5034d5000bd5d63fb384851921bc782a1 https://github.com/modula3/cm3/commit/57ad34f5034d5000bd5d63fb384851921bc782a1 Author: jaykrell Date: 2015-08-08 (Sat, 08 Aug 2015) Changed paths: M m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3 M m3-sys/m3back/src/M3C.m3 M m3-sys/m3cc/gcc/gcc/m3cg/parse.c R m3-sys/m3front/src/misc/Jmpbufs.i3 R m3-sys/m3front/src/misc/Jmpbufs.m3 M m3-sys/m3front/src/misc/M3.i3 M m3-sys/m3front/src/misc/Marker.i3 M m3-sys/m3front/src/misc/Marker.m3 M m3-sys/m3front/src/misc/m3makefile M m3-sys/m3front/src/stmts/TryFinStmt.m3 M m3-sys/m3front/src/stmts/TryStmt.m3 M m3-sys/m3front/src/values/Module.i3 M m3-sys/m3front/src/values/Module.m3 M m3-sys/m3front/src/values/Procedure.m3 M m3-sys/m3middle/src/M3RT.m3 M m3-sys/m3middle/src/Target.i3 M m3-sys/m3middle/src/Target.m3 Log Message: ----------- Revert "allocate jmpbufs with alloca(external variable initialized in C)" This reverts commit 44fbd1608a73affa0625a9df4bff1c7248cc6f3c. _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 19:34:26 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 13 Aug 2015 10:34:26 -0700 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> References: <20150812152155.6D9891A2062@async.async.caltech.edu> <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu> <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu> <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com> Message-ID: Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: > "Yes". > > I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. > > > I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) > > > - Jay > > On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: > >> Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? >> >>> On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote: >>> >>> Sorry, I was distracted by a phone call before - that is the compiler >>> that exhibits the behaviour. Previous to that I was using something >>> that was compiled in 2010, so not very helpful for pinpointing. >>> >>> On 8/12/15, Antony Hosking wrote: >>>> Are you saying that the behavior was still working as of d5.9.0? >>>> Or is that the version you are currently using in which it no longer does >>>> what you want? >>>> We should be able to figure out what commit broke things if we can pinpoint >>>> a time when it worked. >>>> >>>>> On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: >>>>> >>>>> I'm using: >>>>> >>>>> Critical Mass Modula-3 version d5.9.0 >>>>> >>>>> last updated: 2010-07-21 >>>>> >>>>> compiled: 2014-08-27 19:15:04 >>>>> >>>>> configuration: /usr/local/cm3/bin/cm3.cfg >>>>> >>>>> host: AMD64_DARWIN >>>>> >>>>> target: AMD64_DARWIN >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking >>>> > wrote: >>>>> I wasn?t aware this had changed. >>>>> When was this regression introduced? >>>>> >>>>>> On Aug 13, 2015, at 2:11 AM, Darko Volaric >>>>> > wrote: >>>>>> >>>>>> For instance, cm3 used to keep on compiling files after errors were >>>>>> found. Now it seems to stop after one module. That loss of functionality >>>>>> seriously reduces productivity. I couldn't find any switch to reverse the >>>>>> change. When was this change decided? Is there a way to restore it? >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 13 20:47:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 18:47:13 +0000 Subject: [M3devel] cm3cg coverage/profiling features? In-Reply-To: References: , , <20150812152155.6D9891A2062@async.async.caltech.edu>, , <2F825F0E-9BB0-46E7-8053-90EDD3D29264@purdue.edu>, , , , <5497DB30-EF35-4984-AEAA-405CD1620DB7@purdue.edu>, <602F68E4-756B-40FC-9DCA-08B10943389C@gmail.com>, Message-ID: (Note: This was meant to report the unit path but I failed to commit that. This is good enough for today. Later...) From: jay.krell at cornell.edu CC: hosking at purdue.edu; lists at darko.org; jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] cm3cg coverage/profiling features? Date: Thu, 13 Aug 2015 10:34:26 -0700 To: jay.krell at cornell.edu Sorry my mistake in 2013. Fixed: https://github.com/modula3/cm3/commit/ff9855193deeb4c4edde3fe51103f56438ab3206 - Jay On Aug 12, 2015, at 8:43 PM, Jay wrote: "Yes". I will try to figure it out. My changes (that I remember!) were to add C mode. We also have the parallel backend change, off by default. I'd like moving a module between packages to work, correctly incrementally. Like when I moved QPath from circa m3quake to circa m3middle (I don't remember exactly but it is obvious by the inconsistency in module prefix correlation to package -- I didn't rename the module.) - Jay On Aug 12, 2015, at 7:06 PM, Antony Hosking wrote: Jay, I know you made some changes to the builder at some point. Can we try to pinpoint when this regression occurred? On Aug 13, 2015, at 12:05 PM, Darko Volaric wrote:Sorry, I was distracted by a phone call before - that is the compilerthat exhibits the behaviour. Previous to that I was using somethingthat was compiled in 2010, so not very helpful for pinpointing.On 8/12/15, Antony Hosking wrote:Are you saying that the behavior was still working as of d5.9.0? Or is that the version you are currently using in which it no longer does what you want? We should be able to figure out what commit broke things if we can pinpoint a time when it worked. On Aug 13, 2015, at 10:27 AM, Darko Volaric wrote: I'm using: Critical Mass Modula-3 version d5.9.0 last updated: 2010-07-21 compiled: 2014-08-27 19:15:04 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Wed, Aug 12, 2015 at 5:23 PM, Antony Hosking > wrote: I wasn?t aware this had changed. When was this regression introduced? On Aug 13, 2015, at 2:11 AM, Darko Volaric > wrote: For instance, cm3 used to keep on compiling files after errors were found. Now it seems to stop after one module. That loss of functionality seriously reduces productivity. I couldn't find any switch to reverse the change. When was this change decided? Is there a way to restore it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Thu Aug 13 22:27:54 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 13:27:54 -0700 Subject: [M3devel] Build Server - Plan Message-ID: I'm setting up a server for building CM3 that takes a "minimalist" approach. It's a machine running several virtual machines, one for each platform supported by CM3. Each VM will contain clean install of the OS plus any external tool dependencies. It will have a minimal compiler install, basically enough to compile itself for the host target. I'm going create one VM for each target I have a bootstrap compiler for, so if you think you'll find this useful and you want a target supported PLEASE CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. Users can request any version of the compiler from the github repository and all new commits will be automatically built for all platforms. The publicly available build products will be: - minimal executables for bootstrap, eg the frontend and a backend - model compiler config files - compilation logs for bootstrap executables - compilation logs for most modules in the github repository - logs for certain tests Packages, libraries, scripts and non-essential tools or executables will not be built or used, the idea being that people take the minimal bootstraps and build from there. I'll post the URL when it's up and running and any suggestions are welcome. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Aug 13 22:31:29 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 13 Aug 2015 16:31:29 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150813203129.GA20453@topoi.pooq.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. > > I'm going create one VM for each target I have a bootstrap compiler for, so > if you think you'll find this useful and you want a target supported PLEASE > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > Users can request any version of the compiler from the github repository > and all new commits will be automatically built for all platforms. > > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests > > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. > > I'll post the URL when it's up and running and any suggestions are welcome. Will the C backend be one of the platforms (or several, if they're not yet all compatible?) -- hendrik From jay.krell at cornell.edu Fri Aug 14 00:26:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 13 Aug 2015 22:26:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: , <20150813203129.GA20453@topoi.pooq.com> Message-ID: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 02:14:16 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 17:14:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150813203129.GA20453@topoi.pooq.com> References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: I want to try to support every sort of backend I can configure. That's the focus, frontend and backend executables. In particular the idea is to be able to test backends against each other for correctness with a particular front end (and the other way around). On Thu, Aug 13, 2015 at 1:31 PM, Hendrik Boom wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, > so > > if you think you'll find this useful and you want a target supported > PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first > up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 04:11:08 2015 From: lists at darko.org (Darko Volaric) Date: Thu, 13 Aug 2015 19:11:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > > minimal executables for bootstrap, eg the frontend and a backend > > > While I have fiddled with this being just cm3, I currently propose it is > the "min" output of make-dist.py (which does output an "all" variant as > well, > it'll take just a bit of change to make that optional). > > > Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), > libm3 (static suffices). > > > make-dist.py also already outputs make-dist.py.log or such next to > itself I believe. > See "Tee" in pylib.py. > > > cm3 is a valid minimal bootstrap, and I have used it a number of times > all on its own, > but only IF you also have a matching source tree -- > m3cc/config/libm3/m3core. > It doesn't work if it is some "random old working" cm3 and not the rest. > > > "min" can be slightly more min I believe -- it has all the .m3 files for > m3core/libm3, > for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > > > > > Will the C backend be one of the platforms (or several, > > if they're not yet all compatible?) > > > Unfortunately, the C backend output is not compatible with the gcc > backend output. > They vary in how they pass the "display" to nested procedures (oh, if > only we didn't > have nested procedures...) > So we might want to do something about this. > I have considered appending "C" to BUILD_DIR, and have done so with mixed > success. > Alternatively, do everything twice. > > > Also, if isn't clear, the C backend output is still very > target-specific. > I'd like to fix that, but I don't have a full plan in mind and might > give up. > > > Note that the minimal bootstrap can be used no matter which backend > you are next going to use, i.e. any bootstrap can feed into C backend, > gcc backend, LLVM backend. > > > Can you really do Mac? > And if so, multiple versions? > > There is unfortunate combinatorial explosion beyond all this. > Specifically, you could n different C compiler per platform. > gcc x, gcc y, clang x, clang y, etc. > > > > logs for certain tests > > > We have been using m3-sys/m3tests as a start. Just a start. > > > Thank you, > - Jay > > > > > Date: Thu, 13 Aug 2015 16:31:29 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > > > It's a machine running several virtual machines, one for each platform > > > supported by CM3. Each VM will contain clean install of the OS plus any > > > external tool dependencies. It will have a minimal compiler install, > > > basically enough to compile itself for the host target. > > > > > > I'm going create one VM for each target I have a bootstrap compiler > for, so > > > if you think you'll find this useful and you want a target supported > PLEASE > > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be > first up. > > > > > > Users can request any version of the compiler from the github > repository > > > and all new commits will be automatically built for all platforms. > > > > > > The publicly available build products will be: > > > > > > - minimal executables for bootstrap, eg the frontend and a backend > > > - model compiler config files > > > - compilation logs for bootstrap executables > > > - compilation logs for most modules in the github repository > > > - logs for certain tests > > > > > > Packages, libraries, scripts and non-essential tools or executables > will > > > not be built or used, the idea being that people take the minimal > > > bootstraps and build from there. > > > > > > I'll post the URL when it's up and running and any suggestions are > welcome. > > > > Will the C backend be one of the platforms (or several, if they're not > > yet all compatible?) > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 08:36:29 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 06:36:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: Message-ID: <20150814063629.GA3086@zoho.com> On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. From jay.krell at cornell.edu Fri Aug 14 09:02:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 07:02:51 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: , <20150814063629.GA3086@zoho.com> Message-ID: It is very easy to install. "We" run make-dist.py and get out a "min" install and an "all" install (per platform).You pick one, extract it, set $PATH optionally, and you are done -- run cm3 either via $PATH or by full path. There is some chance our binaries built on a "new" system won't run on an "old" system.We can mitigate that by deferring some of the build to install time.I hope to have a better story here, a more "sourcey" distribution, that is easy to install.But the binary distributions usually work well. Sometimes you have to edit the config files, but I have endeavored to make that not necessary. There is another install path that prompts for path to compiler, linker, etc., even tries to figure it out. But I don't use that. I think it isn't needed. Yes we support significantly more than x86/amd64. We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw machines -- very recently even.In the 5.8.6 timeframe, I was setup to build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built tonight -- the gcc-based backend. Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Posix or NT for I/O, networking, threads, etc., and X windows or Win32.That is, while we don't run in a browser, we can run almost anywhere else, generally because other people have ported their underlying systems everywhere. (Run in a browser though: LLVM w/ emcriptem backend) The system was already very portable. I have made it easier to port. I assume the "difficult" goals are meant to be ensure that people don't have to jump through the hoops of "install lots of extra stuff" to get it to work. Most people will overshoot the minimal install requirements, but they will overshoot differently.I - Jay > Date: Fri, 14 Aug 2015 06:36:29 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Fri Aug 14 16:49:56 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Fri, 14 Aug 2015 14:49:56 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150814144956.GA4391@zoho.com> Hi, please see comments within: On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > It is very easy to install. Ok, from what I read here I misunderstood. But I am glad to hear it. > Yes we support significantly more than x86/amd64. Thanks for the explanation. I am following projects that run on non-Intel platforms and I get a little concerned when there is movement to only run on Intel or when there seems to be the position Intel is all there is. While I concede Intel is overwhelmingly what people have there is also some very nice and increasingly cheap really good hardware based on other, better architectures and it's a shame to see that ignored into obvlivion. I was happy to see the broad range of hardware and OS when I looked at the CM3 download page so when somebody noted he was going to tackle the Big 3 I was concerned because many projects and OS are now pretty much Intel only which I think is both unfortunate and also technically limiting in terms of code quality. > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > tonight -- the gcc-based backend. I remember we talked about the confusion of the various flavors of SPARC listed in the downloads section. If people are in a consolidating mood it might be good to just use the Solaris Studio version instead of gcc on that platform. > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, Well, LLVM is a significant inhibitor to running anywhere but Intel as far as I know. LLVM SPARC does not seem to even be in the alpha stage and doesn't look like it's on anybody's list of things to do. LLVM MIPS is also not working well from the looks of things on the Debian MIPS list. Other projects that have adopted LLVM seem to use it only on Intel and have stayed with gcc on other platforms. There is a concern that will eventually be too difficult to maintain and the natural solution will be to abandon non-Intel hardware. I understand and agree with the value of LLVM from the standpoint of reducing code complexity over gcc and also the more liberal license but it's disappointing to see only the Intel support is probably ever going to be worth anything as far as LLVM goes. > The system was already very portable. I have made it easier to port. > > I assume the "difficult" goals are meant to be ensure that people don't > have to jump through the hoops of "install lots of extra stuff" to get it > to work. I really don't know but I would agree with that. The install should go in standard paths and not require any prereqs, if possible. Thank you. From lists at darko.org Fri Aug 14 18:22:08 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:22:08 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 18:30:16 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 09:30:16 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814063629.GA3086@zoho.com> References: <20150814063629.GA3086@zoho.com> Message-ID: What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 19:18:10 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 10:18:10 -0700 Subject: [M3devel] Build Server - Access and Architecture Message-ID: Anyone who wants to setup their own build VMs is welcome to do so and SSH access will be provided. The VMs are controlled from the host command line and you can jump into the command line of any VM. The host will have a local, full, and up to date copy of the github repository. There will be VMs with pristine installs of each of the supported OSes available on the host. These will include MacOS, Windows and various Unixes. To make a build VM you copy an OS VM and setup the build environment by installing the compiler and any required tools. This becomes the baseline for any actual compiling/building. The VM must also have a "build" script that performs the work. This script must pull the requested source version from the local git repository before building. The host build driver performs the following steps for each build VM: - it starts the build VM and calls the build script - when the build script has completed it copies the products (results) from the build VM onto the host where they are consequently uploaded to the publically accessible download server. - the host then rolls back/restores the build VM to its original state and stops it. This means there can be no contamination between builds and you always start building from a known, working state. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 20:15:37 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 11:15:37 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150813203129.GA20453@topoi.pooq.com> Message-ID: Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: > The server will be running MacOS 10.10 as the host. The VMs can handle any > MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need > for anything else? > > If you mean can we have multiple VMs for deploying different compiler > configs, then the answer is yes. Each VM will be suspended until it's > needed for a build. Once it's built the requested products and they are > uploaded to the download server, it shuts down. So there aren't any real > performance or memory limitations, all disks are SSDs. > > > On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > >> > minimal executables for bootstrap, eg the frontend and a backend >> >> >> While I have fiddled with this being just cm3, I currently propose it is >> the "min" output of make-dist.py (which does output an "all" variant as >> well, >> it'll take just a bit of change to make that optional). >> >> >> Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), >> libm3 (static suffices). >> >> >> make-dist.py also already outputs make-dist.py.log or such next to >> itself I believe. >> See "Tee" in pylib.py. >> >> >> cm3 is a valid minimal bootstrap, and I have used it a number of times >> all on its own, >> but only IF you also have a matching source tree -- >> m3cc/config/libm3/m3core. >> It doesn't work if it is some "random old working" cm3 and not the rest. >> >> >> "min" can be slightly more min I believe -- it has all the .m3 files >> for m3core/libm3, >> for debugging. I'm not sure if the .i3 files are needed, or the >> .ig/.mg. >> >> >> >> > Will the C backend be one of the platforms (or several, >> > if they're not yet all compatible?) >> >> >> Unfortunately, the C backend output is not compatible with the gcc >> backend output. >> They vary in how they pass the "display" to nested procedures (oh, if >> only we didn't >> have nested procedures...) >> So we might want to do something about this. >> I have considered appending "C" to BUILD_DIR, and have done so with >> mixed success. >> Alternatively, do everything twice. >> >> >> Also, if isn't clear, the C backend output is still very >> target-specific. >> I'd like to fix that, but I don't have a full plan in mind and might >> give up. >> >> >> Note that the minimal bootstrap can be used no matter which backend >> you are next going to use, i.e. any bootstrap can feed into C backend, >> gcc backend, LLVM backend. >> >> >> Can you really do Mac? >> And if so, multiple versions? >> >> There is unfortunate combinatorial explosion beyond all this. >> Specifically, you could n different C compiler per platform. >> gcc x, gcc y, clang x, clang y, etc. >> >> >> > logs for certain tests >> >> >> We have been using m3-sys/m3tests as a start. Just a start. >> >> >> Thank you, >> - Jay >> >> >> >> > Date: Thu, 13 Aug 2015 16:31:29 -0400 >> > From: hendrik at topoi.pooq.com >> > To: m3devel at elegosoft.com >> > Subject: Re: [M3devel] Build Server - Plan >> >> > >> > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: >> > > I'm setting up a server for building CM3 that takes a "minimalist" >> approach. >> > > >> > > It's a machine running several virtual machines, one for each platform >> > > supported by CM3. Each VM will contain clean install of the OS plus >> any >> > > external tool dependencies. It will have a minimal compiler install, >> > > basically enough to compile itself for the host target. >> > > >> > > I'm going create one VM for each target I have a bootstrap compiler >> for, so >> > > if you think you'll find this useful and you want a target supported >> PLEASE >> > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be >> first up. >> > > >> > > Users can request any version of the compiler from the github >> repository >> > > and all new commits will be automatically built for all platforms. >> > > >> > > The publicly available build products will be: >> > > >> > > - minimal executables for bootstrap, eg the frontend and a backend >> > > - model compiler config files >> > > - compilation logs for bootstrap executables >> > > - compilation logs for most modules in the github repository >> > > - logs for certain tests >> > > >> > > Packages, libraries, scripts and non-essential tools or executables >> will >> > > not be built or used, the idea being that people take the minimal >> > > bootstraps and build from there. >> > > >> > > I'll post the URL when it's up and running and any suggestions are >> welcome. >> > >> > Will the C backend be one of the platforms (or several, if they're not >> > yet all compatible?) >> > >> > -- hendrik >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 20:46:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 18:46:46 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150813203129.GA20453@topoi.pooq.com>, , , Message-ID: I have tried PPC_DARWIN on 10.5. It doesn't *quite* work. Almost. I don't believe Rosetta has adequate compatibility, i.e in suspending threadsand getting their context. When we switch to cooperative suspend that will probably work.And then we can remove a bunch more target-specific code. Generally you don't need to support Rosetta. It just works.But I don't believe Rosetta quite fully works. PPC_DARWIN definitely had a long somewhat recent run of workingon native hardware and probably still does.(Fyi, I386_DARWIN also worked before MacOSX/Intel was public.) - Jay Date: Fri, 14 Aug 2015 11:15:37 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan Actually, if we also have 10.6.8 we should be able to support PPC_DARWIN, no? Does the compiler and toolchain work with Rosetta? On Thu, Aug 13, 2015 at 7:11 PM, Darko Volaric wrote: The server will be running MacOS 10.10 as the host. The VMs can handle any MacOS version but I'm thinking 10.9 and 10.10 only. Is there a real need for anything else? If you mean can we have multiple VMs for deploying different compiler configs, then the answer is yes. Each VM will be suspended until it's needed for a build. Once it's built the requested products and they are uploaded to the download server, it shuts down. So there aren't any real performance or memory limitations, all disks are SSDs. On Thu, Aug 13, 2015 at 3:26 PM, Jay K wrote: > minimal executables for bootstrap, eg the frontend and a backend While I have fiddled with this being just cm3, I currently propose it is the "min" output of make-dist.py (which does output an "all" variant as well, it'll take just a bit of change to make that optional). Specifically -- cm3, cm3cg (optional), config, m3core (static suffices), libm3 (static suffices). make-dist.py also already outputs make-dist.py.log or such next to itself I believe. See "Tee" in pylib.py. cm3 is a valid minimal bootstrap, and I have used it a number of times all on its own, but only IF you also have a matching source tree -- m3cc/config/libm3/m3core. It doesn't work if it is some "random old working" cm3 and not the rest. "min" can be slightly more min I believe -- it has all the .m3 files for m3core/libm3, for debugging. I'm not sure if the .i3 files are needed, or the .ig/.mg. > Will the C backend be one of the platforms (or several, > if they're not yet all compatible?) Unfortunately, the C backend output is not compatible with the gcc backend output. They vary in how they pass the "display" to nested procedures (oh, if only we didn't have nested procedures...) So we might want to do something about this. I have considered appending "C" to BUILD_DIR, and have done so with mixed success. Alternatively, do everything twice. Also, if isn't clear, the C backend output is still very target-specific. I'd like to fix that, but I don't have a full plan in mind and might give up. Note that the minimal bootstrap can be used no matter which backend you are next going to use, i.e. any bootstrap can feed into C backend, gcc backend, LLVM backend. Can you really do Mac? And if so, multiple versions? There is unfortunate combinatorial explosion beyond all this. Specifically, you could n different C compiler per platform. gcc x, gcc y, clang x, clang y, etc. > logs for certain tests We have been using m3-sys/m3tests as a start. Just a start. Thank you, - Jay > Date: Thu, 13 Aug 2015 16:31:29 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > > > I'm going create one VM for each target I have a bootstrap compiler for, so > > if you think you'll find this useful and you want a target supported PLEASE > > CONTRIBUTE A BOOTSTRAP COMPILER. MacOS, Linux and Windows will be first up. > > > > Users can request any version of the compiler from the github repository > > and all new commits will be automatically built for all platforms. > > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > > > I'll post the URL when it's up and running and any suggestions are welcome. > > Will the C backend be one of the platforms (or several, if they're not > yet all compatible?) > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:07:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:07:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: ARM_DARWIN I never quite finished.It was almost working on a jail-broken iPhone 3gs.Setting up and testing just a C development environmentis often the difficult part. If you do have a C cross compiler yes that can shift slightly,easily where some things happen. Let me explain.The compilation process has a few different steps it can go.Including that we have some C code.So that goes .c => .o, or internally .c => .s => .o. Ignoring LLVM...We have the NT386 backend.Code goes right from .m3 => .obj.cm3 does "everything". We have the gcc backend.Code goes.m3 => .mc => .ms => .o .ms files are just assembly source.mc files are an internal format that cm3 writes out. We have the C backend.Code goes.m3 => .c => .s => .ocm3 writes the .c files. And then there is linking. This is all *hightly* automated.From the users point of view, cm3 seems to go from .m3 files to a .a/.dylib/executable. But cm3 only has within it some of the stages.It doesn't have a C compiler, an assembler, or a linker.We depend on a "C" toolchain for this (I'm calling assembler/linker part of the C toolchain). If you don't have the C toolchain, then you are stuck part way through those pipelines.Either with a bunch of .o files you can't link.Or a bunch of .ms files you can't assemble and link.Or a bunch of .c files you can't compile/assemble/link. Native toolchains are historically more common than cross tool chains.So we have a "boot" workflow -- that needs work -- where on the "first" machine,you stop where I say you are "stuck", you archive up the files, copy themto the target machine, which has a native toolchain, and then resume the pipeline --running C compiler/assembler/linker. Now, if you have a cross toolchain there is not quite reason for that. But, it comes down mainly to the config files and the builder, and another detail I'll get to. The "boot" workflow was well automated 20 years ago.Back when "quake" was written in C. The 3.6 release.I don't remember what the 4.0/4.1 release looked like. I have good automated flow for producing a cm3 for cross scenarios,and it is useful, but it requires a matching source tree on the other side. This is boot1.py and boot2.sh. If there is an incompatibility, like I recently added to m3core, then it doesn't work.So it dawns on me now, that boot1.py should produce m3core/libm3, and maybe cm3cg.Though cm3cg would require a cross compiler, so sometimes a pain. "another detail I'll get to".We have a small number of tools that when we build the system, we buildthem and run time. Other than cm3. There is mklib, which I already handle.But there is also m3bundle and PklFonts and some RPC stub generators. Given a cross C toolset, we don't have conventions/automation around,hey, I'm targeting x, building on y, my y-hosted tools might need an update.I need to build all of y, then all of x, and when I'm building x, what is the cm3I use? What is the C compiler I use? Again it is mostly just a thing with config files.The config files tend to run "gcc" and find it in $PATH.They assume it is for the current host/target.But there needs to be arguably some convention, I'm hosting on I386_LINUX,targeting AMD64_DARWIN -- where are the native tools and where are the cross tools?In gcc-land, this is basically you run ad64-apple-darwin-gcc for the AMD64 target toolset,instead of just "gcc" You might also run e.g. /usrl/ocal/cm3/bin//cm3cg. So, very long story short -- we largely need a file system layout. The "GNU platform" mapping I386_DARWIN => i386-apple-darwin8 for example,is somewhat encoded in our tree, but for this to work it needs more formalization. With LLVM, you get all the backends together in on tree and you givethe executable a command line switch and it finds the right .so, I imagine,so this sort of gets easier. We still have to know the switches and work that into config and such.We should just do that for native targets anyway and then cross will reuse that. Again, sorry, long story short -- paths and switches to compiler/assembler/linker is largely the thing. Apple has a good system here btw.You say gcc -arch ppc.And that combines with -x assembler.So you can cross compile with gcc -arch ppc.You can cross assemble with gcc -arch -x assembler.They put the work in the gcc driver. It then farms out to the correct cc1/cc1plus/as. If only this was the norm.. Another thing -- a cross C compiler that can handle something with no #includes andno linking, is really easy to build. It is the headers and libraries that make it more difficult.The assembler/linker are also sometimes a sticking point, because sometimes, if youread the gcc config/install manual, they prefer the native closed source assembler/linker. I have a working VMS C cross compiler from Mac, but I had to copy various filesfrom a running VMS machine to do it -- the headers and libraries. In as much as the assembler/linker is GNU, it is easier.And Apple's stuff is open source and people have ported it.But imagine targeting HP-UX or AIX, or NT using the Microsoft toolset.There you can't just conjure cross tools with arbitrary host, only what the vendor ships. - Jay Date: Fri, 14 Aug 2015 09:30:16 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:11:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:11:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, Message-ID: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:24:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:24:54 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150814144956.GA4391@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com> Message-ID: > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. I already mostly solved this years ago. SOLgnu and SOLsun were always already the same, e.g. same threading,same jmpbuf, except the config files picked a different C compiler and differentoutput directory name. I decided it shouldn't be separate paths everywhere in cm3/config files,only which C compiler to run and output directory names.That is, only the config files should vary. We have a somewhat separate notion of "TARGET" and "BUILD_DIR".We might need to better about that.For example profile build get "p" appended to BUILD_DIR, but still the same TARGET.We need BUILD_DIR=SOLgnu but TARGET=SPARC32_SOLARIS. The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun compiler.I invented a separate variable with better granularity. Just look at SPARC32_SOLARIS: readonly TARGET = "SPARC32_SOLARIS"readonly C_COMPILER = "SUN" % or GNU (SOLgnu configuration)include("SPARC32_SOLARIS.common") and then SPARC32_SOLARIS.common looks for the compiler you ask for,and makes the switches match also. Rodney gave me the term for this elegance the other day -- "Cartesian factoring". Or, only factor out what varies. Where things are the same, don't duplicate. In the old days the config files were very monolithic and there was duplication all over the place.For example, in the old days, HPPA implies HP-UX, but now HPPA can be Linux, NetBSD, FreeBSD, etc.The HPPA part is one file, the Linux/NetBSD/FreeBSD parts are their own single files.We only have a combinatorial explosion of small config files, not large ones.Even this could be improved -- runtime composition? The old days had somewhat less sharing. i.e. we probably used "vendor" cc/as/ld more and GNUless. But there is still good allowance for that now. I similarly refactor I386_NT.It was called NT386.The "mingw" platform is largely the same -- Win32 threads, Win32 I/O, Win32 GUI, GNU compiler/assembler/linker.There I came up with several variables for the factoring: See NT.commonTHREAD_LIBRARY = MS or PTHREADSWINDOW_LIBRARY = MS or XLINKER = MS or GNUC_COMPILER = MS or GNU (should add "clang" or "llvm" as an option here) LINUXLIBC6 is now I386_LINUX. NT386GNU is I386_CYGWIN or I386_MINGW A problem I ran into though wrt SOLgnu/SOLsun/LINUXLIBC6/NT386 is that nobodyusing them wanted to switch the name. So I fixed n systems by renaming to m, leaving us with n + m, but they are justlayered over very common stuff. It doesn't take n + m work to implement.But the system is cluttered. Has anyone switched from LINUXLIBC6 to I386_LINUX? Can we delete LINUXLIBC6?Has anyone/everyone switched from SOLsun/SOLgnu to SPARC32_SOLARIS?FreeBSD4 to I386_FREEBSD? Well, at least all the AMD64 and DARWIN targets are well named.The others maybe aren't used much either way.. - Jay > Date: Fri, 14 Aug 2015 14:49:56 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > Hi, please see comments within: > > On Fri, Aug 14, 2015 at 07:02:51AM +0000, Jay K wrote: > > > It is very easy to install. > > Ok, from what I read here I misunderstood. But I am glad to hear it. > > > Yes we support significantly more than x86/amd64. > > Thanks for the explanation. I am following projects that run on non-Intel > platforms and I get a little concerned when there is movement to only run on > Intel or when there seems to be the position Intel is all there is. While I > concede Intel is overwhelmingly what people have there is also some very > nice and increasingly cheap really good hardware based on other, better > architectures and it's a shame to see that ignored into obvlivion. > > I was happy to see the broad range of hardware and OS when I looked at the > CM3 download page so when somebody noted he was going to tackle the Big 3 I > was concerned because many projects and OS are now pretty much Intel only > which I think is both unfortunate and also technically limiting in terms of > code quality. > > > We have SPARC32_SOLARIS and SPARC64_SOLARIS, that I test on the opencsw > > machines -- very recently even.In the 5.8.6 timeframe, I was setup to > > build and test PPC_DARWIN, PPC_LINUX, SPARC32_LINUX.Look at > > m3-sys/m3cc/src/buildmany.sh. All but ARM_DARWIN there i just built > > tonight -- the gcc-based backend. > > I remember we talked about the confusion of the various flavors of SPARC > listed in the downloads section. If people are in a consolidating mood it > might be good to just use the Solaris Studio version instead of gcc on that > platform. > > > Our portability rests on gcc, or now a C compiler, or now maybe LLVM, > > Well, LLVM is a significant inhibitor to running anywhere but Intel as far > as I know. LLVM SPARC does not seem to even be in the alpha stage and > doesn't look like it's on anybody's list of things to do. LLVM MIPS is also > not working well from the looks of things on the Debian MIPS list. Other > projects that have adopted LLVM seem to use it only on Intel and have stayed > with gcc on other platforms. There is a concern that will eventually be too > difficult to maintain and the natural solution will be to abandon non-Intel > hardware. > > I understand and agree with the value of LLVM from the standpoint of > reducing code complexity over gcc and also the more liberal license but it's > disappointing to see only the Intel support is probably ever going to be > worth anything as far as LLVM goes. > > > The system was already very portable. I have made it easier to port. > > > > I assume the "difficult" goals are meant to be ensure that people don't > > have to jump through the hoops of "install lots of extra stuff" to get it > > to work. > > I really don't know but I would agree with that. The install should go in > standard paths and not require any prereqs, if possible. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:32:02 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:32:02 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 21:42:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 19:42:00 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , Message-ID: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Fri Aug 14 21:48:29 2015 From: lists at darko.org (Darko Volaric) Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: > Reasonable but painful. And varied. > > > Do you want the Apple-supplied tools or fetch various cc/ld source and > build them? > > > For Windows, Microsoft or Cygwin or Mingw? > > Probe the machine to see if the tools are already present? > Run cc/gcc/cl, see if it works? > > > I like the autoconf model here -- which is indeed that last point. > It tries to compile "int main() { return 0; }", if that works, great a > working C compiler. > If it doesn't, it might fish around for a few, but ultimately it gives up, > and the > user can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. > > > It doesn't know how to apt-get install build-essentials or other. > > > I must say -- autoconf -- I don't like sh or m4, and it is slow, but it > does kinda work well. > I wrestle with this. I output C and I need int8/int16/int32/int64. > I have the following choices > 1 assume char/short/int/__int64/long long > 2 #if on compiler for __int64 vs. long long > 3 try to check limits.h -- paying for the cost of #include for every > compile > 4 check for STDC_VERSION and use inttypes.h/stdint.h > 5 autoconf > 6 something like autoconf > > > For now I do some mix of 1/2/3, but I have this nagging feeling that > autoconf would > gain portability to more systems. > > > Modula-3 follows "the Perl way" and is ported everywhere. > Whereas autoconf can kind of cons up a port if the system is the > combination > of already known elements. Only individual elements need to be known about, > not combinations. > > > autoconf again is ugly and slow, but it is better in terms of "cartesian > factoring". > > And it doesn't handle Microsoft tools well or at all. > > > Also I have a similar nagging dilemma on bootstrap archives. > I want some incrementally. So I want e.g. make. > I want (currently lack) some hierarchy. So I want recursive make or maybe > GNU make-specific. > Maybe automake? Which still doesn't buy me MS make. > cmake? > I don't really want to reinvent portability beyond GNU make. > out-of-source builds? With bootstrapping? Overkill? > So I wonder if I should, gasp, generate automake input. > > make does though cater to the new generation and their IDEs, while also > catering to the old ways. > KDE's use of it is a big gote > > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 12:32:02 -0700 > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > I think the Mac and Win installers should also install the required > external tools, so it all "just works", which makes it a bit more > difficult. Either way not a priority right now. > > On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: > > For SPARC we have the opencsw machines. > > make-dist.py builds .deb files already automatically. > Somewhat crude in terms of package granularity and declared depenencies > and such. We have "min" and "all" and no declared dependencies, though you > need a C toolset -- "build-essentials". > > A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some > extra text files, wrapped up in an ar file. > > > make-diet.py builds .msi files for Windows already. > I generate a bunch of .xml and then run the wix tools. > > > We don't have any Mac installers. > You just extract a .tar.gz of binaries and set PATH. > That works on all platforms. > > I also recently wrote the very short and simple capture-min.py > that captures a minimal cm3/m3core/libm3 from your existing install (you > have to edit the path). > For native building. > > Really this stuff isn't so difficult, but I don't explain it well, and I > didn't invest much in "package building". > > - Jay > > > > ------------------------------ > Date: Fri, 14 Aug 2015 09:22:08 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. > > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. > > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " > > At some later date I'll also be working on MacOS, Windows and Debian > installers, and installation will be trivial then. Until then it will be a > matter of following simple instructions to set up CM3 the required tools. > > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. > > > > > > > On Thu, Aug 13, 2015 at 11:36 PM, wrote: > > On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > > I'm setting up a server for building CM3 that takes a "minimalist" > approach. > > > > It's a machine running several virtual machines, one for each platform > > supported by CM3. Each VM will contain clean install of the OS plus any > > external tool dependencies. It will have a minimal compiler install, > > basically enough to compile itself for the host target. > > Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way > to > emulate this from Intel. > > > The publicly available build products will be: > > > > - minimal executables for bootstrap, eg the frontend and a backend > > - model compiler config files > > - compilation logs for bootstrap executables > > - compilation logs for most modules in the github repository > > - logs for certain tests > > This sounds like a setup for experts. Why not make a turn-key tarball > available like was available (I think) before? > > > Packages, libraries, scripts and non-essential tools or executables will > > not be built or used, the idea being that people take the minimal > > bootstraps and build from there. > > That's fine for the 3 or 4 core guys doing all the work! For everybody else > this is a big inhibitor to making CM3 generally useful to the rest of the > world- unless the instructions to get a complete install are very clear and > easy to follow. But it seems from watching the discussion here it is > non-trivial to get CM3 installed. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:22:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:22:10 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , , , , Message-ID: Good thinking...but not easy to pick the one.People will complain no matter what you chose.MS? Cygwin? Mingw? Cross inhibits MS.Cross that isn't prebuilt will take a while to build.MS probably not redistributable. Have to point people at a web page. Chasing annual versions.. - Jay Date: Fri, 14 Aug 2015 12:48:29 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com No, the idea is to pick one strategy. People don't need a lot of choices, just one good one. I see the installer installing its own (static) tools (included in the installer) in its own place, or possibly relying on XCode. This is for people who want something simple, fixed and turnkey. Others can set up their own. On Fri, Aug 14, 2015 at 12:42 PM, Jay K wrote: Reasonable but painful. And varied. Do you want the Apple-supplied tools or fetch various cc/ld source and build them? For Windows, Microsoft or Cygwin or Mingw? Probe the machine to see if the tools are already present?Run cc/gcc/cl, see if it works? I like the autoconf model here -- which is indeed that last point.It tries to compile "int main() { return 0; }", if that works, great a working C compiler.If it doesn't, it might fish around for a few, but ultimately it gives up, and theuser can set CC and CFLAGS. Or PATH to put what it knows about in $PATH. It doesn't know how to apt-get install build-essentials or other. I must say -- autoconf -- I don't like sh or m4, and it is slow, but it does kinda work well.I wrestle with this. I output C and I need int8/int16/int32/int64.I have the following choices 1 assume char/short/int/__int64/long long 2 #if on compiler for __int64 vs. long long 3 try to check limits.h -- paying for the cost of #include for every compile 4 check for STDC_VERSION and use inttypes.h/stdint.h 5 autoconf 6 something like autoconf For now I do some mix of 1/2/3, but I have this nagging feeling that autoconf wouldgain portability to more systems. Modula-3 follows "the Perl way" and is ported everywhere.Whereas autoconf can kind of cons up a port if the system is the combinationof already known elements. Only individual elements need to be known about,not combinations. autoconf again is ugly and slow, but it is better in terms of "cartesian factoring". And it doesn't handle Microsoft tools well or at all. Also I have a similar nagging dilemma on bootstrap archives.I want some incrementally. So I want e.g. make.I want (currently lack) some hierarchy. So I want recursive make or maybe GNU make-specific.Maybe automake? Which still doesn't buy me MS make.cmake?I don't really want to reinvent portability beyond GNU make.out-of-source builds? With bootstrapping? Overkill?So I wonder if I should, gasp, generate automake input. make does though cater to the new generation and their IDEs, while also catering to the old ways.KDE's use of it is a big gote - Jay Date: Fri, 14 Aug 2015 12:32:02 -0700 From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan I think the Mac and Win installers should also install the required external tools, so it all "just works", which makes it a bit more difficult. Either way not a priority right now. On Fri, Aug 14, 2015 at 12:11 PM, Jay K wrote: For SPARC we have the opencsw machines. make-dist.py builds .deb files already automatically.Somewhat crude in terms of package granularity and declared depenencies and such. We have "min" and "all" and no declared dependencies, though you need a C toolset -- "build-essentials". A .deb file is just .tar.gz or .tar.xz or .tar.bz2 or .tar.lzma with some extra text files, wrapped up in an ar file. make-diet.py builds .msi files for Windows already.I generate a bunch of .xml and then run the wix tools. We don't have any Mac installers.You just extract a .tar.gz of binaries and set PATH.That works on all platforms. I also recently wrote the very short and simple capture-min.pythat captures a minimal cm3/m3core/libm3 from your existing install (you have to edit the path).For native building. Really this stuff isn't so difficult, but I don't explain it well, and I didn't invest much in "package building". - Jay Date: Fri, 14 Aug 2015 09:22:08 -0700 From: lists at darko.org To: m3devel at elegosoft.com Subject: Re: [M3devel] Build Server - Plan The solution for SPARC might be to create a cross-compiler. That's not ideal but would be useful for verifying that things compile for the platform at least, short of having the right hardware. I disagree with your "only for experts" assessment. The point of this server is that you don't have to compiler the compiler and a backend just to get the latest (or even a properly working) compiler. That's the very "expert" work I'm trying to automate. What this system spits out is everything you need to compile anything you want - a complete and working compiler. Not everyone wants or needs to use all the packages. If you do, you can. Here are the instructions for doing that: "cd ; cm3 " At some later date I'll also be working on MacOS, Windows and Debian installers, and installation will be trivial then. Until then it will be a matter of following simple instructions to set up CM3 the required tools. I'll be opening up the server to anyone who wants to do something different. If someone wants to produce tarballs they can setup their own VMs and they will get built too. On Thu, Aug 13, 2015 at 11:36 PM, wrote: On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote: > I'm setting up a server for building CM3 that takes a "minimalist" approach. > > It's a machine running several virtual machines, one for each platform > supported by CM3. Each VM will contain clean install of the OS plus any > external tool dependencies. It will have a minimal compiler install, > basically enough to compile itself for the host target. Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to emulate this from Intel. > The publicly available build products will be: > > - minimal executables for bootstrap, eg the frontend and a backend > - model compiler config files > - compilation logs for bootstrap executables > - compilation logs for most modules in the github repository > - logs for certain tests This sounds like a setup for experts. Why not make a turn-key tarball available like was available (I think) before? > Packages, libraries, scripts and non-essential tools or executables will > not be built or used, the idea being that people take the minimal > bootstraps and build from there. That's fine for the 3 or 4 core guys doing all the work! For everybody else this is a big inhibitor to making CM3 generally useful to the rest of the world- unless the instructions to get a complete install are very clear and easy to follow. But it seems from watching the discussion here it is non-trivial to get CM3 installed. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:39:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:39:10 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? Message-ID: Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 14 22:43:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 14 Aug 2015 20:43:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? Message-ID: air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Aug 15 22:43:59 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 16:43:59 -0400 Subject: [M3devel] Current releas(es) Message-ID: <20150815204359.GA15497@topoi.pooq.com> I'm currently installing stuff on an effecively new machine. Where do I find the current Modula 3 release? Is it still http://www.opencm3.net/releng/ ? If not, I'd better update the wikipedia entry. And is CM3 5.8.6 the current release? -- hendrik From jay.krell at cornell.edu Sun Aug 16 00:15:30 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 15 Aug 2015 22:15:30 +0000 Subject: [M3devel] Current releas(es) In-Reply-To: <20150815204359.GA15497@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: 5.8.6 is unfortunately probably current for most architectures.We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 before: foreach root in [ m3cc, bin ] after:foreach root in [ bin ] - Jay > Date: Sat, 15 Aug 2015 16:43:59 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: [M3devel] Current releas(es) > > I'm currently installing stuff on an effecively new machine. > > Where do I find the current Modula 3 release? > > Is it still > > http://www.opencm3.net/releng/ > > ? If not, I'd better update the wikipedia entry. > > And is CM3 5.8.6 the current release? > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From adacore at marino.st Sun Aug 16 01:42:07 2015 From: adacore at marino.st (John Marino) Date: Sun, 16 Aug 2015 01:42:07 +0200 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <55CFCE4F.2070104@marino.st> Don't forget FreeBSD has 5.10.0 easily available: http://www.freshports.org/lang/modula3 John On 8/16/2015 12:15 AM, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. > We have a newer build for MacOSX. > You will need one or two patches, if you want to run upgrade.py to get > latest. > I remember this one: > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > > before: > > foreach root in [ m3cc, bin ] > > after: > foreach root in [ bin ] > > > - Jay > >> Date: Sat, 15 Aug 2015 16:43:59 -0400 >> From: hendrik at topoi.pooq.com >> To: m3devel at elegosoft.com >> Subject: [M3devel] Current releas(es) >> >> I'm currently installing stuff on an effecively new machine. >> >> Where do I find the current Modula 3 release? >> >> Is it still >> >> http://www.opencm3.net/releng/ >> >> ? If not, I'd better update the wikipedia entry. >> >> And is CM3 5.8.6 the current release? >> >> -- hendrik >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From hendrik at topoi.pooq.com Sun Aug 16 04:15:41 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 15 Aug 2015 22:15:41 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> Message-ID: <20150816021541.GA22514@topoi.pooq.com> On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > 5.8.6 is unfortunately probably current for most architectures. The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should really carry a version number. Should I try and find the official Debian syntax for version numbers? -- hendrik We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > > before: > foreach root in [ m3cc, bin ] > after:foreach root in [ bin ] > > > - Jay > > > Date: Sat, 15 Aug 2015 16:43:59 -0400 > > From: hendrik at topoi.pooq.com > > To: m3devel at elegosoft.com > > Subject: [M3devel] Current releas(es) > > > > I'm currently installing stuff on an effecively new machine. > > > > Where do I find the current Modula 3 release? > > > > Is it still > > > > http://www.opencm3.net/releng/ > > > > ? If not, I'd better update the wikipedia entry. > > > > And is CM3 5.8.6 the current release? > > > > -- hendrik > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > From jay.krell at cornell.edu Sun Aug 16 04:20:05 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 15 Aug 2015 19:20:05 -0700 Subject: [M3devel] Current releas(es) In-Reply-To: <20150816021541.GA22514@topoi.pooq.com> References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: "Yes please take over packaging?" :) Start with scripts/python/make-dist.py? - Jay On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: >> 5.8.6 is unfortunately probably current for most architectures. > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > really carry a version number. Should I try and find the official > Debian syntax for version numbers? > > -- hendrik > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 >> >> before: >> foreach root in [ m3cc, bin ] >> after:foreach root in [ bin ] >> >> >> - Jay >> >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 >>> From: hendrik at topoi.pooq.com >>> To: m3devel at elegosoft.com >>> Subject: [M3devel] Current releas(es) >>> >>> I'm currently installing stuff on an effecively new machine. >>> >>> Where do I find the current Modula 3 release? >>> >>> Is it still >>> >>> http://www.opencm3.net/releng/ >>> >>> ? If not, I'd better update the wikipedia entry. >>> >>> And is CM3 5.8.6 the current release? >>> >>> -- hendrik >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> From microcode at zoho.com Sun Aug 16 09:16:08 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:16:08 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> Message-ID: <20150816071607.GA3479@zoho.com> On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > I already mostly solved this years ago. Oh, that is good news. > SOLgnu and SOLsun were always already the same, e.g. same threading,same > jmpbuf, except the config files picked a different C compiler and > differentoutput directory name. I had misunderstood there was a choice because of the download names, suggesting you actually got a compiler built with Studio or gcc depending on which file you downloaded. > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > compiler.I invented a separate variable with better granularity. That sounds much better, thanks. But for the downloadable stuff where presumably you get a working setup already built and just have to untar the tarball, which compiler(s) are you going forward with now? In other words, since you have consolidate the various Solaris versions as you listed above and it is no longer necessarily indicative of Studio v. gcc, are you going to make turnkey builds available as in the past and if so what will they be built with? And I would vote for using Studio if possible... Thanks. From microcode at zoho.com Sun Aug 16 09:27:30 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 07:27:30 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> Message-ID: <20150816072730.GB3479@zoho.com> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. From jay.krell at cornell.edu Sun Aug 16 09:52:13 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 07:52:13 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com> Message-ID: The output of autoconf/automake should have lightweight dependencies.They might stress make, might require GNU make.They might stress the sh, but I think there are adequate shells out there. They are meant to be easy for people building stuff to use.They aren't meant to be easy for people developing stuff to use. Look at this way..while people complain and there are widelyused alternatives like cmake, autoconf/automake are in widespreaduse, and they do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, Aix, Irix, etc. Furthermore, consider any platform that has a native gcc, is likely buildingthat with autoconf/automake. I remain very template for "bootstrap" archives to consist of C and automake/autoconf/libtool output.Or assembly, but still automake/autoconf -- at least to generate makefiles and decide how to run the assembler. - Jay > Date: Sun, 16 Aug 2015 07:27:30 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Sun Aug 16 10:24:09 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 08:24:09 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150816082409.GC3479@zoho.com> On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > The output of autoconf/automake should have lightweight dependencies.They > might stress make, might require GNU make.They might stress the sh, but I > think there are adequate shells out there. That is typically one issue with autoconf, requiring gnu tools in the path. On Solaris this can be annoying since most Solaris people don't use gcc or bash or have them in their path and not all (none of?) the gnu pieces are up to date or even current by any stretch of the imagination. > They are meant to be easy for people building stuff to use.They aren't > meant to be easy for people developing stuff to use. Not sure what you meant here... > Look at this way..while people complain and there are widelyused > alternatives like cmake, autoconf/automake are in widespreaduse, and they > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > Aix, Irix, etc. They often don't "work" for Solaris as-installed but they can most often be made to work. Increasingly, as automake and its prereqs get version bumps there are problems building apps on Solaris because Solaris installs with a very back-level version of gcc and a rather incomplete set of gnu tools. I ran into a problem with the last year where Solaris awk was not good enough to install(!) an app that compiled on Solaris as part of autotools so I had to download a newish copy of gawk and install it. So really, it is much better not to go there if you can avoid it. I am not suggesting an alternative to autotools but just pointing out it is not really accurate to say they work on Linux, Solaris, BSD, etc. except for Linux. I also run BSD on several platforms and because of the same issues as with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, shells, etc. not being good enough for autotools) it is sometimes non-trivial and painful to get Linux apps built on BSD. There is obviously no good/easy solution to this, just to point out it is not the slam-dunk as might be thought. > Furthermore, consider any platform that has a native gcc, is likely > buildingthat with autoconf/automake. Conceded, yet on Solaris it is not clear why gcc is there. It is old and it is in a non-standard location and is often not used. Ideally, apps to be run on Solaris should be able to be built with native (non-gnu) tools. The Studio compilers are very good and have optimizations for SPARC that should be better than what gcc can provide and it is also somewhat of a test of C portability since Studio doesn't necessarily provide all non-standard gcc extensions. From jay.krell at cornell.edu Sun Aug 16 12:04:29 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:04:29 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816071607.GA3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150814144956.GA4391@zoho.com>, , <20150816071607.GA3479@zoho.com> Message-ID: It is like you thought:The SOLsun build used the Sun C compiler, and probably assembler and linker.The SOLgnu build used the GNU C compiler, and maybe assembler, and maybe linker.It isn't a big deal.Turnkey is based on what people have installed.I don't know how to acquire/install the Sun compilers. Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need configurations..just a collection of factors. - Jay > Date: Sun, 16 Aug 2015 07:16:08 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > I already mostly solved this years ago. > > Oh, that is good news. > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > jmpbuf, except the config files picked a different C compiler and > > differentoutput directory name. > > I had misunderstood there was a choice because of the download names, > suggesting you actually got a compiler built with Studio or gcc depending on > which file you downloaded. > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > compiler.I invented a separate variable with better granularity. > > That sounds much better, thanks. But for the downloadable stuff where > presumably you get a working setup already built and just have to untar the > tarball, which compiler(s) are you going forward with now? In other words, > since you have consolidate the various Solaris versions as you listed above > and it is no longer necessarily indicative of Studio v. gcc, are you going > to make turnkey builds available as in the past and if so what will they be > built with? And I would vote for using Studio if possible... > > Thanks. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:11:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:11:15 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816082409.GC3479@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com> Message-ID: Autoconf output has little/none dependency.Sun's SPARC optimization are mostly irrelevant as we have little C code,unless you use the C backend. > somewhat of a test of C portability This I appreciate. We have some C code and the C backend.More compilers have "helped".The Tru64 compiler was another.On HP-UX in-box I had only K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And now clang -- having found and worked around a bug in its assembler.Some time soon I'll expand to Metrowerks and Digital Mars.. - Jay > Date: Sun, 16 Aug 2015 08:24:09 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > The output of autoconf/automake should have lightweight dependencies.They > > might stress make, might require GNU make.They might stress the sh, but I > > think there are adequate shells out there. > > That is typically one issue with autoconf, requiring gnu tools in the > path. On Solaris this can be annoying since most Solaris people don't use > gcc or bash or have them in their path and not all (none of?) the gnu pieces > are up to date or even current by any stretch of the imagination. > > > They are meant to be easy for people building stuff to use.They aren't > > meant to be easy for people developing stuff to use. > > Not sure what you meant here... > > > Look at this way..while people complain and there are widelyused > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > Aix, Irix, etc. > > They often don't "work" for Solaris as-installed but they can most often be > made to work. Increasingly, as automake and its prereqs get version bumps > there are problems building apps on Solaris because Solaris installs with a > very back-level version of gcc and a rather incomplete set of gnu tools. I > ran into a problem with the last year where Solaris awk was not good enough > to install(!) an app that compiled on Solaris as part of autotools so I had > to download a newish copy of gawk and install it. So really, it is much > better not to go there if you can avoid it. > > I am not suggesting an alternative to autotools but just pointing out it is > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > Linux. I also run BSD on several platforms and because of the same issues as > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > shells, etc. not being good enough for autotools) it is sometimes > non-trivial and painful to get Linux apps built on BSD. There is obviously > no good/easy solution to this, just to point out it is not the slam-dunk as > might be thought. > > > Furthermore, consider any platform that has a native gcc, is likely > > buildingthat with autoconf/automake. > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > is in a non-standard location and is often not used. Ideally, apps to be run > on Solaris should be able to be built with native (non-gnu) tools. The > Studio compilers are very good and have optimizations for SPARC that should > be better than what gcc can provide and it is also somewhat of a test of C > portability since Studio doesn't necessarily provide all non-standard gcc > extensions. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 12:12:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 10:12:18 +0000 Subject: [M3devel] Target.Structure_size_boundary? In-Reply-To: References: Message-ID: ok, so this structure_size_boundary isn't entirely bogus.In fact, as is typical, it might have been correct, but itsrelevance has drastically declined as the targets have shifted.Like so much of cm3, it is based closely on gcc. It appears that OpenBSD/68k rounds struct sizes up to a multiple of 2. Other 68k targets do not. Perhaps historical 68k targets did -- perhaps that is a lot of what cm3 was handling. ARM sometimes rounds up to a multiple of 4, but sometimes not, depending on exact platform. sh used to? and still optionally rounds up to a multiple of 4, but I think defaults to not? The only non-8 case we had in cm3 was for HPPA and that was wrong seeming. Getting this value wrong should only lead to a lack of C interop on unusual records.i.e. if you have an array of records with an odd size, the adress of than the firstmight mismatch.It can have an affect in non-array-of-record cases too, if you use BYTESIZE.Or, when the C backend does layout, possibly disagreeing with the frontend. Maybe I'm naive but this all seems like a mistake. sizeof(struct {char a;}) should be 1 for all targets. sizeof(struct {char a[3];}) should be 3 for all targets. but I guess various ABI developers have disagreed through the decadesand we are kinda somewhat stuck with it. There is a gcc command line switch, at least for ARM, to vary the rounding. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: Target.Structure_size_boundary? Date: Fri, 14 Aug 2015 20:43:18 +0000 air:src jay$ grep -i boundary * Target.i3: boundary (i.e. (size + align - 1) DIV align * align).Target.i3: Structure_size_boundary: CARDINAL;Target.i3-old: boundary (i.e. (size + align - 1) DIV align * align).Target.i3-old: Structure_size_boundary: CARDINAL;Target.m3: Structure_size_boundary := 8;Target.m3: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 8;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;Target.m3-old: Structure_size_boundary := 32;Target.m3-old: Structure_size_boundary := 16;TargetT.i3: structure_size_boundary: CARDINAL; Surely we can just consider this 8 and remove all of it uses? Are all those old >8 uses even correct? sizeof(struct {char a;}) really isn't always 1? sizeof(struct {char a[3];}) really isn't always 3? Actually we can verify this easily enough with a cross gcc w/o cross driver/linker/assembler/headers/libraries.I'll do that later. Plus, we can have the C code statically verify struct sizes agreebetween front end backend, as long as they get encoded anywhere by frontend. You know, I want to limit Target.m3 to endian and word size. Even those I wish I could remove. Thanks, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Aug 16 14:18:42 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 16 Aug 2015 08:18:42 -0400 Subject: [M3devel] Current releas(es) In-Reply-To: References: <20150815204359.GA15497@topoi.pooq.com> <20150816021541.GA22514@topoi.pooq.com> Message-ID: <20150816121842.GA4830@topoi.pooq.com> On Sat, Aug 15, 2015 at 07:20:05PM -0700, Jay wrote: > "Yes please take over packaging?" :) :) > Start with scripts/python/make-dist.py? As far as I can tell looking at my package cache, all that's needed is to change the name of the file to cm3-LINUXLIBC6-REL_5.8.6_i386.deb When I install it with dpkg, it appears to know internally what revision it is, so not much would need changing. Havint the version number there would help me know whether to bother downloading another copy if I seem to already have one. I probably wasted somoe of your server time by redownloading. -- hendrik > > - Jay > > On Aug 15, 2015, at 7:15 PM, Hendrik Boom wrote: > > > On Sat, Aug 15, 2015 at 10:15:30PM +0000, Jay K wrote: > >> 5.8.6 is unfortunately probably current for most architectures. > > > > The file http://www.opencm3.net/releng/cm3-LINUXLIBC6-REL.deb should > > really carry a version number. Should I try and find the official > > Debian syntax for version numbers? > > > > -- hendrik > > > > > > We have a newer build for MacOSX.You will need one or two patches, if you want to run upgrade.py to get latest.I remember this one: > >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/cm3cfg.common.diff?r1=1.30.2.8;r2=1.30.2.9 > >> > >> before: > >> foreach root in [ m3cc, bin ] > >> after:foreach root in [ bin ] > >> > >> > >> - Jay > >> > >>> Date: Sat, 15 Aug 2015 16:43:59 -0400 > >>> From: hendrik at topoi.pooq.com > >>> To: m3devel at elegosoft.com > >>> Subject: [M3devel] Current releas(es) > >>> > >>> I'm currently installing stuff on an effecively new machine. > >>> > >>> Where do I find the current Modula 3 release? > >>> > >>> Is it still > >>> > >>> http://www.opencm3.net/releng/ > >>> > >>> ? If not, I'd better update the wikipedia entry. > >>> > >>> And is CM3 5.8.6 the current release? > >>> > >>> -- hendrik > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> From microcode at zoho.com Sun Aug 16 15:39:31 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:39:31 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150814144956.GA4391@zoho.com> <20150816071607.GA3479@zoho.com> Message-ID: <20150816133931.GA5008@zoho.com> On Sun, Aug 16, 2015 at 10:04:29AM +0000, Jay K wrote: > It is like you thought:The SOLsun build used the Sun C compiler, and > probably assembler and linker.The SOLgnu build used the GNU C compiler, and > maybe assembler, and maybe linker.It isn't a big deal. Turnkey is based on > what people have installed. Are you saying now that you have consolidated the builds into the 4 you listed below they will use what the user has installed? If so that sounds ideal, except for possible search order issues. The gcc that "comes with" Solaris is always installed in the same location (/usr/sfw) but the path to Studio depends on the version and I believe you could create non-standard paths using the tarball installers vs. the package installers (Studio offers both options IIRC). And then there is what the user has in their PATH which might be all, one, or none of the above. And there are some users who install new(er) versions of gcc, usually in addition to the stock version and there is no convention for where this gcc should go. So...there should probably be a documented way to use the toolchain of your choice, if this isn't already explained somewhere. > I don't know how to acquire/install the Sun compilers. They are a free download from Oracle (free to use and free runtime). If they are not already installed on your opencsw box(es) and you would like to have them on there I can do that. It is just a matter of downloading and installing the packages. It should take 20 to 30 minutes. > Also,"SOL" is SPARC32_SOLARIS.We now have SPARC64_SOLARIS, I386_SOLARIS, > AMD64_SOLARIS.I didn't want to double up those also.Arguably we don't need > configurations..just a collection of factors. I think this is the right thing to do, I just don't understand how to select which toolchain you want to use, and further, I thought the downloads contained a prebuilt cm3, so I don't understand why a C toolchain would be required at all, unless you are saying in the future the user will compile cm3 rather than get a prebuilt binary. > - Jay > > > > Date: Sun, 16 Aug 2015 07:16:08 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Fri, Aug 14, 2015 at 07:24:54PM +0000, Jay K wrote: > > > > people are in a consolidating mood it might be good to > just use the Solaris Studio version instead of gcc on that platform. > > > I already mostly solved this years ago. > > > > Oh, that is good news. > > > > > SOLgnu and SOLsun were always already the same, e.g. same threading,same > > > jmpbuf, except the config files picked a different C compiler and > > > differentoutput directory name. > > > > I had misunderstood there was a choice because of the download names, > > suggesting you actually got a compiler built with Studio or gcc depending on > > which file you downloaded. > > > > > The "new" target names are SPARC32_SOLARIS, SPARC64_SOLARIS, and > > > I386_SOLARIS,AMD64_SOLARIS. In the config files you can pick GNU or Sun > > > compiler.I invented a separate variable with better granularity. > > > > That sounds much better, thanks. But for the downloadable stuff where > > presumably you get a working setup already built and just have to untar the > > tarball, which compiler(s) are you going forward with now? In other words, > > since you have consolidate the various Solaris versions as you listed above > > and it is no longer necessarily indicative of Studio v. gcc, are you going > > to make turnkey builds available as in the past and if so what will they be > > built with? And I would vote for using Studio if possible... > > > > Thanks. > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From microcode at zoho.com Sun Aug 16 15:43:06 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Sun, 16 Aug 2015 13:43:06 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> Message-ID: <20150816134306.GB5008@zoho.com> On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > Autoconf output has little/none dependency.Sun's SPARC optimization are > mostly irrelevant as we have little C code,unless you use the C backend. Ok, well, it sounds like it will be increasingly important as you go to your C backend though. > > somewhat of a test of C portability > This I appreciate. We have some C code and the C backend.More compilers > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > now clang -- having found and worked around a bug in its assembler.Some > time soon I'll expand to Metrowerks and Digital Mars.. I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have the latest versions. C/C++ are not my thing but if I can help by just building stuff to see what messages we get or if it breaks etc. let me know and I will try to participate. I didn't know Metrowerks was still around. I thought they got swallowed up by Motorola and then removed from all the non-embedded space. > - Jay > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > From: microcode at zoho.com > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Build Server - Plan > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > The output of autoconf/automake should have lightweight dependencies.They > > > might stress make, might require GNU make.They might stress the sh, but I > > > think there are adequate shells out there. > > > > That is typically one issue with autoconf, requiring gnu tools in the > > path. On Solaris this can be annoying since most Solaris people don't use > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > are up to date or even current by any stretch of the imagination. > > > > > They are meant to be easy for people building stuff to use.They aren't > > > meant to be easy for people developing stuff to use. > > > > Not sure what you meant here... > > > > > Look at this way..while people complain and there are widelyused > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > Aix, Irix, etc. > > > > They often don't "work" for Solaris as-installed but they can most often be > > made to work. Increasingly, as automake and its prereqs get version bumps > > there are problems building apps on Solaris because Solaris installs with a > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > ran into a problem with the last year where Solaris awk was not good enough > > to install(!) an app that compiled on Solaris as part of autotools so I had > > to download a newish copy of gawk and install it. So really, it is much > > better not to go there if you can avoid it. > > > > I am not suggesting an alternative to autotools but just pointing out it is > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > Linux. I also run BSD on several platforms and because of the same issues as > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > shells, etc. not being good enough for autotools) it is sometimes > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > no good/easy solution to this, just to point out it is not the slam-dunk as > > might be thought. > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > buildingthat with autoconf/automake. > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > is in a non-standard location and is often not used. Ideally, apps to be run > > on Solaris should be able to be built with native (non-gnu) tools. The > > Studio compilers are very good and have optimizations for SPARC that should > > be better than what gcc can provide and it is also somewhat of a test of C > > portability since Studio doesn't necessarily provide all non-standard gcc > > extensions. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- From jay.krell at cornell.edu Sun Aug 16 20:28:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 18:28:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: Message-ID: Actually, I suggest that endianness might be removable as a factor. I know of the following endian-dependentness: 1) bitfield layout This matters "only" so that bitfields can be declared that line up with C to interoperate with. Or, sort of, to interoperate with hardware -- but hardware probably has no endianness, i.e. much hardware can be used with both little and big endian CPUs, and this variability actually makes portabililiy more difficult. I suggest this might not matter and we can pick an arbitrary endianness, at least for the C backend. 2) To pick code that picks apart floating point numbers. I suggest this code can be rewritten in C and/or do a runtime detection of endianness, in C or Modula-3. I don't think endianess matters in initialization non-bitfields. Or at least integers. I have to check. Potentially leaving us with just 4: 32bit posix 64bit posix 32bit windows 64bit windows I know at least one more Target factor to bring up, later (alignment of closures). - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: posix/nt/32/64/endian platforms for C backend? Date: Fri, 14 Aug 2015 20:39:10 +0000 Now that jmpbuf size is factored out.I'm wondering if, for the C backend, we can have a total of 6-8 targets. L64_UNIX (little endian 64bit posix/pthreads/Xwindows) L32_UNIX (little endian 32bit posix/pthreads/Xwindows) B64_UNIX (big endian 64bit posix/pthreads/Xwindows) B32_UNIX (big endian 32bit posix/pthreads/Xwindows) L64_NT (little endian Win64 threads/gui) L32_NT (little endian Win32 threads/gui) Xbox 360 is big endian, but nevermind that? Windows CE has any big endian targets? Call then WIN instead of NT? I find "endian" too long and "L" too short. No more of this Darwin/OpenBSD/FreeBSD/Solaris/NetBSD/DragonflyBSD stuff?For the C backend. There is a missing element here.What m3core/src/thread/PTHREAD does -- different source files for OpenBSD/FreeBSD/Apple/etc. To address those, well, I'd surround each file with an #ifdef, and eitherconcatenate them all in git, or in the quake file, if using the C backendreference one file that #includes them all -- and says h_file for the constituents or such.Thereby making one common quake path at least for the C backend.There is slight extra I/O cost implied that the current scheme avoids. Longer term I'd like just one target for C, but that might be difficult/impossible.Whereas this seems tractable. Now, unfortunately, there might be more "cartesian factors".e.g. Win32 gui + pthreads. You can do that with Cygwin.e.g. has_stack_walker But I'm hoping to avoid the last.Hopefully has_stack_walker is modeled by higher levelm3cg calls, at the very least: "call_setjmp_if_no_stack_walker"that then flow into the C through #ifdefs. Hypothetically: C output #ifdef _WIN32 .. or whatever platforms have stack walkers, will be most eventually #define call_setjmp(buf) (0) #else #define call_setjmp(buf) setjmp(buf) #endif used within an if. Something like that. or even #ifdef __cpluspls and... i.e. the frontend can issue "both" sets of CG and later one C compilationignores "half" of them. If we output C++ this is simpler, just use its mechanism. Anyway, that is getting ahead of things, the stack walker aspect. Introduce those 6 targets? Yes I know there is one or two more things in Target.m3 to deal with -- standard structsis the same for all C backend. I'll send mail on the other. Too confusing?Not good enough -- given more than one? Keep all the targets so we can write config files that say i386-apple-gcc, amd64-gnu-linux-gcc? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Sun Aug 16 22:59:15 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 13:59:15 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816072730.GB3479@zoho.com> References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 16 23:40:45 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 16 Aug 2015 21:40:45 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, Message-ID: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 02:04:03 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 17:04:03 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, > including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically > get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't > have X installed, it fails. We can make it do nothing. We can link to the > stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap > archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if > using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how > to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse > it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU > build system" > It is slow and bad to author in, but produces easy to use somewhat slow > idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would > be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: > install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. > Even gcc. > > > - Jay > > > > ------------------------------ > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus > different distributions, there are different views on how CM3 should be > setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I > want to take it further than the current installer. In particular I want to > reduce the external, internal and environmental dependencies so that you > can build anything, anywhere with just the minimal set of executables and a > standard set of tools. Others want a more elaborate setup which gives them > more flexibility; that can also be derived from that same minimal set. > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > The solution for SPARC might be to create a cross-compiler. That's not > > ideal but would be useful for verifying that things compile for the > > platform at least, short of having the right hardware. > > You should have access to opencsw as Jay mentioned. If not, I can offer > machine time but I can't keep the machine running all the time. Hopefully > the current setup (opencsw) is still viable. > > > I disagree with your "only for experts" assessment. The point of this > > server is that you don't have to compiler the compiler and a backend just > > to get the latest (or even a properly working) compiler. That's the very > > "expert" work I'm trying to automate. > > That is good to hear because from your list of deliverables it seemed to me > there would still be a lot of tinkering to get a complete install. If you > are scripting that and it can be built effortlessly and mindlessly from the > deliverables then good. I think it would be ideal and very nice if people > could download a turnkey tarball ready to run on their platform, or source > code ready to build and an installer script that installs the final > product. Jay mentioned autotools and they are usually far from optimal on > Solaris because of autotools reliance on so much gnu stuff that is not > always necessarily present on Solaris but that is the general idea- to be > able to configure && make && make install with all the standard stuff > happening like everything being installed in /usr/local/bin|lib|share and > so > forth. > > > What this system spits out is everything you need to compile anything you > > want - a complete and working compiler. Not everyone wants or needs to > use > > all the packages. If you do, you can. Here are the instructions for doing > > that: "cd ; cm3 " > > It was probably just my unfamiliarity with the build setup and terms. I'm > sorry if I wasted anybody's time with this diversion. I have been following > the mailing list and it looks like Jay and Rodney are mostly discussing > things with a very small number of regulars. Coming from the outside as I > do > I think anybody new is going to have similar questions/misunderstandings of > how cm3 is built and installed, etc. > > > I'll be opening up the server to anyone who wants to do something > > different. If someone wants to produce tarballs they can setup their own > > VMs and they will get built too. > > Hopefully this can be integrated with what you already have running on > opencsw. > > Thank you. > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 17 03:24:07 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 16 Aug 2015 18:24:07 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU build system" >> It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. Even gcc. >> >> >> - Jay >> >> >> >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and so >> forth. >> >> > What this system spits out is everything you need to compile anything you >> > want - a complete and working compiler. Not everyone wants or needs to use >> > all the packages. If you do, you can. Here are the instructions for doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I do >> I think anybody new is going to have similar questions/misunderstandings of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Mon Aug 17 05:14:27 2015 From: lists at darko.org (Darko Volaric) Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link > it all. > > > The compiler has to come with a matching m3core/libm3, binary and/or > source. The cm3cg binary can be built from source, but again it has to be > matching -- or use C backend. > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can > be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, > minimal toolset (mostly just gcc), installed in a particular place. I'm not > trying to support or test every configuration, just one. It's for people > who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know > there are no external source dependencies in the frontend and the backend > depends only on the gcc source which is in the repository. The build server > only builds the frontend and backend executables, nothing else, so there > are no other dependencies. > > If you're talking about building an X Windows application, I still don't > get it. Why do I need to support someone wanting to write an X program? I'm > not supporting people who want to do anything other than run the compiler > on their platform. If they want to make X programs they can download and > configure whatever is needed themselves. The CM3 compiler can compile (but > not link) M3 interfaces and modules for X without having X installed. > That's all I'm trying to support. > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > >> What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? >> >> >> What is minimal for a user of cm3 vs. a developer of cm3? >> >> >> I bring up X because there are many programs/demos in the distribution, >> including gui ones. >> On many systems, even X is just an "apt get whatever" away. >> And, if we form our .deb correctly, it is fairly automatic. >> On MacOSX current what happens is if you an X program, you automatically >> get a popup prompting you to install. >> On MacOSX current what happens if you try to build an X program, and >> don't have X installed, it fails. We can make it do nothing. We can link to >> the stub -- but then you have to relink after installing X. >> >> >> How to ease that apart? A sort of dependency on X? >> Likely separate packages, like cm3 and cm3-gui? >> >> >> I keep thinking, for a developer but not user of cm3, the bootstrap >> archives should produce autoconf/automake inputs. >> This handles, for example, "how do I invoke the C compiler?" -- even if >> using the gcc backend, we need a C compiler/linker/assembler. >> Imagine if, you know, our config files dropped away entirely. >> Imagine if the config file was generated at install time by autoconf. >> Imagine if we didn't have a small amount of litter in our system as to >> how to link on HP-UX. Or if AIX supported was "inherited" from libtool. >> Basically, libtool supports more than us an overlaps with us. Maybe reuse >> it. >> >> >> Again, I am somewhat positively and negatively obsessed with "the GNU >> build system" >> It is slow and bad to author in, but produces easy to use somewhat slow >> idiomatic results. >> >> >> Also, to reiterate, I believe the point here was the "build server" would >> be complicated, in order to produce easy to use outputs. >> >> >> One also needs to keep in mind the somewhat opposing but common desires: >> install from source vs. install from binaries. >> Install from binaries is faster. >> Install from source is problematic for compilers written in themselves. >> Even gcc. >> >> >> - Jay >> >> >> >> ------------------------------ >> Date: Sun, 16 Aug 2015 13:59:15 -0700 >> From: lists at darko.org >> To: m3devel at elegosoft.com; microcode at zoho.com >> Subject: Re: [M3devel] Build Server - Plan >> >> Just like there are different views on how Linux should be set up and >> thus different distributions, there are different views on how CM3 should >> be setup or installed, or at least I have a different view. >> >> I definitely believe in the "effortlessly and mindlessly" thing, and I >> want to take it further than the current installer. In particular I want to >> reduce the external, internal and environmental dependencies so that you >> can build anything, anywhere with just the minimal set of executables and a >> standard set of tools. Others want a more elaborate setup which gives them >> more flexibility; that can also be derived from that same minimal set. >> >> >> >> On Sun, Aug 16, 2015 at 12:27 AM, wrote: >> >> On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: >> > The solution for SPARC might be to create a cross-compiler. That's not >> > ideal but would be useful for verifying that things compile for the >> > platform at least, short of having the right hardware. >> >> You should have access to opencsw as Jay mentioned. If not, I can offer >> machine time but I can't keep the machine running all the time. Hopefully >> the current setup (opencsw) is still viable. >> >> > I disagree with your "only for experts" assessment. The point of this >> > server is that you don't have to compiler the compiler and a backend >> just >> > to get the latest (or even a properly working) compiler. That's the very >> > "expert" work I'm trying to automate. >> >> That is good to hear because from your list of deliverables it seemed to >> me >> there would still be a lot of tinkering to get a complete install. If you >> are scripting that and it can be built effortlessly and mindlessly from >> the >> deliverables then good. I think it would be ideal and very nice if people >> could download a turnkey tarball ready to run on their platform, or source >> code ready to build and an installer script that installs the final >> product. Jay mentioned autotools and they are usually far from optimal on >> Solaris because of autotools reliance on so much gnu stuff that is not >> always necessarily present on Solaris but that is the general idea- to be >> able to configure && make && make install with all the standard stuff >> happening like everything being installed in /usr/local/bin|lib|share and >> so >> forth. >> >> > What this system spits out is everything you need to compile anything >> you >> > want - a complete and working compiler. Not everyone wants or needs to >> use >> > all the packages. If you do, you can. Here are the instructions for >> doing >> > that: "cd ; cm3 " >> >> It was probably just my unfamiliarity with the build setup and terms. I'm >> sorry if I wasted anybody's time with this diversion. I have been >> following >> the mailing list and it looks like Jay and Rodney are mostly discussing >> things with a very small number of regulars. Coming from the outside as I >> do >> I think anybody new is going to have similar questions/misunderstandings >> of >> how cm3 is built and installed, etc. >> >> > I'll be opening up the server to anyone who wants to do something >> > different. If someone wants to produce tarballs they can setup their own >> > VMs and they will get built too. >> >> Hopefully this can be integrated with what you already have running on >> opencsw. >> >> Thank you. >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> >> _______________________________________________ M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:42:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:42:03 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , Message-ID: I think you want is roughly: boot1.py over each architecture package those up along with matching m3core/libm3/m3cc source boot2.sh or such on the target system -- but this currently assumes an entire matching source tree, so this needs work But fair enough, you just want compilers, to get around the "crossing". Crossing is easy as far as we do it -- just see boot1.py. It only gets as far as assembly files. Then you move them to the target machine and run native assembler and linker. ARM_LINUX is a sore point for me. Debian has been through at least three different arm ports. For this all to work reasonably cleanly we have to track something like this: https://wiki.debian.org/Multiarch/Tuples Meanwhile, m3front barely cares about platform at all. Almost all platforms generate identical code until they hit the backend. And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. Our "porting" and "supported platforms" is much ado about little. We support "everything and nothing", by leveraging underlying gcc or C compiler and pthreads or Win32, and X or Win32. - Jay Date: Sun, 16 Aug 2015 20:14:27 -0700 Subject: Re: [M3devel] Build Server - Plan From: lists at darko.org To: jay.krell at cornell.edu CC: m3devel at elegosoft.com; microcode at zoho.com If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: I meant like "the entire system" not "just the compiler". Right, to link it all. The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. Python is convenient for cross-platform scripting, but optional. - Jay On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? What is minimal for a user of cm3 vs. a developer of cm3? I bring up X because there are many programs/demos in the distribution, including gui ones. On many systems, even X is just an "apt get whatever" away. And, if we form our .deb correctly, it is fairly automatic. On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. How to ease that apart? A sort of dependency on X? Likely separate packages, like cm3 and cm3-gui? I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. Imagine if, you know, our config files dropped away entirely. Imagine if the config file was generated at install time by autoconf. Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. Basically, libtool supports more than us an overlaps with us. Maybe reuse it. Again, I am somewhat positively and negatively obsessed with "the GNU build system" It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. Install from binaries is faster. Install from source is problematic for compilers written in themselves. Even gcc. - Jay Date: Sun, 16 Aug 2015 13:59:15 -0700 From: lists at darko.org To: m3devel at elegosoft.com; microcode at zoho.com Subject: Re: [M3devel] Build Server - Plan Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. On Sun, Aug 16, 2015 at 12:27 AM, wrote: On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > The solution for SPARC might be to create a cross-compiler. That's not > ideal but would be useful for verifying that things compile for the > platform at least, short of having the right hardware. You should have access to opencsw as Jay mentioned. If not, I can offer machine time but I can't keep the machine running all the time. Hopefully the current setup (opencsw) is still viable. > I disagree with your "only for experts" assessment. The point of this > server is that you don't have to compiler the compiler and a backend just > to get the latest (or even a properly working) compiler. That's the very > "expert" work I'm trying to automate. That is good to hear because from your list of deliverables it seemed to me there would still be a lot of tinkering to get a complete install. If you are scripting that and it can be built effortlessly and mindlessly from the deliverables then good. I think it would be ideal and very nice if people could download a turnkey tarball ready to run on their platform, or source code ready to build and an installer script that installs the final product. Jay mentioned autotools and they are usually far from optimal on Solaris because of autotools reliance on so much gnu stuff that is not always necessarily present on Solaris but that is the general idea- to be able to configure && make && make install with all the standard stuff happening like everything being installed in /usr/local/bin|lib|share and so forth. > What this system spits out is everything you need to compile anything you > want - a complete and working compiler. Not everyone wants or needs to use > all the packages. If you do, you can. Here are the instructions for doing > that: "cd ; cm3 " It was probably just my unfamiliarity with the build setup and terms. I'm sorry if I wasted anybody's time with this diversion. I have been following the mailing list and it looks like Jay and Rodney are mostly discussing things with a very small number of regulars. Coming from the outside as I do I think anybody new is going to have similar questions/misunderstandings of how cm3 is built and installed, etc. > I'll be opening up the server to anyone who wants to do something > different. If someone wants to produce tarballs they can setup their own > VMs and they will get built too. Hopefully this can be integrated with what you already have running on opencsw. Thank you. _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 01:43:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 17 Aug 2015 23:43:37 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu> Message-ID: But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else.It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Aug 18 03:39:09 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 17 Aug 2015 21:39:09 -0400 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> Message-ID: <20150818013909.GA16240@topoi.pooq.com> On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > I think you want is roughly: > boot1.py over each architecture > package those up > along with matching m3core/libm3/m3cc source > boot2.sh or such on the target system -- but this currently > assumes an entire matching source tree, so this needs work > > But fair enough, you just want compilers, to get around the "crossing". > Crossing is easy as far as we do it -- just see boot1.py. > It only gets as far as assembly files. Then you move them to the target > machine and run native assembler and linker. Suppose you used C instead of assembler for this. Would that get you more machine types for less effort? I've had a machine on which there was a dispute which assembler to use! > > ARM_LINUX is a sore point for me. > Debian has been through at least three different arm ports. > For this all to work reasonably cleanly we have to track > something like this: > https://wiki.debian.org/Multiarch/Tuples > > Meanwhile, m3front barely cares about platform at all. > Almost all platforms generate identical code until they hit the backend. > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > Our "porting" and "supported platforms" is much ado about little. > We support "everything and nothing", by leveraging underlying gcc > or C compiler and pthreads or Win32, and X or Win32. > > - Jay > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > Subject: Re: [M3devel] Build Server - Plan > From: lists at darko.org > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; microcode at zoho.com > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > I meant like "the entire system" not "just the compiler". Right, to link it all. > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > Python is convenient for cross-platform scripting, but optional. > - Jay > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > On many systems, even X is just an "apt get whatever" away. > And, if we form our .deb correctly, it is fairly automatic. > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > How to ease that apart? A sort of dependency on X? > Likely separate packages, like cm3 and cm3-gui? > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > Imagine if, you know, our config files dropped away entirely. > Imagine if the config file was generated at install time by autoconf. > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > Install from binaries is faster. > Install from source is problematic for compilers written in themselves. Even gcc. > > > - Jay > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > From: lists at darko.org > To: m3devel at elegosoft.com; microcode at zoho.com > Subject: Re: [M3devel] Build Server - Plan > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > The solution for SPARC might be to create a cross-compiler. That's not > > > ideal but would be useful for verifying that things compile for the > > > platform at least, short of having the right hardware. > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > machine time but I can't keep the machine running all the time. Hopefully > > the current setup (opencsw) is still viable. > > > > > I disagree with your "only for experts" assessment. The point of this > > > server is that you don't have to compiler the compiler and a backend just > > > to get the latest (or even a properly working) compiler. That's the very > > > "expert" work I'm trying to automate. > > > > That is good to hear because from your list of deliverables it seemed to me > > there would still be a lot of tinkering to get a complete install. If you > > are scripting that and it can be built effortlessly and mindlessly from the > > deliverables then good. I think it would be ideal and very nice if people > > could download a turnkey tarball ready to run on their platform, or source > > code ready to build and an installer script that installs the final > > product. Jay mentioned autotools and they are usually far from optimal on > > Solaris because of autotools reliance on so much gnu stuff that is not > > always necessarily present on Solaris but that is the general idea- to be > > able to configure && make && make install with all the standard stuff > > happening like everything being installed in /usr/local/bin|lib|share and so > > forth. > > > > > What this system spits out is everything you need to compile anything you > > > want - a complete and working compiler. Not everyone wants or needs to use > > > all the packages. If you do, you can. Here are the instructions for doing > > > that: "cd ; cm3 " > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > sorry if I wasted anybody's time with this diversion. I have been following > > the mailing list and it looks like Jay and Rodney are mostly discussing > > things with a very small number of regulars. Coming from the outside as I do > > I think anybody new is going to have similar questions/misunderstandings of > > how cm3 is built and installed, etc. > > > > > I'll be opening up the server to anyone who wants to do something > > > different. If someone wants to produce tarballs they can setup their own > > > VMs and they will get built too. > > > > Hopefully this can be integrated with what you already have running on > > opencsw. > > > > Thank you. > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Aug 18 04:31:41 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:31:41 +0000 Subject: [M3devel] posix/nt/32/64/endian platforms for C backend? In-Reply-To: References: , , <16BDCA79-E67D-4622-96D4-1172CB3BA57A@purdue.edu>, , Message-ID: Is bitfield layout really portable enough and predictable enough that interop is achievable?I don't know the layout algorithms myself.I don't really know what packed does.Change the alignment rules for the record?Remove all padding for alignment? And then access fields piecemeal in case anything is no longer aligned?I should experiment. - Jay Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? From: hosking at purdue.edu Date: Tue, 18 Aug 2015 11:57:52 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu On Aug 18, 2015, at 9:43 AM, Jay K wrote:But if these are new targets, that can only generate C?I claim the endianness is arbitrary and only serves for interop with C bitfields and nothing else. And doesn?t interrupt make sense wherever possible. I would hope that there is a reasonable correspondence between C types and M3 types, particularly for RECORD and PACKED elements. It doesn't matter what endianness you declare. It is likely to work correctly either way.I have to double check that integers and floats are initialized "at once", and not a byte at a time. - Jay > Subject: Re: [M3devel] posix/nt/32/64/endian platforms for C backend? > From: hosking at purdue.edu > Date: Mon, 17 Aug 2015 09:49:37 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I strongly prefer that the front-end remain aware of target endianness (as it currently is for packing/unpacking bits in memory). > It is important that there be some correspondence with the target. > So, endianness should remain. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 18 04:37:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 18 Aug 2015 02:37:47 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150818013909.GA16240@topoi.pooq.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , , , , , , <20150818013909.GA16240@topoi.pooq.com> Message-ID: Yes, but use of C is tied to the C backend.If you want to use cm3cg or LLVM it isn't an option.cm3cg was implied, but the C backend is quite good and workingand I recommend it for all targets. The reason for assembly is that is cm3cg output and probablyoptional LLVM output. I'm not sure people are ready/willing to give up cm3cg yet. ? In fact, with the C backend, we are close to having just a small number, like 6-8,service all targets, as discussed in another thread.The only factors become word size, possibly endian, NT vs. posix.(I have to send a mail yet on "aligned procedure types" -- which correlateswith processor, i.e. the same for all x86 targets). "exact" target doesn't matter.esp. with the jmpbuf thing removed. > I've had a machine on which there was a dispute which assembler to use! Which assembler to use is even a problem on modern Mac OSX 10.10.4 Yosemite.The default LLVM assembler, invoked by the default "gcc" (which is actually clang),is not compatible with gcc/cm3cg. It produces the wrong code.The default assembler works. I put a workaround into cm3cg though. It is really just a mess that any compiler these days doesn't default to outputing.o files directly, but that is gcc. - Jay > Date: Mon, 17 Aug 2015 21:39:09 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Mon, Aug 17, 2015 at 11:42:03PM +0000, Jay K wrote: > > I think you want is roughly: > > boot1.py over each architecture > > package those up > > along with matching m3core/libm3/m3cc source > > boot2.sh or such on the target system -- but this currently > > assumes an entire matching source tree, so this needs work > > > > But fair enough, you just want compilers, to get around the "crossing". > > Crossing is easy as far as we do it -- just see boot1.py. > > It only gets as far as assembly files. Then you move them to the target > > machine and run native assembler and linker. > > Suppose you used C instead of assembler for this. Would that get > you more machine types for less effort? I've had a machine on which > there was a dispute which assembler to use! > > > > > ARM_LINUX is a sore point for me. > > Debian has been through at least three different arm ports. > > For this all to work reasonably cleanly we have to track > > something like this: > > https://wiki.debian.org/Multiarch/Tuples > > > > Meanwhile, m3front barely cares about platform at all. > > Almost all platforms generate identical code until they hit the backend. > > And even then they are identical, e.g. I386_LINUX, I386_FREEBSD, I386_OPENBSD, > > I386_NETBSD, I386_SOLARIS likely as identical -- the ABI and the object file format. > > > > Our "porting" and "supported platforms" is much ado about little. > > We support "everything and nothing", by leveraging underlying gcc > > or C compiler and pthreads or Win32, and X or Win32. > > > > - Jay > > > > > > > > > > Date: Sun, 16 Aug 2015 20:14:27 -0700 > > Subject: Re: [M3devel] Build Server - Plan > > From: lists at darko.org > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com; microcode at zoho.com > > > > If you want the "entire system" then go ahead and build it. But you can't do that without "just the compiler" (a bootstrap compiler). That's what I'm trying to provide for every supported platform: the bare essentials. > > If you say "that's not very useful," then I'd say that having no compiler at all is much less useful, since you can't have any system, complete or not, unless you go through the oneous task of building a cross compiler. > > For example, we have a working ARM_LINUX platform, but no publically available bootstrap compiler. What's the point of that? How is being able to compile X programs on Darwin going to fix that? Asking potential users to first get a system that does have a compiler then build a cross compiler is not realistic. > > (You can compile the m3core and libm3 libraries from source having just the bootstrap compiler. If you can't compile m3core/m3lib without an existing m3core/m3lib then I'd call that a bug. Surely the dependency is having the right version of the source, not having a compiled library included with your compiler.) > > > > > > On Sun, Aug 16, 2015 at 6:24 PM, Jay wrote: > > I meant like "the entire system" not "just the compiler". Right, to link it all. > > > > The compiler has to come with a matching m3core/libm3, binary and/or source. The cm3cg binary can be built from source, but again it has to be matching -- or use C backend. > > > > Python is convenient for cross-platform scripting, but optional. > > - Jay > > On Aug 16, 2015, at 5:04 PM, Darko Volaric wrote: > > > > Minimal is cm3 + cm3cg + config file. I'm pretty sure everything else can be built from that, but someone might correct me. > > The required tools issue is solved by only supporting one standard, minimal toolset (mostly just gcc), installed in a particular place. I'm not trying to support or test every configuration, just one. It's for people who are starting from scratch and also serves as a baseline for testing. > > I don't understand what your "dependency on X" issue is. As far as I know there are no external source dependencies in the frontend and the backend depends only on the gcc source which is in the repository. The build server only builds the frontend and backend executables, nothing else, so there are no other dependencies. > > If you're talking about building an X Windows application, I still don't get it. Why do I need to support someone wanting to write an X program? I'm not supporting people who want to do anything other than run the compiler on their platform. If they want to make X programs they can download and configure whatever is needed themselves. The CM3 compiler can compile (but not link) M3 interfaces and modules for X without having X installed. That's all I'm trying to support. > > > > > > On Sun, Aug 16, 2015 at 2:40 PM, Jay K wrote: > > > > > > > > What is minimal? sh? gcc? make? python? perl? A large .tar.gz file? X? > > > > > > What is minimal for a user of cm3 vs. a developer of cm3? > > > > > > I bring up X because there are many programs/demos in the distribution, including gui ones. > > On many systems, even X is just an "apt get whatever" away. > > And, if we form our .deb correctly, it is fairly automatic. > > On MacOSX current what happens is if you an X program, you automatically get a popup prompting you to install. > > On MacOSX current what happens if you try to build an X program, and don't have X installed, it fails. We can make it do nothing. We can link to the stub -- but then you have to relink after installing X. > > > > > > How to ease that apart? A sort of dependency on X? > > Likely separate packages, like cm3 and cm3-gui? > > > > > > I keep thinking, for a developer but not user of cm3, the bootstrap archives should produce autoconf/automake inputs. > > This handles, for example, "how do I invoke the C compiler?" -- even if using the gcc backend, we need a C compiler/linker/assembler. > > Imagine if, you know, our config files dropped away entirely. > > Imagine if the config file was generated at install time by autoconf. > > Imagine if we didn't have a small amount of litter in our system as to how to link on HP-UX. Or if AIX supported was "inherited" from libtool. > > Basically, libtool supports more than us an overlaps with us. Maybe reuse it. > > > > > > Again, I am somewhat positively and negatively obsessed with "the GNU build system" > > It is slow and bad to author in, but produces easy to use somewhat slow idiomatic results. > > > > > > Also, to reiterate, I believe the point here was the "build server" would be complicated, in order to produce easy to use outputs. > > > > > > One also needs to keep in mind the somewhat opposing but common desires: install from source vs. install from binaries. > > Install from binaries is faster. > > Install from source is problematic for compilers written in themselves. Even gcc. > > > > > > - Jay > > > > > > > > Date: Sun, 16 Aug 2015 13:59:15 -0700 > > From: lists at darko.org > > To: m3devel at elegosoft.com; microcode at zoho.com > > Subject: Re: [M3devel] Build Server - Plan > > > > Just like there are different views on how Linux should be set up and thus different distributions, there are different views on how CM3 should be setup or installed, or at least I have a different view. > > I definitely believe in the "effortlessly and mindlessly" thing, and I want to take it further than the current installer. In particular I want to reduce the external, internal and environmental dependencies so that you can build anything, anywhere with just the minimal set of executables and a standard set of tools. Others want a more elaborate setup which gives them more flexibility; that can also be derived from that same minimal set. > > > > > > On Sun, Aug 16, 2015 at 12:27 AM, wrote: > > On Fri, Aug 14, 2015 at 09:22:08AM -0700, Darko Volaric wrote: > > > > > The solution for SPARC might be to create a cross-compiler. That's not > > > > > ideal but would be useful for verifying that things compile for the > > > > > platform at least, short of having the right hardware. > > > > > > > > You should have access to opencsw as Jay mentioned. If not, I can offer > > > > machine time but I can't keep the machine running all the time. Hopefully > > > > the current setup (opencsw) is still viable. > > > > > > > > > I disagree with your "only for experts" assessment. The point of this > > > > > server is that you don't have to compiler the compiler and a backend just > > > > > to get the latest (or even a properly working) compiler. That's the very > > > > > "expert" work I'm trying to automate. > > > > > > > > That is good to hear because from your list of deliverables it seemed to me > > > > there would still be a lot of tinkering to get a complete install. If you > > > > are scripting that and it can be built effortlessly and mindlessly from the > > > > deliverables then good. I think it would be ideal and very nice if people > > > > could download a turnkey tarball ready to run on their platform, or source > > > > code ready to build and an installer script that installs the final > > > > product. Jay mentioned autotools and they are usually far from optimal on > > > > Solaris because of autotools reliance on so much gnu stuff that is not > > > > always necessarily present on Solaris but that is the general idea- to be > > > > able to configure && make && make install with all the standard stuff > > > > happening like everything being installed in /usr/local/bin|lib|share and so > > > > forth. > > > > > > > > > What this system spits out is everything you need to compile anything you > > > > > want - a complete and working compiler. Not everyone wants or needs to use > > > > > all the packages. If you do, you can. Here are the instructions for doing > > > > > that: "cd ; cm3 " > > > > > > > > It was probably just my unfamiliarity with the build setup and terms. I'm > > > > sorry if I wasted anybody's time with this diversion. I have been following > > > > the mailing list and it looks like Jay and Rodney are mostly discussing > > > > things with a very small number of regulars. Coming from the outside as I do > > > > I think anybody new is going to have similar questions/misunderstandings of > > > > how cm3 is built and installed, etc. > > > > > > > > > I'll be opening up the server to anyone who wants to do something > > > > > different. If someone wants to produce tarballs they can setup their own > > > > > VMs and they will get built too. > > > > > > > > Hopefully this can be integrated with what you already have running on > > > > opencsw. > > > > > > > > Thank you. > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 00:20:25 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 19 Aug 2015 17:20:25 -0500 Subject: [M3devel] cm3 is broken Message-ID: <55D50129.6030600@lcwb.coop> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into runaway recursion trying to report a fault. This happens immediately upon running it, before it produces any output. Here is one cycle of the backtrace: #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 #1748 0x00007fd9b0b0ccfe in Raise (act= RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) at ../src/runtime/common/RTHooks.m3:79 #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 So far, I haven't had the patience to keep hitting return until I get to the bottom of it. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 01:57:54 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 19 Aug 2015 23:57:54 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: Did you upgrade.py? (or possibly upgrade.sh?) I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. I run upgrade.py frequently and recommend it. - Jay > Date: Wed, 19 Aug 2015 17:20:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: [M3devel] cm3 is broken > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 20 11:38:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Thu, 20 Aug 2015 09:38:37 +0000 Subject: [M3devel] First_readable_addr? Message-ID: First_readable_addr looks off by a factor of 8.But I haven't gotten it to trigger at all yet. I'll debug further. Proposals: - remove First_readable_addr as wrong or non functional? - fix First_readable_addr - Most importantly, First_readable_addr is I believe usually 4k, sometimes 8k.It is 8k for sparc. It "should" also be 8k for ia64 and alpha. However, the proposal is to make it always 4k, for more commonality among targets.Lower is safe, but a possible very slight loss in efficiency when it is too low. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Aug 20 17:44:39 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 10:44:39 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> Message-ID: <55D5F5E7.5000109@lcwb.coop> On 08/19/2015 06:57 PM, Jay K wrote: > Did you upgrade.py? (or possibly upgrade.sh?) > upgrade.py does this: rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py Traceback (most recent call last): File "upgrade.py", line 4, in import pylib File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in if Target.startswith("NT386"): AttributeError: 'NoneType' object has no attribute 'startswith' I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', with front removed from the m3cc line in pkginfo.txt, since there were no changes to it, and it takes longer to compile than everything else combined. upgrade.sh now segfaults after cleaning things out, but that is no doubt just the now-broken cm3. I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing a compiler, but restoring what I had before does not fix the problem. I suppose it is in m3core, which I guess I need to add to my backup habit. > I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > I run upgrade.py frequently and recommend it. > > - Jay > > > > Date: Wed, 19 Aug 2015 17:20:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: [M3devel] cm3 is broken > > > > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > runaway recursion trying to report a fault. This happens immediately upon > > running it, before it produces any output. > > > > Here is one cycle of the backtrace: > > > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > #1748 0x00007fd9b0b0ccfe in Raise (act= > > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > at ../src/runtime/common/RTHooks.m3:79 > > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Aug 20 19:19:01 2015 From: jay.krell at cornell.edu (Jay) Date: Thu, 20 Aug 2015 10:19:01 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D5F5E7.5000109@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: Upgrade is failing because cm3 is already broken. I can see about making the error clearer. I often upgrade.py nocleangcc to speed it up. Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. Do you have recourse? Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) - Jay On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > > On 08/19/2015 06:57 PM, Jay K wrote: >> Did you upgrade.py? (or possibly upgrade.sh?) > > upgrade.py does this: > > rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > Traceback (most recent call last): > File "upgrade.py", line 4, in > import pylib > File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > if Target.startswith("NT386"): > AttributeError: 'NoneType' object has no attribute 'startswith' > > I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > with front removed from the m3cc line in pkginfo.txt, since there were no > changes to it, and it takes longer to compile than everything else combined. > > upgrade.sh now segfaults after cleaning things out, but that is no doubt just > the now-broken cm3. > > I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > a compiler, but restoring what I had before does not fix the problem. I suppose > it is in m3core, which I guess I need to add to my backup habit. > >> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> >> I run upgrade.py frequently and recommend it. >> >> - Jay >> >> >> > Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com >> > Subject: [M3devel] cm3 is broken >> > >> > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > runaway recursion trying to report a fault. This happens immediately upon >> > running it, before it produces any output. >> > >> > Here is one cycle of the backtrace: >> > >> > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > #1748 0x00007fd9b0b0ccfe in Raise (act= >> > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > at ../src/runtime/common/RTHooks.m3:79 >> > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 20 20:39:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 20 Aug 2015 13:39:58 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> Message-ID: <55D61EFE.1040302@lcwb.coop> I have some old complete installations. One of them will probably be enough to build a compiler. No time just yet. On 08/20/2015 12:19 PM, Jay wrote: > Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > > I often upgrade.py nocleangcc to speed it up. > > > Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > > m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > > Do you have recourse? > Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > - Jay > > On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >> >> >> On 08/19/2015 06:57 PM, Jay K wrote: >>> Did you upgrade.py? (or possibly upgrade.sh?) >> >> upgrade.py does this: >> >> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> Traceback (most recent call last): >> File "upgrade.py", line 4, in >> import pylib >> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> if Target.startswith("NT386"): >> AttributeError: 'NoneType' object has no attribute 'startswith' >> >> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> with front removed from the m3cc line in pkginfo.txt, since there were no >> changes to it, and it takes longer to compile than everything else combined. >> >> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> the now-broken cm3. >> >> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> a compiler, but restoring what I had before does not fix the problem. I suppose >> it is in m3core, which I guess I need to add to my backup habit. >> >>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>> >>> I run upgrade.py frequently and recommend it. >>> >>> - Jay >>> >>> >>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com >>>> Subject: [M3devel] cm3 is broken >>>> >>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>> runaway recursion trying to report a fault. This happens immediately upon >>>> running it, before it produces any output. >>>> >>>> Here is one cycle of the backtrace: >>>> >>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>> at ../src/runtime/common/RTHooks.m3:79 >>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>> >>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From wagner at elego.de Fri Aug 21 14:46:44 2015 From: wagner at elego.de (Olaf Wagner) Date: Fri, 21 Aug 2015 14:46:44 +0200 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: References: Message-ID: <20150821144644.345ac1a25b429e6c35854d32@elego.de> On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rodney_bates at lcwb.coop Fri Aug 21 18:38:32 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 21 Aug 2015 11:38:32 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> Message-ID: <55D75408.3010900@lcwb.coop> On 08/20/2015 06:32 PM, Antony Hosking wrote: > cm3 should build with static link to m3core so as to allow backup as you describe. > Did this change recently? Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting a breakpoint at one of these places gives: (m3gdb) b RTExFrame.m3:175 No source file named RTExFrame.m3. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (RTExFrame.m3:175) pending. (m3gdb) run ........ Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. Pending breakpoint "RTExFrame.m3:175" resolved > >> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> >> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> >> >> I often upgrade.py nocleangcc to speed it up. >> >> >> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> >> >> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> >> >> Do you have recourse? >> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> >> - Jay >> >> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> >>> >>> >>> On 08/19/2015 06:57 PM, Jay K wrote: >>>> Did you upgrade.py? (or possibly upgrade.sh?) >>> >>> upgrade.py does this: >>> >>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >>> Traceback (most recent call last): >>> File "upgrade.py", line 4, in >>> import pylib >>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >>> if Target.startswith("NT386"): >>> AttributeError: 'NoneType' object has no attribute 'startswith' >>> >>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >>> with front removed from the m3cc line in pkginfo.txt, since there were no >>> changes to it, and it takes longer to compile than everything else combined. >>> >>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >>> the now-broken cm3. >>> >>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >>> a compiler, but restoring what I had before does not fix the problem. I suppose >>> it is in m3core, which I guess I need to add to my backup habit. >>> >>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >>>> >>>> I run upgrade.py frequently and recommend it. >>>> >>>> - Jay >>>> >>>> >>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >>>>> From: rodney_bates at lcwb.coop >>>>> To: m3devel at elegosoft.com >>>>> Subject: [M3devel] cm3 is broken >>>>> >>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>> running it, before it produces any output. >>>>> >>>>> Here is one cycle of the backtrace: >>>>> >>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>> >>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>> >>>>> -- >>>>> Rodney Bates >>>>> rodney.m.bates at acm.org >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 11:28:01 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 09:28:01 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 12:00:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 10:00:18 +0000 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: It works for me. Can you go back to a working install, maybe 5.8.6 with that slight config patch,run upgrade.py, add see what you get?Send the build log maybe? jair:python jay$ otool -L /cm3/bin/cm3/cm3/bin/cm3: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) jair:python jay$ /cm3/bin/cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-22 09:57:19 configuration: /cm3/bin/cm3.cfg host: I386_DARWIN target: I386_DARWIN On most other systems, use ldd.On Windows use link /dump /imports or link /dump /dependents. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu; rodney.m.bates at acm.org Date: Sat, 22 Aug 2015 09:28:01 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] cm3 is broken This was not meant to change and I haven't seen that it did change.Maybe my cleaning up of the config files. I'll check. I understand the risks well. I have a potential plan to change it. But that is another matter. You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. The wrong uprade sequence still breaks cm3. That is at it should be. Tangentially, here is my plan: The upgrade process must be "to the side and switch all at once", not "in place". It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all into the install in tight sequence (not atomically). The trick on NT would be to unmap the executable, and avoid returning to it -- some sort of "continuation passing style", possibly off into a little bit of C++, whose main task would be to move a few files and then exit. That'd still leave all the platforms to test on to see if a running executable can be replaced. esp. including AIX. Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of thisand just leave it as an idea/plan that I don't get to - Jay > Subject: Re: [M3devel] cm3 is broken > From: hosking at purdue.edu > Date: Sat, 22 Aug 2015 06:48:15 +1000 > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > To: rodney.m.bates at acm.org > > That is a regression. > > Sent from my iPad > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > >> cm3 should build with static link to m3core so as to allow backup as you describe. > >> Did this change recently? > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > a breakpoint at one of these places gives: > > > > > > (m3gdb) b RTExFrame.m3:175 > > No source file named RTExFrame.m3. > > Make breakpoint pending on future shared library load? (y or [n]) y > > Breakpoint 1 (RTExFrame.m3:175) pending. > > (m3gdb) run > > > > ........ > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > >>> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > >>> > >>> > >>> I often upgrade.py nocleangcc to speed it up. > >>> > >>> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > >>> > >>> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > >>> > >>> > >>> Do you have recourse? > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > >>> > >>> - Jay > >>> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > >>>> > >>>> > >>>> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > >>>> > >>>> upgrade.py does this: > >>>> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > >>>> Traceback (most recent call last): > >>>> File "upgrade.py", line 4, in > >>>> import pylib > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > >>>> if Target.startswith("NT386"): > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > >>>> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > >>>> changes to it, and it takes longer to compile than everything else combined. > >>>> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > >>>> the now-broken cm3. > >>>> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > >>>> it is in m3core, which I guess I need to add to my backup habit. > >>>> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > >>>>> > >>>>> I run upgrade.py frequently and recommend it. > >>>>> > >>>>> - Jay > >>>>> > >>>>> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > >>>>>> From: rodney_bates at lcwb.coop > >>>>>> To: m3devel at elegosoft.com > >>>>>> Subject: [M3devel] cm3 is broken > >>>>>> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>>> -- > >>>>>> Rodney Bates > >>>>>> rodney.m.bates at acm.org > >>>>>> _______________________________________________ > >>>>>> M3devel mailing list > >>>>>> M3devel at elegosoft.com > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Aug 22 17:44:04 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 10:44:04 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, , , <55D5F5E7.5000109@lcwb.coop>, , , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, , <55D75408.3010900@lcwb.coop>, , <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu>, Message-ID: <55D898C4.90305@lcwb.coop> Going back several cm3 backups until I see a change, I have: root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 linux-vdso.so.1 => (0x00007fff211fe000) libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-07-31 18:44:16 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 linux-vdso.so.1 => (0x00007fffa47a7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-06-04 14:24:48 configuration: ./cm3.cfg host: AMD64_LINUX target: AMD64_LINUX So I compiled the change to dynamic between June 4 and July 31. On 08/22/2015 05:00 AM, Jay K wrote: > It works for me. > > Can you go back to a working install, maybe 5.8.6 with that slight config patch, > run upgrade.py, add see what you get? > Send the build log maybe? > > > jair:python jay$ otool -L /cm3/bin/cm3 > /cm3/bin/cm3: > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) > /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) > > jair:python jay$ /cm3/bin/cm3 -version > > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-22 09:57:19 > configuration: /cm3/bin/cm3.cfg > host: I386_DARWIN > target: I386_DARWIN > > > On most other systems, use ldd. > On Windows use link /dump /imports or link /dump /dependents. > > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: hosking at purdue.edu; rodney.m.bates at acm.org > Date: Sat, 22 Aug 2015 09:28:01 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken > > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:07:01 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:07:01 -0500 Subject: [M3devel] cm3 is broken In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D5F5E7.5000109@lcwb.coop>, , <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu>, <55D75408.3010900@lcwb.coop>, <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> Message-ID: <55D89E25.9000002@lcwb.coop> On 08/22/2015 04:28 AM, Jay K wrote: > This was not meant to change and I haven't seen that it did change. > Maybe my cleaning up of the config files. I'll check. > > > I understand the risks well. I have a potential plan to change it. But that is another matter. > > You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. > > The wrong uprade sequence still breaks cm3. That is at it should be. > > > Tangentially, here is my plan: > The upgrade process must be "to the side and switch all at once", not "in place". I think this is a very good idea. The in-place rebuilds always make me very nervous. I never feel like I quite know what is going on, and I never quite trust any of the processes we have (and rightly so, from experience). When it works, or appears to, it always is more a matter of experiment than understanding. Whenever I am the least bit suspicious, I feel the need to copy two whole subtrees (source & installed) to update in place, keeping the originals to make another copy from, if problems arise. But that's a pretty tedious process to go through every time, so I have to evaluate the risks and decide when to take them and when to play safe. And when I touch the compiler or runtime, I feel the need to rebuild all of front twice, by whatever process I am using, then look for at least identical sized executables and libraries. > It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all > into the install in tight sequence (not atomically). The trick on NT would be to unmap the > executable, and avoid returning to it -- some sort of "continuation passing style", > possibly off into a little bit of C++, whose main task would be to move a few files > and then exit. > > > That'd still leave all the platforms to test on to see if a running executable can be replaced. > esp. including AIX. > > > Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this > and just leave it as an idea/plan that I don't get to > > - Jay > > > > > > Subject: Re: [M3devel] cm3 is broken > > From: hosking at purdue.edu > > Date: Sat, 22 Aug 2015 06:48:15 +1000 > > CC: jay.krell at cornell.edu; m3devel at elegosoft.com > > To: rodney.m.bates at acm.org > > > > That is a regression. > > > > Sent from my iPad > > > > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: > > > > > > > > > > > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: > > >> cm3 should build with static link to m3core so as to allow backup as you describe. > > >> Did this change recently? > > > > > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting > > > a breakpoint at one of these places gives: > > > > > > > > > (m3gdb) b RTExFrame.m3:175 > > > No source file named RTExFrame.m3. > > > Make breakpoint pending on future shared library load? (y or [n]) y > > > Breakpoint 1 (RTExFrame.m3:175) pending. > > > (m3gdb) run > > > > > > ........ > > > > > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. > > > Pending breakpoint "RTExFrame.m3:175" resolved > > > > > > > > > > > >> > > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: > > >>> > > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. > > >>> > > >>> > > >>> I often upgrade.py nocleangcc to speed it up. > > >>> > > >>> > > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. > > >>> > > >>> > > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. > > >>> > > >>> > > >>> Do you have recourse? > > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) > > >>> > > >>> - Jay > > >>> > > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > > >>>> > > >>>> > > >>>> > > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: > > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) > > >>>> > > >>>> upgrade.py does this: > > >>>> > > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py > > >>>> Traceback (most recent call last): > > >>>> File "upgrade.py", line 4, in > > >>>> import pylib > > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in > > >>>> if Target.startswith("NT386"): > > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' > > >>>> > > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', > > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no > > >>>> changes to it, and it takes longer to compile than everything else combined. > > >>>> > > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just > > >>>> the now-broken cm3. > > >>>> > > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing > > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose > > >>>> it is in m3core, which I guess I need to add to my backup habit. > > >>>> > > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. > > >>>>> > > >>>>> I run upgrade.py frequently and recommend it. > > >>>>> > > >>>>> - Jay > > >>>>> > > >>>>> > > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 > > >>>>>> From: rodney_bates at lcwb.coop > > >>>>>> To: m3devel at elegosoft.com > > >>>>>> Subject: [M3devel] cm3 is broken > > >>>>>> > > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > > >>>>>> running it, before it produces any output. > > >>>>>> > > >>>>>> Here is one cycle of the backtrace: > > >>>>>> > > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >>>>>> > > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >>>>>> > > >>>>>> -- > > >>>>>> Rodney Bates > > >>>>>> rodney.m.bates at acm.org > > >>>>>> _______________________________________________ > > >>>>>> M3devel mailing list > > >>>>>> M3devel at elegosoft.com > > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>> -- > > >>>> Rodney Bates > > >>>> rodney.m.bates at acm.org > > >>> _______________________________________________ > > >>> M3devel mailing list > > >>> M3devel at elegosoft.com > > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >> > > >> > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:33:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:33:12 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D50129.6030600@lcwb.coop> References: <55D50129.6030600@lcwb.coop> Message-ID: <55D8A448.30703@lcwb.coop> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = VAR p := LOOPHOLE (f, PF1); BEGIN IF DEBUG THEN PutExcept ("INVOKE HANDLER", a); RTIO.PutText (" frame="); RTIO.PutAddr (f); RTIO.PutText (" class="); RTIO.PutInt (f.class); RTIO.PutText ("\n"); RTIO.Flush (); END; RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) p.info := a; (* copy the exception to the new frame *) <* ASSERT p.jmpbuf # NIL *> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) RAISE OUCH; END InvokeHandler; (m3gdb) frame 0 #0 InvokeHandler (f=16_00007fff509f4af0, a= RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 175 <* ASSERT p.jmpbuf # NIL *> (m3gdb) p p.jmpbuf $31 = NIL --------^^^ (m3gdb) The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = VAR status: File.Status; fname := M3toC.SharedTtoS(pn); BEGIN IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; ---------^^^^^^^^^^^^^^^^^^^^^^ M3toC.FreeSharedS(pn, fname); RETURN status END Status; (m3gdb) #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p pn $30 = (*16_00000000013e4ac0*) "./cm3.cfg" --------------------------------^^^^^^^^^^^ (m3gdb) I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? Also, I notice one other thing, that probably isn't part of this problem, but maybe needs to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, after the failing CStatus above. Obviously not thread safe. Does it matter here? As I recall from many years ago, there was a change in C library that provided a thread safe alternative to this, and I even think I had to change something in pm3 or SRC to adapt to it. Do we care? PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = VAR err := Cerrno.GetErrno(); ---------------^^^^^^^^^^^^^^^^^ BEGIN M3toC.FreeSharedS(p, f); OSErrorPosix.Raise0(err); END Fail; On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > runaway recursion trying to report a fault. This happens immediately upon > running it, before it produces any output. > > Here is one cycle of the backtrace: > > #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > #1748 0x00007fd9b0b0ccfe in Raise (act= > RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > at ../src/runtime/common/RTHooks.m3:79 > #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 18:41:14 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 11:41:14 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A448.30703@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> Message-ID: <55D8A62A.8030205@lcwb.coop> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > VAR p := LOOPHOLE (f, PF1); > BEGIN > IF DEBUG THEN > PutExcept ("INVOKE HANDLER", a); > RTIO.PutText (" frame="); RTIO.PutAddr (f); > RTIO.PutText (" class="); RTIO.PutInt (f.class); > RTIO.PutText ("\n"); > RTIO.Flush (); > END; > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > p.info := a; (* copy the exception to the new frame *) > <* ASSERT p.jmpbuf # NIL *> > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > RAISE OUCH; > END InvokeHandler; > > (m3gdb) frame 0 > #0 InvokeHandler (f=16_00007fff509f4af0, a= > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > 175 <* ASSERT p.jmpbuf # NIL *> > (m3gdb) p p.jmpbuf > $31 = NIL > --------^^^ > (m3gdb) > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > BEGIN > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > ---------^^^^^^^^^^^^^^^^^^^^^^ > M3toC.FreeSharedS(pn, fname); > RETURN status > END Status; > > (m3gdb) > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p pn > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > --------------------------------^^^^^^^^^^^ > (m3gdb) > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? A bit more info: (m3gdb) down #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) at ../src/os/POSIX/FSPosix.m3:328 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; (m3gdb) p Process.GetWorkingDirectory() $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (m3gdb) This is where I ran cm3 from. > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > after the failing CStatus above. Obviously not thread safe. Does it matter here? > As I recall from many years ago, there was a change in C library that provided a thread > safe alternative to this, and I even think I had to change something in pm3 or SRC to > adapt to it. Do we care? > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > VAR err := Cerrno.GetErrno(); > ---------------^^^^^^^^^^^^^^^^^ > BEGIN > M3toC.FreeSharedS(p, f); > OSErrorPosix.Raise0(err); > END Fail; > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> runaway recursion trying to report a fault. This happens immediately upon >> running it, before it produces any output. >> >> Here is one cycle of the backtrace: >> >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> at ../src/runtime/common/RTHooks.m3:79 >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 19:26:33 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 22 Aug 2015 17:26:33 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> Message-ID: It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) The interface here between cm3's generated code/data and m3core has changed. You cannot mix them. It will fail. - Jay > Date: Sat, 22 Aug 2015 11:41:14 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] cm3 is broken: More info > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > VAR p := LOOPHOLE (f, PF1); > > BEGIN > > IF DEBUG THEN > > PutExcept ("INVOKE HANDLER", a); > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > RTIO.PutText ("\n"); > > RTIO.Flush (); > > END; > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > p.info := a; (* copy the exception to the new frame *) > > <* ASSERT p.jmpbuf # NIL *> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > RAISE OUCH; > > END InvokeHandler; > > > > (m3gdb) frame 0 > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > 175 <* ASSERT p.jmpbuf # NIL *> > > (m3gdb) p p.jmpbuf > > $31 = NIL > > --------^^^ > > (m3gdb) > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > BEGIN > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > M3toC.FreeSharedS(pn, fname); > > RETURN status > > END Status; > > > > (m3gdb) > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p pn > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > --------------------------------^^^^^^^^^^^ > > (m3gdb) > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > As I recall from many years ago, there was a change in C library that provided a thread > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > adapt to it. Do we care? > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > VAR err := Cerrno.GetErrno(); > > ---------------^^^^^^^^^^^^^^^^^ > > BEGIN > > M3toC.FreeSharedS(p, f); > > OSErrorPosix.Raise0(err); > > END Fail; > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >> runaway recursion trying to report a fault. This happens immediately upon > >> running it, before it produces any output. > >> > >> Here is one cycle of the backtrace: > >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >> at ../src/runtime/common/RTHooks.m3:79 > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >> > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Aug 22 20:47:00 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 11:47:00 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8A62A.8030205@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> Message-ID: On the other matter -- Cerrno.GetErrno() is thread safe. It ends up as a function that accesses a thread local. At least on Windows and surely that is the only logical approach. It is still a bad design. Thread locals don't make things all better. They are still "fragile" (easy to change by accident), slow, and don't provide for reentrance. - Jay On Aug 22, 2015, at 9:41 AM, "Rodney M. Bates" wrote: > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> >> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> VAR p := LOOPHOLE (f, PF1); >> BEGIN >> IF DEBUG THEN >> PutExcept ("INVOKE HANDLER", a); >> RTIO.PutText (" frame="); RTIO.PutAddr (f); >> RTIO.PutText (" class="); RTIO.PutInt (f.class); >> RTIO.PutText ("\n"); >> RTIO.Flush (); >> END; >> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> p.info := a; (* copy the exception to the new frame *) >> <* ASSERT p.jmpbuf # NIL *> >> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> RAISE OUCH; >> END InvokeHandler; >> >> (m3gdb) frame 0 >> #0 InvokeHandler (f=16_00007fff509f4af0, a= >> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> 175 <* ASSERT p.jmpbuf # NIL *> >> (m3gdb) p p.jmpbuf >> $31 = NIL >> --------^^^ >> (m3gdb) >> >> >> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> >> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> BEGIN >> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> ---------^^^^^^^^^^^^^^^^^^^^^^ >> M3toC.FreeSharedS(pn, fname); >> RETURN status >> END Status; >> >> (m3gdb) >> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> at ../src/os/POSIX/FSPosix.m3:328 >> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> (m3gdb) p pn >> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> --------------------------------^^^^^^^^^^^ >> (m3gdb) >> >> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > A bit more info: > > (m3gdb) down > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > at ../src/os/POSIX/FSPosix.m3:328 > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > (m3gdb) p Process.GetWorkingDirectory() > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (m3gdb) > > This is where I ran cm3 from. > > >> >> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> after the failing CStatus above. Obviously not thread safe. Does it matter here? >> As I recall from many years ago, there was a change in C library that provided a thread >> safe alternative to this, and I even think I had to change something in pm3 or SRC to >> adapt to it. Do we care? >> >> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> VAR err := Cerrno.GetErrno(); >> ---------------^^^^^^^^^^^^^^^^^ >> BEGIN >> M3toC.FreeSharedS(p, f); >> OSErrorPosix.Raise0(err); >> END Fail; >> >> >> >> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>> runaway recursion trying to report a fault. This happens immediately upon >>> running it, before it produces any output. >>> >>> Here is one cycle of the backtrace: >>> >>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>> at ../src/runtime/common/RTHooks.m3:79 >>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>> >>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Aug 22 23:50:42 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 14:50:42 -0700 Subject: [M3devel] cm3 is broken In-Reply-To: <55D898C4.90305@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D5F5E7.5000109@lcwb.coop> <9C70E2F3-5D39-4D03-A93F-6B6B66AB43CC@purdue.edu> <55D75408.3010900@lcwb.coop> <9645D481-DDDD-4278-8BFC-4995139E88D2@purdue.edu> <55D898C4.90305@lcwb.coop> Message-ID: <2A7A1326-A801-4BFF-8C1D-4E50DB646272@gmail.com> Go back to one that is statically linked and upgrade from that? upgrade.py nocleangcc doesn't take too long. The backup must have, in general, cm3 cm3cg config libm3 m3core. We can relax the libm3 requirement I believe, but let's deal with that later separately. - Jay On Aug 22, 2015, at 8:44 AM, "Rodney M. Bates" wrote: > Going back several cm3 backups until I see a change, I have: > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak22 > linux-vdso.so.1 => (0x00007fff211fe000) > libsysutils.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libsysutils.so.5 (0x00007f9fed722000) > libm3.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3.so.5 (0x00007f9fed394000) > libm3core.so.5 => /usr/local/cm3-uniboot/bin/./../lib/libm3core.so.5 (0x00007f9fec8c2000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9fec685000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9fec2bc000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9febfb8000) > /lib64/ld-linux-x86-64.so.2 (0x00007f9fed957000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak22 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-07-31 18:44:16 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > root at allegheny:/usr/local/cm3/bin# ldd cm3.bak21 > linux-vdso.so.1 => (0x00007fffa47a7000) > libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7beee7d000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7beec5f000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7bee897000) > /lib64/ld-linux-x86-64.so.2 (0x00007f7bef1a3000) > root at allegheny:/usr/local/cm3/bin# ./cm3.bak21 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-06-04 14:24:48 > configuration: ./cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > So I compiled the change to dynamic between June 4 and July 31. > > > On 08/22/2015 05:00 AM, Jay K wrote: >> It works for me. >> >> Can you go back to a working install, maybe 5.8.6 with that slight config patch, >> run upgrade.py, add see what you get? >> Send the build log maybe? >> >> >> jair:python jay$ otool -L /cm3/bin/cm3 >> /cm3/bin/cm3: >> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) >> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0) >> >> jair:python jay$ /cm3/bin/cm3 -version >> >> Critical Mass Modula-3 version d5.10.0 >> last updated: 2015-05-21 >> compiled: 2015-08-22 09:57:19 >> configuration: /cm3/bin/cm3.cfg >> host: I386_DARWIN >> target: I386_DARWIN >> >> >> On most other systems, use ldd. >> On Windows use link /dump /imports or link /dump /dependents. >> >> >> - Jay >> >> >> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- >> From: jay.krell at cornell.edu >> To: hosking at purdue.edu; rodney.m.bates at acm.org >> Date: Sat, 22 Aug 2015 09:28:01 +0000 >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 is broken >> >> This was not meant to change and I haven't seen that it did change. >> Maybe my cleaning up of the config files. I'll check. >> >> >> I understand the risks well. I have a potential plan to change it. But that is another matter. >> >> You still need to backup libm3/m3core with cm3 -- so that cm3 can build anything that works. >> >> The wrong uprade sequence still breaks cm3. That is at it should be. >> >> >> Tangentially, here is my plan: >> The upgrade process must be "to the side and switch all at once", not "in place". >> It might go over and back though -- i.e. cm3 can move a new cm3, m3core, and libm3 all >> into the install in tight sequence (not atomically). The trick on NT would be to unmap the >> executable, and avoid returning to it -- some sort of "continuation passing style", >> possibly off into a little bit of C++, whose main task would be to move a few files >> and then exit. >> >> >> That'd still leave all the platforms to test on to see if a running executable can be replaced. >> esp. including AIX. >> >> >> Perhaps it isn't worth it. I have at least endeavored to priortize everything else ahead of this >> and just leave it as an idea/plan that I don't get to >> >> - Jay >> >> >> >> >> > Subject: Re: [M3devel] cm3 is broken >> > From: hosking at purdue.edu >> > Date: Sat, 22 Aug 2015 06:48:15 +1000 >> > CC: jay.krell at cornell.edu; m3devel at elegosoft.com >> > To: rodney.m.bates at acm.org >> > >> > That is a regression. >> > >> > Sent from my iPad >> > >> > > On 22 Aug 2015, at 2:38 am, Rodney M. Bates wrote: >> > > >> > > >> > > >> > >> On 08/20/2015 06:32 PM, Antony Hosking wrote: >> > >> cm3 should build with static link to m3core so as to allow backup as you describe. >> > >> Did this change recently? >> > > >> > > Apparently it is now dynamic, as I see libm3core.so.5 in the backtrace. Also, setting >> > > a breakpoint at one of these places gives: >> > > >> > > >> > > (m3gdb) b RTExFrame.m3:175 >> > > No source file named RTExFrame.m3. >> > > Make breakpoint pending on future shared library load? (y or [n]) y >> > > Breakpoint 1 (RTExFrame.m3:175) pending. >> > > (m3gdb) run >> > > >> > > ........ >> > > >> > > Breakpoint 2 at 0x7f446c889043: file ../src/runtime/ex_frame/RTExFrame.m3, line 175. >> > > Pending breakpoint "RTExFrame.m3:175" resolved >> > > >> > > >> > > >> > >> >> > >>> On Aug 21, 2015, at 3:19 AM, Jay wrote: >> > >>> >> > >>> Upgrade is failing because cm3 is already broken. I can see about making the error clearer. >> > >>> >> > >>> >> > >>> I often upgrade.py nocleangcc to speed it up. >> > >>> >> > >>> >> > >>> Right, backup must include libm3 m3core cm3cg cm3 config. See capture-min or capture-boot, I think I settled on capture-min. >> > >>> >> > >>> >> > >>> m3cc either needs to keep the "front" tag or the scripts need to mention it explicitly. You can't safely always skip it. >> > >>> >> > >>> >> > >>> Do you have recourse? >> > >>> Old release maybe? (Be sure to edit the 5.8.6 config to avoid new cm3cg.) >> > >>> >> > >>> - Jay >> > >>> >> > >>>> On Aug 20, 2015, at 8:44 AM, "Rodney M. Bates" wrote: >> > >>>> >> > >>>> >> > >>>> >> > >>>>> On 08/19/2015 06:57 PM, Jay K wrote: >> > >>>>> Did you upgrade.py? (or possibly upgrade.sh?) >> > >>>> >> > >>>> upgrade.py does this: >> > >>>> >> > >>>> rodney at allegheny:~/proj/m3/git/cm3/scripts/python$ python upgrade.py >> > >>>> Traceback (most recent call last): >> > >>>> File "upgrade.py", line 4, in >> > >>>> import pylib >> > >>>> File "/home/rodney/proj/m3/git/cm3/scripts/python/pylib.py", line 656, in >> > >>>> if Target.startswith("NT386"): >> > >>>> AttributeError: 'NoneType' object has no attribute 'startswith' >> > >>>> >> > >>>> I did 'do-cm3-front.sh' buildship' and 'install-cm3-compiler.sh upgrade', >> > >>>> with front removed from the m3cc line in pkginfo.txt, since there were no >> > >>>> changes to it, and it takes longer to compile than everything else combined. >> > >>>> >> > >>>> upgrade.sh now segfaults after cleaning things out, but that is no doubt just >> > >>>> the now-broken cm3. >> > >>>> >> > >>>> I always make backup copies of cm3 & cm3cg in /usr/local/cm3/bin before installing >> > >>>> a compiler, but restoring what I had before does not fix the problem. I suppose >> > >>>> it is in m3core, which I guess I need to add to my backup habit. >> > >>>> >> > >>>>> I changed the cm3/m3core interface, so you must do the ugprade in the usual proper order. >> > >>>>> >> > >>>>> I run upgrade.py frequently and recommend it. >> > >>>>> >> > >>>>> - Jay >> > >>>>> >> > >>>>> >> > >>>>>> Date: Wed, 19 Aug 2015 17:20:25 -0500 >> > >>>>>> From: rodney_bates at lcwb.coop >> > >>>>>> To: m3devel at elegosoft.com >> > >>>>>> Subject: [M3devel] cm3 is broken >> > >>>>>> >> > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >>>>>> runaway recursion trying to report a fault. This happens immediately upon >> > >>>>>> running it, before it produces any output. >> > >>>>>> >> > >>>>>> Here is one cycle of the backtrace: >> > >>>>>> >> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >>>>>> at ../src/runtime/common/RTHooks.m3:79 >> > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >>>>>> >> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >>>>>> >> > >>>>>> -- >> > >>>>>> Rodney Bates >> > >>>>>> rodney.m.bates at acm.org >> > >>>>>> _______________________________________________ >> > >>>>>> M3devel mailing list >> > >>>>>> M3devel at elegosoft.com >> > >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >>>> >> > >>>> -- >> > >>>> Rodney Bates >> > >>>> rodney.m.bates at acm.org >> > >>> _______________________________________________ >> > >>> M3devel mailing list >> > >>> M3devel at elegosoft.com >> > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > >> >> > >> >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> >> _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 22 23:54:26 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 22 Aug 2015 16:54:26 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop> Message-ID: <55D8EF92.20403@lcwb.coop> On 08/22/2015 12:26 PM, Jay K wrote: > It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > > > Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run independently. After a good lot of successful-looking output, I get: new source -> compiling LLGen.i3 new source -> compiling LLGen.m3 new source -> compiling Version.i3 new source -> compiling M3Backend.i3 new source -> compiling Arg.i3 new source -> compiling Utils.i3 new source -> compiling Msg.i3 new source -> compiling M3Backend.m3 new source -> compiling UtilsPosix.m3 new source -> compiling Arg.m3 new source -> compiling M3Loc.i3 new source -> compiling M3Unit.i3 new source -> compiling Builder.i3 new source -> compiling M3Options.i3 new source -> compiling WebFile.i3 new source -> compiling Dirs.i3 new source -> compiling Builder.m3 new source -> compiling Dirs.m3 new source -> compiling M3Build.i3 new source -> compiling M3Build.m3 new source -> compiling M3Loc.m3 new source -> compiling M3Options.m3 new source -> compiling M3Unit.m3 new source -> compiling Makefile.i3 new source -> compiling Makefile.m3 new source -> compiling Msg.m3 new source -> compiling Utils.m3 new source -> compiling WebFile.m3 new source -> compiling Main.m3 "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure 1 warning encountered new source -> compiling cm3unix.c new exporters -> recompiling Utils.i3 -> linking cm3 --- shipping from AMD64_LINUX --- . => /usr/local/cm3/pkg/cm3/AMD64_LINUX .M3EXPORTS .M3WEB ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX cm3unix.c Version.i3 ../src => /usr/local/cm3/pkg/cm3/src M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 Arg.m3 Builder.m3 Builder.i3 Dirs.i3 Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 Msg.m3 Utils.m3 Utils.i3 WebFile.i3 WebFile.m3 Main.m3 ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy LLGen.m3 LLGen.i3 . => /usr/local/cm3/bin cm3 . => /usr/local/cm3/man/man1 cm3.1 . => /usr/local/cm3/man/man7 m3makefile.7 Segmentation fault (core dumped) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done ------------------------------------------------------------------------------------- several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: ------------------------------------------------------------------------------------- /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ *** execution of [, ] failed *** > The interface here between cm3's generated code/data and m3core has changed. > You cannot mix them. It will fail. > > > - Jay > > > > > > Date: Sat, 22 Aug 2015 11:41:14 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: Re: [M3devel] cm3 is broken: More info > > > > > > > > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > > > > > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > > > VAR p := LOOPHOLE (f, PF1); > > > BEGIN > > > IF DEBUG THEN > > > PutExcept ("INVOKE HANDLER", a); > > > RTIO.PutText (" frame="); RTIO.PutAddr (f); > > > RTIO.PutText (" class="); RTIO.PutInt (f.class); > > > RTIO.PutText ("\n"); > > > RTIO.Flush (); > > > END; > > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > > > p.info := a; (* copy the exception to the new frame *) > > > <* ASSERT p.jmpbuf # NIL *> > > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > > > RAISE OUCH; > > > END InvokeHandler; > > > > > > (m3gdb) frame 0 > > > #0 InvokeHandler (f=16_00007fff509f4af0, a= > > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > > 175 <* ASSERT p.jmpbuf # NIL *> > > > (m3gdb) p p.jmpbuf > > > $31 = NIL > > > --------^^^ > > > (m3gdb) > > > > > > > > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > > > > > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > > > BEGIN > > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > ---------^^^^^^^^^^^^^^^^^^^^^^ > > > M3toC.FreeSharedS(pn, fname); > > > RETURN status > > > END Status; > > > > > > (m3gdb) > > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > > at ../src/os/POSIX/FSPosix.m3:328 > > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > > (m3gdb) p pn > > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > > > --------------------------------^^^^^^^^^^^ > > > (m3gdb) > > > > > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > > > > A bit more info: > > > > (m3gdb) down > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > > at ../src/os/POSIX/FSPosix.m3:328 > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > > (m3gdb) p Process.GetWorkingDirectory() > > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > (m3gdb) > > > > This is where I ran cm3 from. > > > > > > > > > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > > > after the failing CStatus above. Obviously not thread safe. Does it matter here? > > > As I recall from many years ago, there was a change in C library that provided a thread > > > safe alternative to this, and I even think I had to change something in pm3 or SRC to > > > adapt to it. Do we care? > > > > > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > > > VAR err := Cerrno.GetErrno(); > > > ---------------^^^^^^^^^^^^^^^^^ > > > BEGIN > > > M3toC.FreeSharedS(p, f); > > > OSErrorPosix.Raise0(err); > > > END Fail; > > > > > > > > > > > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > > >> runaway recursion trying to report a fault. This happens immediately upon > > >> running it, before it produces any output. > > >> > > >> Here is one cycle of the backtrace: > > >> > > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > > >> #1748 0x00007fd9b0b0ccfe in Raise (act= > > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > > >> at ../src/runtime/common/RTHooks.m3:79 > > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > > >> > > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > > >> > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 23 01:37:19 2015 From: jay.krell at cornell.edu (Jay) Date: Sat, 22 Aug 2015 16:37:19 -0700 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55D8EF92.20403@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> Message-ID: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. I upgraded multiple installations on multiple targets. I'll try again from 5.8.6. I suggest upgrade also check for the bad dynamic linking. Anyone else able/unable to upgrade? - Jay On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > On 08/22/2015 12:26 PM, Jay K wrote: >> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >> >> >> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > > OK, I did that. Got a working cm3 going, backed up my source/build directories and install > directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > independently. > > After a good lot of successful-looking output, I get: > > new source -> compiling LLGen.i3 > new source -> compiling LLGen.m3 > new source -> compiling Version.i3 > new source -> compiling M3Backend.i3 > new source -> compiling Arg.i3 > new source -> compiling Utils.i3 > new source -> compiling Msg.i3 > new source -> compiling M3Backend.m3 > new source -> compiling UtilsPosix.m3 > new source -> compiling Arg.m3 > new source -> compiling M3Loc.i3 > new source -> compiling M3Unit.i3 > new source -> compiling Builder.i3 > new source -> compiling M3Options.i3 > new source -> compiling WebFile.i3 > new source -> compiling Dirs.i3 > new source -> compiling Builder.m3 > new source -> compiling Dirs.m3 > new source -> compiling M3Build.i3 > new source -> compiling M3Build.m3 > new source -> compiling M3Loc.m3 > new source -> compiling M3Options.m3 > new source -> compiling M3Unit.m3 > new source -> compiling Makefile.i3 > new source -> compiling Makefile.m3 > new source -> compiling Msg.m3 > new source -> compiling Utils.m3 > new source -> compiling WebFile.m3 > new source -> compiling Main.m3 > "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > 1 warning encountered > new source -> compiling cm3unix.c > new exporters -> recompiling Utils.i3 > -> linking cm3 > --- shipping from AMD64_LINUX --- > > . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > .M3EXPORTS .M3WEB > ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > cm3unix.c Version.i3 > ../src => /usr/local/cm3/pkg/cm3/src > M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > WebFile.m3 Main.m3 > ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > LLGen.m3 LLGen.i3 > . => /usr/local/cm3/bin > cm3 > . => /usr/local/cm3/man/man1 > cm3.1 > . => /usr/local/cm3/man/man7 > m3makefile.7 > Segmentation fault (core dumped) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > > > ------------------------------------------------------------------------------------- > several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > ------------------------------------------------------------------------------------- > > > /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > > +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > *** execution of [, ] failed *** > > > > > > > > > >> The interface here between cm3's generated code/data and m3core has changed. >> You cannot mix them. It will fail. >> >> >> - Jay >> >> >> >> >> > Date: Sat, 22 Aug 2015 11:41:14 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: Re: [M3devel] cm3 is broken: More info >> > >> > >> > >> > On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >> > > There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >> > > >> > > PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >> > > VAR p := LOOPHOLE (f, PF1); >> > > BEGIN >> > > IF DEBUG THEN >> > > PutExcept ("INVOKE HANDLER", a); >> > > RTIO.PutText (" frame="); RTIO.PutAddr (f); >> > > RTIO.PutText (" class="); RTIO.PutInt (f.class); >> > > RTIO.PutText ("\n"); >> > > RTIO.Flush (); >> > > END; >> > > RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >> > > p.info := a; (* copy the exception to the new frame *) >> > > <* ASSERT p.jmpbuf # NIL *> >> > > ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >> > > Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >> > > RAISE OUCH; >> > > END InvokeHandler; >> > > >> > > (m3gdb) frame 0 >> > > #0 InvokeHandler (f=16_00007fff509f4af0, a= >> > > RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > > 175 <* ASSERT p.jmpbuf # NIL *> >> > > (m3gdb) p p.jmpbuf >> > > $31 = NIL >> > > --------^^^ >> > > (m3gdb) >> > > >> > > >> > > The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >> > > >> > > PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >> > > VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >> > > BEGIN >> > > IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > ---------^^^^^^^^^^^^^^^^^^^^^^ >> > > M3toC.FreeSharedS(pn, fname); >> > > RETURN status >> > > END Status; >> > > >> > > (m3gdb) >> > > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > > at ../src/os/POSIX/FSPosix.m3:328 >> > > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > > (m3gdb) p pn >> > > $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >> > > --------------------------------^^^^^^^^^^^ >> > > (m3gdb) >> > > >> > > I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >> > >> > A bit more info: >> > >> > (m3gdb) down >> > #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >> > at ../src/os/POSIX/FSPosix.m3:328 >> > 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >> > (m3gdb) p Process.GetWorkingDirectory() >> > $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >> > --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> > (m3gdb) >> > >> > This is where I ran cm3 from. >> > >> > >> > > >> > > Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >> > > to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >> > > after the failing CStatus above. Obviously not thread safe. Does it matter here? >> > > As I recall from many years ago, there was a change in C library that provided a thread >> > > safe alternative to this, and I even think I had to change something in pm3 or SRC to >> > > adapt to it. Do we care? >> > > >> > > PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >> > > VAR err := Cerrno.GetErrno(); >> > > ---------------^^^^^^^^^^^^^^^^^ >> > > BEGIN >> > > M3toC.FreeSharedS(p, f); >> > > OSErrorPosix.Raise0(err); >> > > END Fail; >> > > >> > > >> > > >> > > On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >> > >> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >> > >> runaway recursion trying to report a fault. This happens immediately upon >> > >> running it, before it produces any output. >> > >> >> > >> Here is one cycle of the backtrace: >> > >> >> > >> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >> > >> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >> > >> #1748 0x00007fd9b0b0ccfe in Raise (act= >> > >> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >> > >> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >> > >> at ../src/runtime/common/RTHooks.m3:79 >> > >> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >> > >> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >> > >> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >> > >> >> > >> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >> > >> >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 24 19:41:51 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 24 Aug 2015 17:41:51 +0000 Subject: [M3devel] critical mass 3 auf window10 In-Reply-To: <20150821144644.345ac1a25b429e6c35854d32@elego.de> References: , <20150821144644.345ac1a25b429e6c35854d32@elego.de> Message-ID: It is a bug and there is a workaround. See: https://connect.microsoft.com/VisualStudio/feedback/details/1651822/incorrect-mspdb140-dll-version-picked-in-x86-x64-cross-tools-environment I can probably workaround it in pylib.py also. - Jay Date: Fri, 21 Aug 2015 14:46:44 +0200 From: wagner at elego.de To: m3devel at elegosoft.com CC: sevettel at hotmail.de Subject: Re: [M3devel] critical mass 3 auf window10 On Fri, 21 Aug 2015 14:36:39 +0200 Stefan Vettel wrote: > Hallo Olaf, > sorry to poke you again just installed the binary part of CM3 on OS windows 10. > There is one thing still open. > A buildrun delivers following in the content of "test.lst": > Microsoft (R) Incremental Linker Version 14.00.23026.0Copyright (C) Microsoft Corporation. All rights reserved. > -out:test.exe -subsystem:console -entry:mainCRTStartup -nodefaultlib -debug -incremental:no -opt:ref -delayload:ws2_32.dll -delayload:advapi32.dll -delayload:gdi32.dll -delayload:netapi32.dll -delayload:user32.dll -delayload:comctl32.dll -delayload:rpcrt4.dll -delayload:iphlpapid.dll delayimp.lib _m3main.obj Test.mo c:\Users\..\cm3\pkg\libm3\NT386\m3.lib c:\Users\..\cm3\pkg\m3core\NT386\m3core.lib iphlpapi.lib rpcrt4.lib winspool.lib comctl32.lib ws2_32.lib comdlg32.lib netapi32.lib gdi32.lib user32.lib advapi32.lib kernel32.lib msvcrt.lib LINK : fatal error LNK1101: incorrect MSPDB140.DLL version; recheck installation of this product > > I'm searching for a leagal way to fix this problem.Is there any Idea ? This is for sure something that Jay should answer, who is our Microsoft specialist. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 04:08:09 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 24 Aug 2015 21:08:09 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop> <55D8A62A.8030205@lcwb.coop> <55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com> Message-ID: <55DBCE09.3020005@lcwb.coop> Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field jmpbuf of an exception frame being NIL. The only place I find that this field could be set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, in procedure CaptureState. From a version without this rework, it will require compiling the compiler twice, once to get a compiler that would generate this code, and again to get a compiler that executes it when run. You are saying the middle compiler legitimately raises and catches an exception, in order to run, so we can't get past this step. And it could do that in other places too. I am guessing you have re-bootstrapped your compiler through more than one intermediate state of source code, so you don't get this problem. But it has to work somehow from the original version, for everybody else. Will it work with the new compiler-generated code but old runtime? Seems like a long shot. On 08/22/2015 06:37 PM, Jay wrote: > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > I upgraded multiple installations on multiple targets. > I'll try again from 5.8.6. > > > I suggest upgrade also check for the bad dynamic linking. > > > Anyone else able/unable to upgrade? > > - Jay > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > >> >> >> On 08/22/2015 12:26 PM, Jay K wrote: >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. >>> >>> >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) >> >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run >> independently. >> >> After a good lot of successful-looking output, I get: >> >> new source -> compiling LLGen.i3 >> new source -> compiling LLGen.m3 >> new source -> compiling Version.i3 >> new source -> compiling M3Backend.i3 >> new source -> compiling Arg.i3 >> new source -> compiling Utils.i3 >> new source -> compiling Msg.i3 >> new source -> compiling M3Backend.m3 >> new source -> compiling UtilsPosix.m3 >> new source -> compiling Arg.m3 >> new source -> compiling M3Loc.i3 >> new source -> compiling M3Unit.i3 >> new source -> compiling Builder.i3 >> new source -> compiling M3Options.i3 >> new source -> compiling WebFile.i3 >> new source -> compiling Dirs.i3 >> new source -> compiling Builder.m3 >> new source -> compiling Dirs.m3 >> new source -> compiling M3Build.i3 >> new source -> compiling M3Build.m3 >> new source -> compiling M3Loc.m3 >> new source -> compiling M3Options.m3 >> new source -> compiling M3Unit.m3 >> new source -> compiling Makefile.i3 >> new source -> compiling Makefile.m3 >> new source -> compiling Msg.m3 >> new source -> compiling Utils.m3 >> new source -> compiling WebFile.m3 >> new source -> compiling Main.m3 >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure >> 1 warning encountered >> new source -> compiling cm3unix.c >> new exporters -> recompiling Utils.i3 >> -> linking cm3 >> --- shipping from AMD64_LINUX --- >> >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> .M3EXPORTS .M3WEB >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX >> cm3unix.c Version.i3 >> ../src => /usr/local/cm3/pkg/cm3/src >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 >> WebFile.m3 Main.m3 >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy >> LLGen.m3 LLGen.i3 >> . => /usr/local/cm3/bin >> cm3 >> . => /usr/local/cm3/man/man1 >> cm3.1 >> . => /usr/local/cm3/man/man7 >> m3makefile.7 >> Segmentation fault (core dumped) >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done >> >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done >> >> >> ------------------------------------------------------------------------------------- >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: >> ------------------------------------------------------------------------------------- >> >> >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == >> >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ >> *** execution of [, ] failed *** >> >> >> >> >> >> >> >> >> >>> The interface here between cm3's generated code/data and m3core has changed. >>> You cannot mix them. It will fail. >>> >>> >>> - Jay >>> >>> >>> >>> >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu >>>> Subject: Re: [M3devel] cm3 is broken: More info >>>> >>>> >>>> >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 >>>>> >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = >>>>> VAR p := LOOPHOLE (f, PF1); >>>>> BEGIN >>>>> IF DEBUG THEN >>>>> PutExcept ("INVOKE HANDLER", a); >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); >>>>> RTIO.PutText ("\n"); >>>>> RTIO.Flush (); >>>>> END; >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) >>>>> p.info := a; (* copy the exception to the new frame *) >>>>> <* ASSERT p.jmpbuf # NIL *> >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) >>>>> RAISE OUCH; >>>>> END InvokeHandler; >>>>> >>>>> (m3gdb) frame 0 >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>> 175 <* ASSERT p.jmpbuf # NIL *> >>>>> (m3gdb) p p.jmpbuf >>>>> $31 = NIL >>>>> --------^^^ >>>>> (m3gdb) >>>>> >>>>> >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: >>>>> >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); >>>>> BEGIN >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ >>>>> M3toC.FreeSharedS(pn, fname); >>>>> RETURN status >>>>> END Status; >>>>> >>>>> (m3gdb) >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>>> at ../src/os/POSIX/FSPosix.m3:328 >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>>> (m3gdb) p pn >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" >>>>> --------------------------------^^^^^^^^^^^ >>>>> (m3gdb) >>>>> >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? >>>> >>>> A bit more info: >>>> >>>> (m3gdb) down >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) >>>> at ../src/os/POSIX/FSPosix.m3:328 >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; >>>> (m3gdb) p Process.GetWorkingDirectory() >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> (m3gdb) >>>> >>>> This is where I ran cm3 from. >>>> >>>> >>>>> >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? >>>>> As I recall from many years ago, there was a change in C library that provided a thread >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to >>>>> adapt to it. Do we care? >>>>> >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = >>>>> VAR err := Cerrno.GetErrno(); >>>>> ---------------^^^^^^^^^^^^^^^^^ >>>>> BEGIN >>>>> M3toC.FreeSharedS(p, f); >>>>> OSErrorPosix.Raise0(err); >>>>> END Fail; >>>>> >>>>> >>>>> >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into >>>>>> runaway recursion trying to report a fault. This happens immediately upon >>>>>> running it, before it produces any output. >>>>>> >>>>>> Here is one cycle of the backtrace: >>>>>> >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) >>>>>> at ../src/runtime/common/RTHooks.m3:79 >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 >>>>>> >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. >>>>>> >>>>> >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Aug 25 06:12:15 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: <55DBCE09.3020005@lcwb.coop> References: <55D50129.6030600@lcwb.coop> <55D8A448.30703@lcwb.coop>,<55D8A62A.8030205@lcwb.coop> ,<55D8EF92.20403@lcwb.coop> <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop> Message-ID: Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 25 11:56:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 25 Aug 2015 09:56:36 +0000 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: I should have said you are close, it makes mostly sense, but not entirely. Anyway.. upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: % cm3 -versionCritical Mass Modula-3 version 5.8.6 last updated: 2010-04-11 compiled: 2010-07-12 20:10:34 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX to % cm3 -versionCritical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 09:33:26 configuration: /home/jkrell/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX in "one" step (automating many steps) Here is from scripts/python/capture-boot.py: https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz Do you have any local edits? Removed stuff or reordered in pkginfo.txt? (I edited one line in 5.8.6's config, to stop looking for m3cc). - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Date: Tue, 25 Aug 2015 04:12:15 +0000 Subject: Re: [M3devel] cm3 is broken: More info Rodney, no this doesn't make sense. The new runtime is never expected to work with an old compiler. No mix and match is required to work. But very few changes get made here, so the mix & match often does work, almost always. upgrade.py does build the compiler twice for this reason. You are right. And you are correct as to where it is set. The first uses the old runtime itself, the second uses the new runtime itself. Removing the assert will not fix it. Sorry, I'm late in: 1) re-bootstrapping from 5.8.6 -- though I did this multiple times already While I iterated a bunch on Darwin, I was able to upgrade Linux and Solaris (multiple targets) in one easy step each. I will go back to 5.8.6 and try again to be extra extra certain. 2) getting you a new min to start from, which will work and not really prove anything. Can you go back to 5.8.6, edit its config to avoid using the old cm3cg, run upgrade.py, and send the full log? I will also go back to 5.8.6 maybe tonight and upgrade.py. I understand that, at least, is required to work. - Jay > Date: Mon, 24 Aug 2015 21:08:09 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 is broken: More info > > Jay, I have looked into this, and I think there is a bootstrap barrier. The ASSERT at > m3-libs/m3core/src/runtime/ex_frame/RTExFrame.m3:175, procedure InvokeHandler, fails with field > jmpbuf of an exception frame being NIL. The only place I find that this field could be > set is by compiler-generated code, generated at m3-sys/m3front/src/misc/Marker.m3:234, > in procedure CaptureState. > > From a version without this rework, it will require compiling the compiler twice, once to > get a compiler that would generate this code, and again to get a compiler that executes > it when run. You are saying the middle compiler legitimately raises and catches an exception, > in order to run, so we can't get past this step. And it could do that in other places too. > > I am guessing you have re-bootstrapped your compiler through more than one intermediate state > of source code, so you don't get this problem. But it has to work somehow from the original version, > for everybody else. > > Will it work with the new compiler-generated code but old runtime? Seems like a long shot. > > On 08/22/2015 06:37 PM, Jay wrote: > > Hm. Something is not right. Can I build you an AMD64_LINUX min dist and you try upgrading from that? That is NOT my proposed general path forward, but a troubleshooting move. > > > > > > I upgraded multiple installations on multiple targets. > > I'll try again from 5.8.6. > > > > > > I suggest upgrade also check for the bad dynamic linking. > > > > > > Anyone else able/unable to upgrade? > > > > - Jay > > > > On Aug 22, 2015, at 2:54 PM, "Rodney M. Bates" wrote: > > > >> > >> > >> On 08/22/2015 12:26 PM, Jay K wrote: > >>> It is normal to get an exception here -- I think cm3 looks in current working directory..not great but that's how it is. > >>> That is how I know my stuff is working -- exceptions are a normal part of operation. If you break them, you know it quickly. > >>> > >>> > >>> Anyway, you need to go back to a working cm3 + cm3cg + config + libm3 + m3core and run upgrade.py. > >>> Nothing else is going to work. (ok, maybe upgrade.sh works, but I didn't try it.) > >> > >> OK, I did that. Got a working cm3 going, backed up my source/build directories and install > >> directory, ran scripts/python/upgrade.py. Right after it (as well as I can read it) installed > >> cm3, there is a segfault, and things after that look bad. A brand new cm3 is in my > >> /usr/local/cm3/bin, and it has the same symptoms as with my previous build method, when run > >> independently. > >> > >> After a good lot of successful-looking output, I get: > >> > >> new source -> compiling LLGen.i3 > >> new source -> compiling LLGen.m3 > >> new source -> compiling Version.i3 > >> new source -> compiling M3Backend.i3 > >> new source -> compiling Arg.i3 > >> new source -> compiling Utils.i3 > >> new source -> compiling Msg.i3 > >> new source -> compiling M3Backend.m3 > >> new source -> compiling UtilsPosix.m3 > >> new source -> compiling Arg.m3 > >> new source -> compiling M3Loc.i3 > >> new source -> compiling M3Unit.i3 > >> new source -> compiling Builder.i3 > >> new source -> compiling M3Options.i3 > >> new source -> compiling WebFile.i3 > >> new source -> compiling Dirs.i3 > >> new source -> compiling Builder.m3 > >> new source -> compiling Dirs.m3 > >> new source -> compiling M3Build.i3 > >> new source -> compiling M3Build.m3 > >> new source -> compiling M3Loc.m3 > >> new source -> compiling M3Options.m3 > >> new source -> compiling M3Unit.m3 > >> new source -> compiling Makefile.i3 > >> new source -> compiling Makefile.m3 > >> new source -> compiling Msg.m3 > >> new source -> compiling Utils.m3 > >> new source -> compiling WebFile.m3 > >> new source -> compiling Main.m3 > >> "../src/Main.m3", line 231: warning: potentially unhandled exception: Wr.Failure > >> 1 warning encountered > >> new source -> compiling cm3unix.c > >> new exporters -> recompiling Utils.i3 > >> -> linking cm3 > >> --- shipping from AMD64_LINUX --- > >> > >> . => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> .M3EXPORTS .M3WEB > >> ../AMD64_LINUX => /usr/local/cm3/pkg/cm3/AMD64_LINUX > >> cm3unix.c Version.i3 > >> ../src => /usr/local/cm3/pkg/cm3/src > >> M3Backend.i3 M3Backend.m3 UtilsPosix.m3 Arg.i3 > >> Arg.m3 Builder.m3 Builder.i3 Dirs.i3 > >> Dirs.m3 M3Build.m3 M3Build.i3 M3Loc.m3 > >> M3Loc.i3 M3Options.m3 M3Options.i3 M3Unit.i3 > >> M3Unit.m3 Makefile.i3 Makefile.m3 Msg.i3 > >> Msg.m3 Utils.m3 Utils.i3 WebFile.i3 > >> WebFile.m3 Main.m3 > >> ../src/llvmdummy => /usr/local/cm3/pkg/cm3/src/llvmdummy > >> LLGen.m3 LLGen.i3 > >> . => /usr/local/cm3/bin > >> cm3 > >> . => /usr/local/cm3/man/man1 > >> cm3.1 > >> . => /usr/local/cm3/man/man7 > >> m3makefile.7 > >> Segmentation fault (core dumped) > >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >> -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3objfile done > >> > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3linker == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3linker done > >> > >> > >> ------------------------------------------------------------------------------------- > >> several more almost same sequences as m3linker omitted, for m3back, m3cggen, m3quake, > >> m3front, m3cgcat, m3bundle, mklib, cm3 again. Then, for m3cc: > >> ------------------------------------------------------------------------------------- > >> > >> > >> /home/rodney/proj/m3/git/cm3/m3-sys/m3cggen/AMD64_LINUX/m3cggen > /home/rodney/proj/m3/git/cm3/m3-sys/m3cc/gcc/gcc/m3cg/m3cg.h > >> == package /home/rodney/proj/m3/git/cm3/m3-sys/m3cc == > >> > >> +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ > >> *** execution of [, ] failed *** > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>> The interface here between cm3's generated code/data and m3core has changed. > >>> You cannot mix them. It will fail. > >>> > >>> > >>> - Jay > >>> > >>> > >>> > >>> > >>>> Date: Sat, 22 Aug 2015 11:41:14 -0500 > >>>> From: rodney_bates at lcwb.coop > >>>> To: m3devel at elegosoft.com; jay.krell at cornell.edu > >>>> Subject: Re: [M3devel] cm3 is broken: More info > >>>> > >>>> > >>>> > >>>> On 08/22/2015 11:33 AM, Rodney M. Bates wrote: > >>>>> There are at least two problems. The runaway recursion happens in RTExFrame.m3:175 > >>>>> > >>>>> PROCEDURE InvokeHandler (f: Frame; READONLY a: RT0.RaiseActivation) RAISES ANY = > >>>>> VAR p := LOOPHOLE (f, PF1); > >>>>> BEGIN > >>>>> IF DEBUG THEN > >>>>> PutExcept ("INVOKE HANDLER", a); > >>>>> RTIO.PutText (" frame="); RTIO.PutAddr (f); > >>>>> RTIO.PutText (" class="); RTIO.PutInt (f.class); > >>>>> RTIO.PutText ("\n"); > >>>>> RTIO.Flush (); > >>>>> END; > >>>>> RTThread.SetCurrentHandlers (f.next); (* cut to the new handler *) > >>>>> p.info := a; (* copy the exception to the new frame *) > >>>>> <* ASSERT p.jmpbuf # NIL *> > >>>>> ------^^^^^^^^^^^^^^^^^^^^^^^^^^^ This assertion fails and tries to raise another fault. > >>>>> Csetjmp.ulongjmp (p.jmpbuf, 1); (* and jump... *) > >>>>> RAISE OUCH; > >>>>> END InvokeHandler; > >>>>> > >>>>> (m3gdb) frame 0 > >>>>> #0 InvokeHandler (f=16_00007fff509f4af0, a= > >>>>> RECORD exception = 16_00007ff696883aa0; arg = 16_00000000013e4ae8; module = 16_00007ff6968a3060; line = 50; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>> 175 <* ASSERT p.jmpbuf # NIL *> > >>>>> (m3gdb) p p.jmpbuf > >>>>> $31 = NIL > >>>>> --------^^^ > >>>>> (m3gdb) > >>>>> > >>>>> > >>>>> The original problem starts in FSPosix.m3:328, when cm3.cfg is not found: > >>>>> > >>>>> PROCEDURE Status(pn: Pathname.T): File.Status RAISES {OSError.E} = > >>>>> VAR status: File.Status; fname := M3toC.SharedTtoS(pn); > >>>>> BEGIN > >>>>> IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> ---------^^^^^^^^^^^^^^^^^^^^^^ > >>>>> M3toC.FreeSharedS(pn, fname); > >>>>> RETURN status > >>>>> END Status; > >>>>> > >>>>> (m3gdb) > >>>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>>> (m3gdb) p pn > >>>>> $30 = (*16_00000000013e4ac0*) "./cm3.cfg" > >>>>> --------------------------------^^^^^^^^^^^ > >>>>> (m3gdb) > >>>>> > >>>>> I have a cm3.cfg beside cm3, where it should be. Could the current directory be wrong? > >>>> > >>>> A bit more info: > >>>> > >>>> (m3gdb) down > >>>> #6 0x00007ff696593c14 in Status (pn=16_00000000013e4ac0, _result=RECORD type = 16_0000000000000001; modificationTime = ; size = 1; END) > >>>> at ../src/os/POSIX/FSPosix.m3:328 > >>>> 328 IF CStatus(fname, status) < 0 THEN Fail(pn, fname); END; > >>>> (m3gdb) p Process.GetWorkingDirectory() > >>>> $32 = (*16_0000000001420170*) "/home/rodney/proj/m3/exp/trytemp/src" > >>>> --------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >>>> (m3gdb) > >>>> > >>>> This is where I ran cm3 from. > >>>> > >>>> > >>>>> > >>>>> Also, I notice one other thing, that probably isn't part of this problem, but maybe needs > >>>>> to be fixed. Fail, when called from Status, uses Cerrno.GetErrno in a separate call, > >>>>> after the failing CStatus above. Obviously not thread safe. Does it matter here? > >>>>> As I recall from many years ago, there was a change in C library that provided a thread > >>>>> safe alternative to this, and I even think I had to change something in pm3 or SRC to > >>>>> adapt to it. Do we care? > >>>>> > >>>>> PROCEDURE Fail(p: Pathname.T; f: Ctypes.char_star) RAISES {OSError.E} = > >>>>> VAR err := Cerrno.GetErrno(); > >>>>> ---------------^^^^^^^^^^^^^^^^^ > >>>>> BEGIN > >>>>> M3toC.FreeSharedS(p, f); > >>>>> OSErrorPosix.Raise0(err); > >>>>> END Fail; > >>>>> > >>>>> > >>>>> > >>>>> On 08/19/2015 05:20 PM, Rodney M. Bates wrote: > >>>>>> As of a recent pull from modula3-cm3, done around 16:00 CDT, cm3 is going into > >>>>>> runaway recursion trying to report a fault. This happens immediately upon > >>>>>> running it, before it produces any output. > >>>>>> > >>>>>> Here is one cycle of the backtrace: > >>>>>> > >>>>>> #1745 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> #1746 0x00007fd9b0b0d05d in InvokeHandler (f=16_00007fff488bcc20, a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:175 > >>>>>> #1747 0x00007fd9b0b0cef1 in ResumeRaise (a= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:145 > >>>>>> #1748 0x00007fd9b0b0ccfe in Raise (act= > >>>>>> RECORD exception = 16_00007fd9b0d4b780; arg = 16_000000000000001d; module = 16_00007fd9b0d62d40; line = 14; pc = NIL; info0 = NIL; info1 = NIL; un_except = NIL; un_arg = NIL; END) at ../src/runtime/ex_frame/RTExFrame.m3:91 > >>>>>> #1749 0x00007fd9b0ae9693 in Raise (ex=16_00007fd9b0d4b780, arg=16_000000000000001d, module=16_00007fd9b0d62d40, line=14) > >>>>>> at ../src/runtime/common/RTHooks.m3:79 > >>>>>> #1750 0x00007fd9b0ae9951 in Self () at ../src/runtime/common/RuntimeError.m3:14 > >>>>>> #1751 0x00007fd9b0ae96ef in ReportFault (module=16_00007fd9b0d6a520, info=5600) at ../src/runtime/common/RTHooks.m3:98 > >>>>>> #1752 0x00007fd9b0b0d935 in _m3_fault (arg=5600) from /usr/local/cm3-uniboot/bin/../lib/libm3core.so.5 > >>>>>> > >>>>>> So far, I haven't had the patience to keep hitting return until I get to the bottom of it. > >>>>>> > >>>>> > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>>> _______________________________________________ > >>>> M3devel mailing list > >>>> M3devel at elegosoft.com > >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>> > >>> _______________________________________________ > >>> M3devel mailing list > >>> M3devel at elegosoft.com > >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Aug 25 22:08:41 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 25 Aug 2015 15:08:41 -0500 Subject: [M3devel] cm3 is broken: More info In-Reply-To: References: <55D50129.6030600@lcwb.coop>, <55D8A448.30703@lcwb.coop>, <55D8A62A.8030205@lcwb.coop>, , <55D8EF92.20403@lcwb.coop>, <53E6DCB7-4FF6-4F50-93D5-6B4731EAB93D@gmail.com>, <55DBCE09.3020005@lcwb.coop>, Message-ID: <55DCCB49.80103@lcwb.coop> OK, I got a working compiler from 5.8.6, and it is now statically linked to all modula-3 libs. rodney at allegheny:~/proj/m3/exp/trytemp/src$ cm3 -version Critical Mass Modula-3 version d5.10.0 last updated: 2015-05-21 compiled: 2015-08-25 19:02:01 configuration: /usr/local/cm3/bin/cm3.cfg host: AMD64_LINUX target: AMD64_LINUX rodney at allegheny:~/proj/m3/exp/trytemp/src$ ldd /usr/local/cm3/bin/cm3 linux-vdso.so.1 => (0x00007fff54bfe000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8cca5b8000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8cca39a000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8cc9fd2000) /lib64/ld-linux-x86-64.so.2 (0x00007f8cca8de000) I did a fresh pull of modula3-cm3, and removed and re-checked out a couple of locally edited git-tracked files, so my working is as github. git diff HEAD shows only a prompt. Then I make a full backup of my 5.8.6 installation directory, switched to it (the original), went to scripts/python, and did ./upgrade-full.sh 2>&1 | tee upgrade-full.sh.log. I did not do anything to disable rebuilding of m3cc. I had some changes in the last pull, and lots since 5.8.6. Eventually the rebuild of everything failed in cvsup, which is probably entirely irrelevant. There are many clean steps that look like: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ rm -rf AMD64_LINUX +++ ==> [] returned 0 Are they cleaning or failing to clean? In selected cases, they are followed by build and ship steps that do nothing: == package /home/rodney/proj/m3/git/cm3/m3-libs/m3core == +++ /usr/local/cm3/bin/cm3 -build -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done +++ /usr/local/cm3/bin/cm3 -ship -DROOT=/home/rodney/proj/m3/git/cm3 +++ ==> /home/rodney/proj/m3/git/cm3/m3-libs/m3core done So I must have had some non-bootstrappable intermediate version of something? > On 08/25/2015 04:56 AM, Jay K wrote: > I should have said you are close, it makes mostly sense, but not entirely. Anyway.. > > > upgrade-full.sh, which is upgrade.py followed by rebuilding everything (in some cases, a third, unnecessary time), just brought me again from: > > > > % cm3 -version > Critical Mass Modula-3 version 5.8.6 > last updated: 2010-04-11 > compiled: 2010-07-12 20:10:34 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > to > > > % cm3 -version > Critical Mass Modula-3 version d5.10.0 > last updated: 2015-05-21 > compiled: 2015-08-25 09:33:26 > configuration: /home/jkrell/cm3/bin/cm3.cfg > host: AMD64_LINUX > target: AMD64_LINUX > > > in "one" step (automating many steps) > > Here is from scripts/python/capture-boot.py: > https://modula3.elegosoft.com/cm3/uploaded-archives/cm3-boot-AMD64_LINUX-1.tar.gz > > > Do you have any local edits? > > > Removed stuff or reordered in pkginfo.txt? > > > (I edited one line in 5.8.6's config, to stop looking for m3cc). > > - Jay > > > >-- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 02:28:48 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 00:28:48 +0000 Subject: [M3devel] Target.First_readable_addr Message-ID: Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Aug 26 07:54:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 26 Aug 2015 05:54:25 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> References: , <0077363C-11E9-47BC-9946-ED6C2F0A2F49@purdue.edu> Message-ID: At the very least, can we use the same number for all targets, 4K? I don't like this inconsistency: IF RTParams.IsPresent("checked") THEN EVAL F4096x8p2.F1(NIL); (* large -- gets null checked *) END; IF RTParams.IsPresent("unchecked") THEN EVAL F0.F1(NIL); (* small -- does not get null check *) END; jair:p263 jay$ ./I386_DARWIN/pgm @M3checked ****** runtime error:*** Attempt to reference an illegal memory location.*** file "../F4096x8p2.m3", line 8*** Abort trap: 6jair:p263 jay$ ./I386_DARWIN/pgm @M3unchecked ****** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0xb8dc5 = F1 + 0x6 in ../F0.m3*** I wonder if we have even implemented this for Windows, the unchecked form.The inefficient checked form is easy.Notice how "large" gives the file/line of the null deref, "small" does not. What do Java, gcj, C#, mono, .NET Native, go, rust do? (Extra credit to anyone who can show meboth their compiler outputting the checksand their generated code in a debugger..For Modula-3/cm3cg, grep for fault in the *ms files for *.c files) - Jay Subject: Re: [M3devel] Target.First_readable_addr From: hosking at purdue.edu Date: Wed, 26 Aug 2015 14:44:30 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu This optimization has some performance benefit.Surely we can simply fix the problems rather than removing the capability. On Aug 26, 2015, at 10:28 AM, Jay K wrote:Of course, my agenda is to remove target-specificity. Target.First_readable_addr is a target-specific optimizationto omit null checks. Or maybe range checks.. The assumption is that dereferencing "actual" null is checkedby the hardware, so code doesn't have to be generated to check for null. I'm not even sure I like this -- as the behavior of dereferencing null is not portable. The assumption is further that accesses to the first page, at least,are subject to the same hardware checks, and can also be optimized away. At first, glance, I thought this was based on offset + size of a static access. For example: a: ARRAY [0...large] OF CHAR a[0..4095] no checka[4096 and up] would check Target.First_deadable_addr is 4k on many targets; 8k on others.Setting it too low would yield extra checks.Setting it too high would yield missing checks and a loss of safety. Here is what I actually found though. - The check is based on the size of the type being accessed. - It is off by a factor of 8 -- there is confusing between m3middle and m3front as to if it is bit number of a byte number. small: ARRAY[0..100] OF CHARlarge:ARRAY[0..100000] OF CHAR no access to small gets checked and every access to larger gets checked. Should we do anything about this? In m3-sys/m3tests/src/p2/p263:cm3 -realcleancm3 -keepgrep fault /*ms All the accesses are to offset 0.So, by some expectation, no null checks are needed.Null checks are output when the size of thecontaining type is, for x86, larger than 4096*8. The checks have been largely but not completely wrong/missing.Safety behooves us to check though. - fix the factor of 8? - make it 0?? too slow? - make it 4k on all target? until such time as a target manifests with a smaller page size? - base the checks on access offset + size, not containing size? Containing size is conservative. It checks more than i think is meant. I couldn't actually figure out the code here, I added various: IF RTParams.IsPresent("qual") THEN RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); RTIO.Flush(); END; and such to m3front to figure it out. Thanks,- Jay _______________________________________________M3devel mailing listM3devel at elegosoft.comhttps://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 26 17:37:54 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 10:37:54 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: References: Message-ID: <55DDDD52.5090008@lcwb.coop> I have always thought "segfault, could it be a NIL pointer deref?", or whatever words to that effect, was a rather lame excuse of an error message, and not much help. Since this scheme doesn't fault at the point of the real deref, but afterwards, when a memory reference is actually made, possibly computed from that address, could the location given for the fault (whether line no or just code displacement) be wrong? Very wrong? TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; PROCEDURE Q ( ) = VAR Ptr : REF A := NIL; BEGIN P(Ptr^); <---deref occurs here END Q; .... lots of other code, different module, etc. ... PROCEDURE P ( VAR x : A ) = BEGIN x[119] := 17 <---memory ref & segfault don't happen until here. END P; Or is this scheme only used in selective places, such as implicit deref when putting a subscript onto a pointer? e.g.: Ptr[117] := 43; On 08/25/2015 07:28 PM, Jay K wrote: > Of course, my agenda is to remove target-specificity. > > > > Target.First_readable_addr is a target-specific optimization > to omit null checks. Or maybe range checks.. > > > The assumption is that dereferencing "actual" null is checked > by the hardware, so code doesn't have to be generated to check for null. > > > I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > > > The assumption is further that accesses to the first page, at least, > are subject to the same hardware checks, and can also be optimized away. > > > At first, glance, I thought this was based on offset + size of a static access. > > > For example: > > a: ARRAY [0...large] OF CHAR > > > a[0..4095] no check > a[4096 and up] would check > > > Target.First_deadable_addr is 4k on many targets; 8k on others. > Setting it too low would yield extra checks. > Setting it too high would yield missing checks and a loss of safety. > > > Here is what I actually found though. > > > - The check is based on the size of the type being accessed. > > - It is off by a factor of 8 -- there is confusing between m3middle and m3front > as to if it is bit number of a byte number. > > > small: ARRAY[0..100] OF CHAR > large:ARRAY[0..100000] OF CHAR > > no access to small gets checked and every access to larger gets checked. > > Should we do anything about this? > > In m3-sys/m3tests/src/p2/p263: > cm3 -realclean > cm3 -keep > grep fault /*ms > > > All the accesses are to offset 0. > So, by some expectation, no null checks are needed. > Null checks are output when the size of the > containing type is, for x86, larger than 4096*8. > > > The checks have been largely but not completely wrong/missing. > Safety behooves us to check though. > > - fix the factor of 8? > - make it 0?? too slow? > - make it 4k on all target? until such time as a target manifests with a smaller page size? > - base the checks on access offset + size, not containing size? > Containing size is conservative. It checks more than i think is meant. > > > I couldn't actually figure out the code here, I added various: > > IF RTParams.IsPresent("qual") THEN > RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > RTIO.Flush(); > END; > > and such to m3front to figure it out. > > > Thanks, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 26 18:10:36 2015 From: jay.krell at cornell.edu (Jay) Date: Wed, 26 Aug 2015 09:10:36 -0700 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DDDD52.5090008@lcwb.coop> References: <55DDDD52.5090008@lcwb.coop> Message-ID: <08316510-3124-4372-A514-3D4881D26966@gmail.com> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. - Jay On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > I have always thought "segfault, could it be a NIL pointer deref?", or whatever > words to that effect, was a rather lame excuse of an error message, and not > much help. > > Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > memory reference is actually made, possibly computed from that address, could the location > given for the fault (whether line no or just code displacement) be wrong? Very wrong? > > TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > > PROCEDURE Q ( ) > = VAR Ptr : REF A := NIL; > BEGIN > P(Ptr^); <---deref occurs here > END Q; > > .... lots of other code, different module, etc. ... > > PROCEDURE P ( VAR x : A ) > = BEGIN > x[119] := 17 <---memory ref & segfault don't happen until here. > END P; > > Or is this scheme only used in selective places, such as implicit deref > when putting a subscript onto a pointer? e.g.: > > Ptr[117] := 43; > > On 08/25/2015 07:28 PM, Jay K wrote: >> Of course, my agenda is to remove target-specificity. >> >> >> >> Target.First_readable_addr is a target-specific optimization >> to omit null checks. Or maybe range checks.. >> >> >> The assumption is that dereferencing "actual" null is checked >> by the hardware, so code doesn't have to be generated to check for null. >> >> >> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >> >> >> The assumption is further that accesses to the first page, at least, >> are subject to the same hardware checks, and can also be optimized away. >> >> >> At first, glance, I thought this was based on offset + size of a static access. >> >> >> For example: >> >> a: ARRAY [0...large] OF CHAR >> >> >> a[0..4095] no check >> a[4096 and up] would check >> >> >> Target.First_deadable_addr is 4k on many targets; 8k on others. >> Setting it too low would yield extra checks. >> Setting it too high would yield missing checks and a loss of safety. >> >> >> Here is what I actually found though. >> >> >> - The check is based on the size of the type being accessed. >> >> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >> as to if it is bit number of a byte number. >> >> >> small: ARRAY[0..100] OF CHAR >> large:ARRAY[0..100000] OF CHAR >> >> no access to small gets checked and every access to larger gets checked. >> >> Should we do anything about this? >> >> In m3-sys/m3tests/src/p2/p263: >> cm3 -realclean >> cm3 -keep >> grep fault /*ms >> >> >> All the accesses are to offset 0. >> So, by some expectation, no null checks are needed. >> Null checks are output when the size of the >> containing type is, for x86, larger than 4096*8. >> >> >> The checks have been largely but not completely wrong/missing. >> Safety behooves us to check though. >> >> - fix the factor of 8? >> - make it 0?? too slow? >> - make it 4k on all target? until such time as a target manifests with a smaller page size? >> - base the checks on access offset + size, not containing size? >> Containing size is conservative. It checks more than i think is meant. >> >> >> I couldn't actually figure out the code here, I added various: >> >> IF RTParams.IsPresent("qual") THEN >> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >> RTIO.Flush(); >> END; >> >> and such to m3front to figure it out. >> >> >> Thanks, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Aug 27 04:23:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 26 Aug 2015 21:23:05 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <08316510-3124-4372-A514-3D4881D26966@gmail.com> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> Message-ID: <55DE7489.9000609@lcwb.coop> On 08/26/2015 11:10 AM, Jay wrote: > Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > No, I disagree adamantly. The whole idea of linguistic support of parameters passed by reference, instead of the programmer manually concocting it by taking addresses and contents of pointers, is that the language ensures to the called procedure that the hidden pointer will be neither NIL, uninitialized, nor point to something of the wrong type. In fact, pass by ref is not even defined in terms of the compiler lowering into pointer twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the type system that should only happen in UNSAFE code. NIL deref, as a runtime error, should in concept happen when and where ^ is applied to a pointer, including the implied dereference when a subscript or field selection is applied directly without the ^ operator. It's purely an implementation question how to do this. If a segfault message will point to the line of code where the dereference happens, and before execution proceeds beyond that code, that would be marginally OK. (Only marginally, because the error message should be able to say with certainty that it's a NIL deref. Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his obligation, its effects can escape into safe code. > > - Jay > > On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >> words to that effect, was a rather lame excuse of an error message, and not >> much help. >> >> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >> memory reference is actually made, possibly computed from that address, could the location >> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >> >> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >> >> PROCEDURE Q ( ) >> = VAR Ptr : REF A := NIL; >> BEGIN >> P(Ptr^); <---deref occurs here >> END Q; >> >> .... lots of other code, different module, etc. ... >> >> PROCEDURE P ( VAR x : A ) >> = BEGIN >> x[119] := 17 <---memory ref & segfault don't happen until here. >> END P; >> >> Or is this scheme only used in selective places, such as implicit deref >> when putting a subscript onto a pointer? e.g.: >> >> Ptr[117] := 43; >> >> On 08/25/2015 07:28 PM, Jay K wrote: >>> Of course, my agenda is to remove target-specificity. >>> >>> >>> >>> Target.First_readable_addr is a target-specific optimization >>> to omit null checks. Or maybe range checks.. >>> >>> >>> The assumption is that dereferencing "actual" null is checked >>> by the hardware, so code doesn't have to be generated to check for null. >>> >>> >>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>> >>> >>> The assumption is further that accesses to the first page, at least, >>> are subject to the same hardware checks, and can also be optimized away. >>> >>> >>> At first, glance, I thought this was based on offset + size of a static access. >>> >>> >>> For example: >>> >>> a: ARRAY [0...large] OF CHAR >>> >>> >>> a[0..4095] no check >>> a[4096 and up] would check >>> >>> >>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>> Setting it too low would yield extra checks. >>> Setting it too high would yield missing checks and a loss of safety. >>> >>> >>> Here is what I actually found though. >>> >>> >>> - The check is based on the size of the type being accessed. >>> >>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>> as to if it is bit number of a byte number. >>> >>> >>> small: ARRAY[0..100] OF CHAR >>> large:ARRAY[0..100000] OF CHAR >>> >>> no access to small gets checked and every access to larger gets checked. >>> >>> Should we do anything about this? >>> >>> In m3-sys/m3tests/src/p2/p263: >>> cm3 -realclean >>> cm3 -keep >>> grep fault /*ms >>> >>> >>> All the accesses are to offset 0. >>> So, by some expectation, no null checks are needed. >>> Null checks are output when the size of the >>> containing type is, for x86, larger than 4096*8. >>> >>> >>> The checks have been largely but not completely wrong/missing. >>> Safety behooves us to check though. >>> >>> - fix the factor of 8? >>> - make it 0?? too slow? >>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>> - base the checks on access offset + size, not containing size? >>> Containing size is conservative. It checks more than i think is meant. >>> >>> >>> I couldn't actually figure out the code here, I added various: >>> >>> IF RTParams.IsPresent("qual") THEN >>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>> RTIO.Flush(); >>> END; >>> >>> and such to m3front to figure it out. >>> >>> >>> Thanks, >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Aug 28 00:20:58 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 27 Aug 2015 17:20:58 -0500 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> References: <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com> <55DE7489.9000609@lcwb.coop> <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu> Message-ID: <55DF8D4A.5050707@lcwb.coop> In any case, I would shed no tears if First_readable_addr were set target-indepenently to the lowest value on any target, since that would only increase the number of cases that give the proper error at the proper place & time. On 08/26/2015 10:07 PM, Antony Hosking wrote: > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: >> >> >> >> On 08/26/2015 11:10 AM, Jay wrote: >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. >>> >> >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by >> reference, instead of the programmer manually concocting it by taking addresses and >> contents of pointers, is that the language ensures to the called procedure that the >> hidden pointer will be neither NIL, uninitialized, nor point to something of the >> wrong type. >> >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the >> type system that should only happen in UNSAFE code. >> >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied >> to a pointer, including the implied dereference when a subscript or field selection is >> applied directly without the ^ operator. It's purely an implementation question how to >> do this. If a segfault message will point to the line of code where the dereference >> happens, and before execution proceeds beyond that code, that would be marginally OK. >> (Only marginally, because the error message should be able to say with certainty that >> it's a NIL deref. >> >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his >> obligation, its effects can escape into safe code. >> >>> >>> - Jay >>> >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: >>> >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever >>>> words to that effect, was a rather lame excuse of an error message, and not >>>> much help. >>>> >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a >>>> memory reference is actually made, possibly computed from that address, could the location >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? >>>> >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; >>>> >>>> PROCEDURE Q ( ) >>>> = VAR Ptr : REF A := NIL; >>>> BEGIN >>>> P(Ptr^); <---deref occurs here >>>> END Q; >>>> >>>> .... lots of other code, different module, etc. ... >>>> >>>> PROCEDURE P ( VAR x : A ) >>>> = BEGIN >>>> x[119] := 17 <---memory ref & segfault don't happen until here. >>>> END P; >>>> >>>> Or is this scheme only used in selective places, such as implicit deref >>>> when putting a subscript onto a pointer? e.g.: >>>> >>>> Ptr[117] := 43; >>>> >>>> On 08/25/2015 07:28 PM, Jay K wrote: >>>>> Of course, my agenda is to remove target-specificity. >>>>> >>>>> >>>>> >>>>> Target.First_readable_addr is a target-specific optimization >>>>> to omit null checks. Or maybe range checks.. >>>>> >>>>> >>>>> The assumption is that dereferencing "actual" null is checked >>>>> by the hardware, so code doesn't have to be generated to check for null. >>>>> >>>>> >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. >>>>> >>>>> >>>>> The assumption is further that accesses to the first page, at least, >>>>> are subject to the same hardware checks, and can also be optimized away. >>>>> >>>>> >>>>> At first, glance, I thought this was based on offset + size of a static access. >>>>> >>>>> >>>>> For example: >>>>> >>>>> a: ARRAY [0...large] OF CHAR >>>>> >>>>> >>>>> a[0..4095] no check >>>>> a[4096 and up] would check >>>>> >>>>> >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. >>>>> Setting it too low would yield extra checks. >>>>> Setting it too high would yield missing checks and a loss of safety. >>>>> >>>>> >>>>> Here is what I actually found though. >>>>> >>>>> >>>>> - The check is based on the size of the type being accessed. >>>>> >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front >>>>> as to if it is bit number of a byte number. >>>>> >>>>> >>>>> small: ARRAY[0..100] OF CHAR >>>>> large:ARRAY[0..100000] OF CHAR >>>>> >>>>> no access to small gets checked and every access to larger gets checked. >>>>> >>>>> Should we do anything about this? >>>>> >>>>> In m3-sys/m3tests/src/p2/p263: >>>>> cm3 -realclean >>>>> cm3 -keep >>>>> grep fault /*ms >>>>> >>>>> >>>>> All the accesses are to offset 0. >>>>> So, by some expectation, no null checks are needed. >>>>> Null checks are output when the size of the >>>>> containing type is, for x86, larger than 4096*8. >>>>> >>>>> >>>>> The checks have been largely but not completely wrong/missing. >>>>> Safety behooves us to check though. >>>>> >>>>> - fix the factor of 8? >>>>> - make it 0?? too slow? >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? >>>>> - base the checks on access offset + size, not containing size? >>>>> Containing size is conservative. It checks more than i think is meant. >>>>> >>>>> >>>>> I couldn't actually figure out the code here, I added various: >>>>> >>>>> IF RTParams.IsPresent("qual") THEN >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); >>>>> RTIO.Flush(); >>>>> END; >>>>> >>>>> and such to m3front to figure it out. >>>>> >>>>> >>>>> Thanks, >>>>> - Jay >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Aug 28 02:32:22 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:32:22 +0000 Subject: [M3devel] github history In-Reply-To: References: <73FE1A29-7F80-46DB-8B3C-1CA168C6FC92@purdue.edu>, , Message-ID: I don't really know what I'm doing with git. And it seems dangerous. I do know what I'm doing with Perforce, but oh well. I am a big fan of preserving history, so if I lost any, you can assume it was an accident. (That said, "local history" on my machine maybe isn't so valuable, noisy.) I really don't understand git merging. I tried using stash when there was a conflict, and that didn't go well. I don't know how to have multiple "checkouts" w/o multiple repositories. Wasteful. The web suggests some ways, but they didn't work for me -- e.g. cloning a local; I couldn't push from that. I gave up and created another large local repository. At least it was easy and fast enough. I don't want little temporary branches for this -- I don't want my one working directory to bounce between branches, I want multiple independent working directories against the same branch. - Jay Subject: Re: github history From: hosking at purdue.edu Date: Thu, 27 Aug 2015 14:54:57 +1000 CC: m3devel at elegosoft.com; jay.krell at cornell.edu; dragisha at m3w.org To: hosking at purdue.edu Update: I have successfully brought back history by: 1. setting the default branch to our old trunk,2. merging from master,3. deleting master,4. branching a new master from the (up-to-date) trunk, and5. setting the default branch back to master. On Aug 27, 2015, at 1:46 PM, Antony Hosking wrote:PS Jay, in poking around on github it seems that the source of this weirdness is a merge you did earlier this year. I suspect you were not using proper github workflow and it messed up the history (perhaps you inverted the merge?). Does anyone out there have better github fu than I do, who can dig into this and try to repair? Dragisha? On Aug 27, 2015, at 1:14 PM, Antony Hosking wrote:I am dismayed to find that the full commit histories on github do not appear to have made it over from the transition from svn to github. For example, I find the following very abbreviated history in the master tree for NilChkExpr.m3 in package m3front (see below). Did something strange happen in a merge somewhere along the way from the initial import into github? It looks like it occurred at the time of the commit "jaykrell on Jan 4, 2008 initial diff from ../config/PPC_LINUX" when it looks like the master got clobbered somehow. Anyone have any ideas if we can recover? Commits on May 4, 2008<1635728.png>add more information printed for some assertion failures .jaykrell authored on May 4, 200866139d4 Commits on Jan 4, 2008<1635728.png>initial diff from ../config/PPC_LINUXjaykrell authored on Jan 4, 20082412437 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 28 02:40:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 28 Aug 2015 00:40:32 +0000 Subject: [M3devel] Target.First_readable_addr In-Reply-To: <55DF8D4A.5050707@lcwb.coop> References: , <55DDDD52.5090008@lcwb.coop> <08316510-3124-4372-A514-3D4881D26966@gmail.com>, <55DE7489.9000609@lcwb.coop>, <68BAA952-5760-46EE-98FA-EE597B9A0752@purdue.edu>, <55DF8D4A.5050707@lcwb.coop> Message-ID: That is done -- it is a constant 4k now. Hypothetically Sparc/IA64/Alpha could be 8k. Sparc used to be. I'm still leary of the omitted checks and suggest we compare against C# (JIT and native, Microsoft and Mono), Java, Go, Rust, Gnat, etc. If most/all of them check aggressively, we should to imho. If they actually omit more aggressively, which I believe is possible, we should consider that. In particular, I think the omission should be based on the offset+size of the access, not of the containing type. i.e. the field and not the record. However this might be a fallacy also. e.g. TYPE BigRecord = RECORD ...lots.... INTEGER smallFieldPast4K END; PROCEDURE F1(VAR a:INTEGER); PROCEDURE F2(b:REF BigRecord) BEGIN F1(b.a); END F2; Hm. I see. Maybe you have to check at b.a and a (not shown here)? The recipient of a "small" type doesn't know its offset, if any, within a larger type, and it can vary. I can have: TYPE SmallRecord = RECORD .. nothing .. INTEGER smallFieldAtStart END; F1(smallRecord.a); So maybe you are right anyway -- check needed at the "address generation" not just the deref. I need to study this more. Another idea, perhaps, is that if there is any dereference of a parameter within a function, do all the checks up front, and then none in the function. You know -- in case there are loops. But I also don't know the guarantees around "dynamic paths" -- turning conditional dereferences into unconditional dereferences. I need to study this more. :) - Jay > Date: Thu, 27 Aug 2015 17:20:58 -0500 > From: rodney_bates at lcwb.coop > To: hosking at purdue.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] Target.First_readable_addr > > In any case, I would shed no tears if First_readable_addr were set target-indepenently > to the lowest value on any target, since that would only increase the number of cases > that give the proper error at the proper place & time. > > On 08/26/2015 10:07 PM, Antony Hosking wrote: > > The compiler does in fact leave a nil check on every deref (both implicit and explicit) except in the case that the offset falls within the range that can be caught with a trap. In which case, yes the check is delayed until the actual memory access. It is perhaps not so bad that the offending deref will be visible up the stack in the case of VAR params. So, pragmatically I think it is nice to have zero-cost null checks, but it does compromise Rodney?s desire for every use of ^ to have the null check at that point. > > > >> On Aug 27, 2015, at 12:23 PM, Rodney M. Bates wrote: > >> > >> > >> > >> On 08/26/2015 11:10 AM, Jay wrote: > >>> Passing a ^ to a VAR imho should probably not count as a deref -- I agree with the implementation. Just like in C++ dereferencing a pointer to get a reference instead of a value. One wants a stack trace, and a runtime check maybe applied after the fact. > >>> Ie. once we fault, show all nil parameters on the stack to all functions to help me fish for it. Not easy. > >>> > >> > >> No, I disagree adamantly. The whole idea of linguistic support of parameters passed by > >> reference, instead of the programmer manually concocting it by taking addresses and > >> contents of pointers, is that the language ensures to the called procedure that the > >> hidden pointer will be neither NIL, uninitialized, nor point to something of the > >> wrong type. > >> > >> In fact, pass by ref is not even defined in terms of the compiler lowering into pointer > >> twiddling. It is defined abstractly as the formal is "bound to", i.e. "aliases" the actual > >> (2.3.2). This is a highly proper subset of the full semantics of what pointers can do. > >> NIL or uninitialized have no meaning in this definition, and wrong type is a failure of the > >> type system that should only happen in UNSAFE code. > >> > >> NIL deref, as a runtime error, should in concept happen when and where ^ is applied > >> to a pointer, including the implied dereference when a subscript or field selection is > >> applied directly without the ^ operator. It's purely an implementation question how to > >> do this. If a segfault message will point to the line of code where the dereference > >> happens, and before execution proceeds beyond that code, that would be marginally OK. > >> (Only marginally, because the error message should be able to say with certainty that > >> it's a NIL deref. > >> > >> Fortunately, in safe Modula-3 code, there are not a lot of other ways to segfault > >> (any?). Unfortunately, there's plenty of unsafe code, and if its programmer fails his > >> obligation, its effects can escape into safe code. > >> > >>> > >>> - Jay > >>> > >>> On Aug 26, 2015, at 8:37 AM, "Rodney M. Bates" wrote: > >>> > >>>> I have always thought "segfault, could it be a NIL pointer deref?", or whatever > >>>> words to that effect, was a rather lame excuse of an error message, and not > >>>> much help. > >>>> > >>>> Since this scheme doesn't fault at the point of the real deref, but afterwards, when a > >>>> memory reference is actually made, possibly computed from that address, could the location > >>>> given for the fault (whether line no or just code displacement) be wrong? Very wrong? > >>>> > >>>> TYPE A = ARRAY [ 0 .. 121 ] OF INTEGER; > >>>> > >>>> PROCEDURE Q ( ) > >>>> = VAR Ptr : REF A := NIL; > >>>> BEGIN > >>>> P(Ptr^); <---deref occurs here > >>>> END Q; > >>>> > >>>> .... lots of other code, different module, etc. ... > >>>> > >>>> PROCEDURE P ( VAR x : A ) > >>>> = BEGIN > >>>> x[119] := 17 <---memory ref & segfault don't happen until here. > >>>> END P; > >>>> > >>>> Or is this scheme only used in selective places, such as implicit deref > >>>> when putting a subscript onto a pointer? e.g.: > >>>> > >>>> Ptr[117] := 43; > >>>> > >>>> On 08/25/2015 07:28 PM, Jay K wrote: > >>>>> Of course, my agenda is to remove target-specificity. > >>>>> > >>>>> > >>>>> > >>>>> Target.First_readable_addr is a target-specific optimization > >>>>> to omit null checks. Or maybe range checks.. > >>>>> > >>>>> > >>>>> The assumption is that dereferencing "actual" null is checked > >>>>> by the hardware, so code doesn't have to be generated to check for null. > >>>>> > >>>>> > >>>>> I'm not even sure I like this -- as the behavior of dereferencing null is not portable. > >>>>> > >>>>> > >>>>> The assumption is further that accesses to the first page, at least, > >>>>> are subject to the same hardware checks, and can also be optimized away. > >>>>> > >>>>> > >>>>> At first, glance, I thought this was based on offset + size of a static access. > >>>>> > >>>>> > >>>>> For example: > >>>>> > >>>>> a: ARRAY [0...large] OF CHAR > >>>>> > >>>>> > >>>>> a[0..4095] no check > >>>>> a[4096 and up] would check > >>>>> > >>>>> > >>>>> Target.First_deadable_addr is 4k on many targets; 8k on others. > >>>>> Setting it too low would yield extra checks. > >>>>> Setting it too high would yield missing checks and a loss of safety. > >>>>> > >>>>> > >>>>> Here is what I actually found though. > >>>>> > >>>>> > >>>>> - The check is based on the size of the type being accessed. > >>>>> > >>>>> - It is off by a factor of 8 -- there is confusing between m3middle and m3front > >>>>> as to if it is bit number of a byte number. > >>>>> > >>>>> > >>>>> small: ARRAY[0..100] OF CHAR > >>>>> large:ARRAY[0..100000] OF CHAR > >>>>> > >>>>> no access to small gets checked and every access to larger gets checked. > >>>>> > >>>>> Should we do anything about this? > >>>>> > >>>>> In m3-sys/m3tests/src/p2/p263: > >>>>> cm3 -realclean > >>>>> cm3 -keep > >>>>> grep fault /*ms > >>>>> > >>>>> > >>>>> All the accesses are to offset 0. > >>>>> So, by some expectation, no null checks are needed. > >>>>> Null checks are output when the size of the > >>>>> containing type is, for x86, larger than 4096*8. > >>>>> > >>>>> > >>>>> The checks have been largely but not completely wrong/missing. > >>>>> Safety behooves us to check though. > >>>>> > >>>>> - fix the factor of 8? > >>>>> - make it 0?? too slow? > >>>>> - make it 4k on all target? until such time as a target manifests with a smaller page size? > >>>>> - base the checks on access offset + size, not containing size? > >>>>> Containing size is conservative. It checks more than i think is meant. > >>>>> > >>>>> > >>>>> I couldn't actually figure out the code here, I added various: > >>>>> > >>>>> IF RTParams.IsPresent("qual") THEN > >>>>> RTIO.PutText("NilChkExpr.Compile p.offset:" & Fmt.Int(p.offset) & "\n"); > >>>>> RTIO.Flush(); > >>>>> END; > >>>>> > >>>>> and such to m3front to figure it out. > >>>>> > >>>>> > >>>>> Thanks, > >>>>> - Jay > >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> M3devel mailing list > >>>>> M3devel at elegosoft.com > >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>> -- > >>>> Rodney Bates > >>>> rodney.m.bates at acm.org > >>> > >> > >> -- > >> Rodney Bates > >> rodney.m.bates at acm.org > >> _______________________________________________ > >> M3devel mailing list > >> M3devel at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Aug 28 21:45:16 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 28 Aug 2015 14:45:16 -0500 Subject: [M3devel] llvm backend integration Message-ID: <55E0BA4C.7060802@lcwb.coop> The llvm back end now has an initial degree of integration into cm3. To get it, add M3_BACKEND_MODE="StAloneLlvmAsm" near the front of the m3makefile, or elsewhere in quake code that will get interpreted. It does the following: Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm bitcode IR, in file *.mb or *.ib. Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. Runs 'as' to assemble to object code, in file *.mo or *.io. Prerequisites: llc on the path. m3llvm on the path. as on the path. In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, see the m3makefile of llvm, which must give their locations. 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR and llvm IR are created, but you can get human-readable versions of them with 'm3cgcat' and 'llvm-dis'. Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line parameters to m3llvm. (Other targets are not supported). -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:17:05 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:17:05 -0500 Subject: [M3devel] llvm backend integration In-Reply-To: <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> References: <55E0BA4C.7060802@lcwb.coop> <99800DED-0690-4BE0-9981-B1170A510CB3@purdue.edu> Message-ID: <55E1F721.1090106@lcwb.coop> On 08/28/2015 07:08 PM, Antony Hosking wrote: > Sounds great! What's missing? > No provision for starting from an intermediate file format, as there is for m3cc. Also, it might be nice to get the human-readable forms of the two IRs as side outputs, for development work. Just a convenience, though. And, it just feels like Builder.m3 could use a good code review in general. > Sent from my iPad > >> On 29 Aug 2015, at 5:45 am, Rodney M. Bates wrote: >> >> The llvm back end now has an initial degree of integration into cm3. To get it, add >> >> M3_BACKEND_MODE="StAloneLlvmAsm" >> >> near the front of the m3makefile, or elsewhere in quake code that will get interpreted. >> >> It does the following: >> >> Runs the M3 front end, producing cm3 IR in a *.mc or *.mi file. >> >> Runs the stand-alone program 'm3llvm', compiled by package m3-sys/llvm, to produce llvm >> bitcode IR, in file *.mb or *.ib. >> >> Runs 'llc' (a main program that is part of llvm), to produce assembly code, in file *.ms or *.is. >> >> Runs 'as' to assemble to object code, in file *.mo or *.io. >> >> Prerequisites: >> llc on the path. >> m3llvm on the path. >> as on the path. >> >> In turn, building m3llvm will require llvm 3.5.0 headers and compiled libraries, >> see the m3makefile of llvm, which must give their locations. >> >> 'cm3 -keep' will retain all the intermediate files. Only binary forms of the cm3 IR >> and llvm IR are created, but you can get human-readable versions of them with >> 'm3cgcat' and 'llvm-dis'. >> >> Look in m3-sys/cminstall/src/config-no-install/Unix.common, (which is installed to >> /usr/local/cm3/bin/config/Unix.common) to make simple alterations to the command-line >> parameters to m3llvm. (Other targets are not supported). >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Aug 29 20:54:46 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 29 Aug 2015 13:54:46 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop> Message-ID: <55E1FFF6.1070102@lcwb.coop> On 08/29/2015 06:02 AM, Peter McKinna wrote: > Hi Rodney, > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, instead of the 3.5.0. > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > upgrade.py worked OK for me after I went back to the release compiler and did it from there, which is the case we really care about. Llvm changes themselves should not affect the upgrade process, since no llvm mode is used in building the distribution. > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm changed the debug info data structure design to make it not derived from Value. Not only did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, though systematic and simple. These are in separate M3 package llvmbindings. In the case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather than just what is currently used. I still am uncomfortable about how the complex C++ type hierarchies should be reflected in the bindings. I doesn't look feasible to reflect these into true M3 hierarcharies. But clients need some kind of conversions. Some functions return results that need to be passed back in later, but to a parameter having a static subtype or supertype thereof. The unwrap functions used in llvm appear to sometimes just silently convert (the equivalent of) failing NARROWs into NIL pointers, rather than runtime errors. This will inevitably make some bugs much harder to track down. I'm not even sure dynamic type checks alway happen. Maybe some undetected type errors can propagate into C++ and give inscrutable segfaults. I am hoping future llvm releases will not affect this so much. They do change things a lot. I also like to avoid chasing the new releases too often. Just as I was getting llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. This all compiled and linked, as last I checked, but I have done very little execution checking. These do require getting some llvm stuff in the right places and states. I have had similar frustrations as Jay has with trying to use git branches, so for now, I just made this a separate package. They generate alternative executables with comparable function, so should be easy to switch between. Eventually, we will have to decide what to do about getting needed llvm stuff in, in a standard place and also, how much work will be done outside of the cm3 build system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc would be cleanest for users, but it is very big, takes hours to compile, and maybe not everybody will want to use it badly enough to pay that price. Right now, you have to do building and installing of llvm on the side, then edit a few paths in m3makefiles. Keeping m3llvm as a completely separate executable means those who don't want to use it can be unbothered. With its still being in major development, this seems best for now. I originally set out on the path of having the functions of m3llvm linked in to cm3, as with the C backend, but am skeptical about all the llvm libraries that have to be built and linked in. > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > Those things will take some work, but should not be conceptually difficult. > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > Regards Peter > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > Peter, > > How actively are you working on the llvm back end? I don't see a lot of updates to > the github repository. I have checked in some minor things and think I am about > ready to do more. But I think it best we keep things merged as much as reasonable. > > One change is adding specialized support for calls on alloca. These are very > recently being generated as library calls by the front end, and come through > the whole process as link failures. I think it best to detect these and convert > to llvm alloca instructions. m3cc does something like this now. > > > -- > Rodney Bates > rodney.m.bates at acm.org > > -- Rodney Bates rodney.m.bates at acm.org From lemming at henning-thielemann.de Sat Aug 29 21:56:33 2015 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Sat, 29 Aug 2015 21:56:33 +0200 (CEST) Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> Message-ID: On Sat, 29 Aug 2015, Rodney M. Bates wrote: > I still am uncomfortable about how the complex C++ type hierarchies > should be reflected in the bindings. In Haskell I am calling the C interface to LLVM, like certainly many other high-level language bindings do. Would this be an option for Modula-3 bindings to LLVM, too? However, the C interface of LLVM is not well documented and maybe not complete. From jay.krell at cornell.edu Sun Aug 30 00:12:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:12:37 +0000 Subject: [M3devel] paths too truncated in assertion failures Message-ID: These messages: ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** 1) It should really be a full path. I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. Yes, full paths could "leak" across machines but I think that is ok. I did work on this long ago but people disagreed with at the time. 2) We could/should trim the root of the git checkout, so it sayse.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. Some C compilers have that feature, when people are conceredwith cross machine consistency for the sake of caching and similarbuild optimizations. I still prefer #1. 3) Or at least m3front/src/convert/Convert.m33b) or ../../m3front/src/convert/Convert.m3 if we really want it to be a correct relative path from the target directory. A big part of the problem is the build system is geared toward packages,not multiple packages. I'd like to solve this problem too somehow someday. Really -- #1 -- source paths should be full paths. C compilers vary here btw, in terms of what __FILE__ is and what isin debugging information.Sometimes whatever is passed on the command line is used.Sometimes full paths are computed by the compiler. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 00:20:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 29 Aug 2015 22:20:16 +0000 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E1FFF6.1070102@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: I understand your dilemas.Static linking vs. dynamic.Cloning the source or not.Significantly growing our repository or not.Forking or not -- do we need any patches? Hopefully not but ok if we do. Clearly the C backend cheats -- the C compiler appears on the systemas if by magic, or various external means. I am somewhat keen on seeing this LLVM stuff work.And somewhat interested in helping. First, you know, there are many sayings and their redundancies and opposites: 1a too many cooks spoil the broth 1b 9 women cannot have a baby in one month (but they can have 9 in 9) 2 more/many hands make less/little work Which is it here, 1 or 2?i.e. do you need/want help? I have built LLVM now a few times.Yes it is large and takes time.Do we have any required patches for it?I'm still on the fence as to if we should "import" it, vs. depend on its presence"somehow". If there really is version sensitivity, that argues for importing.While "apt-get install llvm" might be in widespread availability,the exact "apt-get install llvm-3.5.0" I imagine isn't. The current setup obviously needs work.I'll likely commit something at least a little better. Do we really need 3.5.0, or can I use 3.5.2? > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > problems other people have had, guess I will have to trust upgrade.py. Really, I run this stuff all the time. On various hosts. What you/we/I do need though is a safe way to rebuild the compiler when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. In particular, if you have an ABI change, you need two different m3cores. The one for the working compiler and the one for the new compiler. Maybe I just need to use "buildlocal" more. It doesn't help that the C backend isn't ABI compatible with cm3cg, due to nested functions and static links. cm3cg uses a target-dependent extra register not accessible to C as far as I know. C backend uses an extra last parameter. - Jay > Date: Sat, 29 Aug 2015 13:54:46 -0500 > From: rodney_bates at lcwb.coop > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > Subject: Re: [M3devel] Modula-3 llvm backend > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > Hi Rodney, > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > instead of the 3.5.0. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > from there, which is the case we really care about. Llvm changes themselves should > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > changed the debug info data structure design to make it not derived from Value. Not only > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > though systematic and simple. These are in separate M3 package llvmbindings. In the > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > than just what is currently used. > > I still am uncomfortable about how the complex C++ type hierarchies should be > reflected in the bindings. I doesn't look feasible to reflect these into true > M3 hierarcharies. But clients need some kind of conversions. Some functions > return results that need to be passed back in later, but to a parameter having > a static subtype or supertype thereof. The unwrap functions used in llvm appear > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > pointers, rather than runtime errors. This will inevitably make some bugs much > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > I am hoping future llvm releases will not affect this so much. They do change things > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > This all compiled and linked, as last I checked, but I have done very little execution > checking. These do require getting some llvm stuff in the right places and states. > I have had similar frustrations as Jay has with trying to use git branches, so > for now, I just made this a separate package. They generate alternative executables > with comparable function, so should be easy to switch between. > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > in a standard place and also, how much work will be done outside of the cm3 build > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > would be cleanest for users, but it is very big, takes hours to compile, and maybe > not everybody will want to use it badly enough to pay that price. Right now, you > have to do building and installing of llvm on the side, then edit a few paths in > m3makefiles. > > Keeping m3llvm as a completely separate executable means those who don't want to use > it can be unbothered. With its still being in major development, this seems best > for now. I originally set out on the path of having the functions of m3llvm linked > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > have to be built and linked in. > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > Those things will take some work, but should not be conceptually difficult. > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > Regards Peter > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > Peter, > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > the github repository. I have checked in some minor things and think I am about > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > One change is adding specialized support for calls on alloca. These are very > > recently being generated as library calls by the front end, and come through > > the whole process as link failures. I think it best to detect these and convert > > to llvm alloca instructions. m3cc does something like this now. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:45:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? Message-ID: The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Aug 30 09:57:58 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 30 Aug 2015 07:57:58 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: On the IA64 matter, it appears the bundle is the lower 5 bits.And the bundles 6, 7, 14, 15, 1A, 1B, 1E, 1F are invalid.It appears IA64 can be little or big endian, but the instructionsare always little endian.Therefore a 64bit value of -1 would seem invalid and ok for a marker,but 0 is less clear -- might still be invalid. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Sun, 30 Aug 2015 07:45:21 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? The agenda remains seeing if Target variables can be made constants. The discussion in this case is more complicated and some facts are unclear. Background: Nested functions are a problem.In particular, if you can take their address.Taking the address of a nested function presents a problem.You are presented with two or three solutions. - runtime code gen - either on the stack - or somewhere more expensive, possibly with garbage collection - possibly "templated" instead of "arbitrary"; the meaning of this is a lot to explain -- related to libffi and mremap, which isn't entirely portable, but is e.g. portable to Windows - or instead of runtime codegen, altering how function pointers are called; you can only do this from Modula-3 code, not e.g. C or C++. The solution Modula-3 has taken is to alter how funtion pointers are called.The sequence is roughly: Check if it is a "regular" function pointer or a "closure". If it is a "regular" function pointer, just call it. If it is a "closure", it contains a function pointer and a "static link". Call the function pointer, passing the static link. To tell if it is "regular" function pointer or a "closure", roughly what is done is the data at the function pointer is read and compared against a marker value. If it equals the marker value, it is a closure. The size of the marker is the size of an integer or pointer (4 or 8 bytes). The value of the marker checked for is 0 or -1, I'd have to check. The alignment of the pointer might be a factor. In particular, on most architectures, all instructions have a certain alignment. If the pointer has less alignment, it can't be an instruction. Or maybe on those architectures, the bytes are read one at a time to avoid alignment faults. In particular, as far as I know, the following: x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all 4 bytes and 4 byte aligned, so functions are at least also as much arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction pointer is actually odd as well, and the low bit is removed to really find the instructions That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned I could use confirmation on much of this. I find the use of a marker value a little dubious. It'd be good to research if there is one value that works on all. I find the choice of a marker size to be pointer-sized dubious on most platforms. In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits for the marker value doesn't buy much. If the marker value is actually a legal instruction, then checking for two in a row reduces the odds of a false positive. However, given that the closure is a marker and two pointers, it isn't like you are going to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. However, I want less target-variation not more. Here are some my lingering questions: - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? - Is a 64bit marker value actually sufficient on IA64? The way to help here, I think, is to ensure that a 64bit marker, not a 128bit marker, contains the "template", and an invalid "template". - Barring the previous, a solution might be to use a 128 bit marker on all platforms. i believe all of these function pointers are rare. I hope/believe the object method calls do not check for closures -- though actually that is related to a useful language construct, that I doubt we have. The simplest solution is likely: - ignore IA64, or research it further - keep marker size at integer - for the C backend, assume no alignment of function pointers -- give up any of the optimization, esp. x86/amd64. For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. Thoughts? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Aug 30 19:26:21 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:26:21 -0500 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <55E33CBD.307@lcwb.coop> I too have long wished for a longer path in these messages. Although all source files in a package are required to have unique simple names, I doubt this is true between packages, except for visible interfaces. Even when unique, unless you have it all memorized, it is just an extra step to find a source file from only a simple name, when you are trying concentrate on the bug. "../src/" usually provides no information. I would be happy if the paths only went back as far as, say, cm3, for things in the repo, but how would we define where to stop for outside code? In a way, I think a full path as on the machine where the program was compiled might be more helpful anyway. Otherwise, that is really lost information, if you have any reason to care about it. A partial path that translates in an apparent way from the compiling to the executing machine could even be misleading, as the files could be quite different. OTOH, this can happen even when it's all the same machine, though probably less likely. On 08/29/2015 05:12 PM, Jay K wrote: > These messages: > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/convert/Convert.m3", line 47 > *** > > > 1) It should really be a full path. > > I know people will disagree with me. > You want more commonality across machines. > I'm not sure that is worth it. > In particular, debuggers always work more easily with full paths, for local private builds. > Hopefully for debugging someone else's, some search path > with "prefix replacement" is viable. > But debugging your own build is more common and ideally > no special setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > > 2) We could/should trim the root of the git checkout, so it says > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > Some C compilers have that feature, when people are concered > with cross machine consistency for the sake of caching and similar > build optimizations. > > > I still prefer #1. > > > 3) Or at least m3front/src/convert/Convert.m3 > 3b) or ../../m3front/src/convert/Convert.m3 > > > if we really want it to be a correct relative path from the target directory. > > > A big part of the problem is the build system is geared toward packages, > not multiple packages. I'd like to solve this problem too somehow someday. > > > Really -- #1 -- source paths should be full paths. > > C compilers vary here btw, in terms of what __FILE__ is and what is > in debugging information. > Sometimes whatever is passed on the command line is used. > Sometimes full paths are computed by the compiler. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 19:52:59 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 12:52:59 -0500 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: References: <55E0BBEC.2080905@lcwb.coop>, , <55E1FFF6.1070102@lcwb.coop> Message-ID: <55E342FB.7060503@lcwb.coop> On 08/29/2015 05:20 PM, Jay K wrote: > I understand your dilemas. > Static linking vs. dynamic. > Cloning the source or not. > Significantly growing our repository or not. > Forking or not -- do we need any patches? Hopefully not but ok if we do. > > > Clearly the C backend cheats -- the C compiler appears on the system > as if by magic, or various external means. > > > I am somewhat keen on seeing this LLVM stuff work. > And somewhat interested in helping. > > > First, you know, there are many sayings and their redundancies and opposites: > > 1a too many cooks spoil the broth > 1b 9 women cannot have a baby in one month (but they can have 9 in 9) > > 2 more/many hands make less/little work > Which is it here, 1 or 2? > i.e. do you need/want help? > Help is surely needed/wanted, as long as we don't start too much tripping over each other. Just exactly where needs some thought. > I have built LLVM now a few times. > Yes it is large and takes time. > Do we have any required patches for it? I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain it is to retrofit patches to new versions. But that may be unrealistic. The last I checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be fixed in the llvm source tree. Maybe we can get them to do that for us. > I'm still on the fence as to if we should "import" it, vs. depend on its presence > "somehow". > > If there really is version sensitivity, that argues for importing. There is definitely version sensitivity. There were big changes in the data structure for debug info and the calls to construct it, somewhere between 3.5.0 and 3.6.1, require big rework of bindings (which they do not provide.) They do not hesitate to change interface things. There has been a roaring discussion about whether/how to start keeping the "Core" interface stable--something apparently has not be a goal so far. Core is a C interface they provide to certain things, notably not debug info. > While "apt-get install llvm" might be in widespread availability, > the exact "apt-get install llvm-3.5.0" I imagine isn't. > > > The current setup obviously needs work. > I'll likely commit something at least a little better. > > Do we really need 3.5.0, or can I use 3.5.2? > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything pretending to be a binding for. I prefer to at least start with 3.6.1, since it is later and significantly different, so will require just that much less retrofitting. > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the > > problems other people have had, guess I will have to trust upgrade.py. > > Really, I run this stuff all the time. > On various hosts. > What you/we/I do need though is a safe way to rebuild the compiler > when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. > In particular, if you have an ABI change, you need two different m3cores. > The one for the working compiler and the one for the new compiler. > Maybe I just need to use "buildlocal" more. > It doesn't help that the C backend isn't ABI compatible with cm3cg, due to > nested functions and static links. cm3cg uses a target-dependent extra > register not accessible to C as far as I know. C backend uses an extra > last parameter. > > - Jay > > > > > > Date: Sat, 29 Aug 2015 13:54:46 -0500 > > From: rodney_bates at lcwb.coop > > To: peter.mckinna at gmail.com; m3devel at elegosoft.com > > Subject: Re: [M3devel] Modula-3 llvm backend > > > > > > > > On 08/29/2015 06:02 AM, Peter McKinna wrote: > > > Hi Rodney, > > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. > > > > > > > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, > > instead of the 3.5.0. > > > > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. > > > > > > > upgrade.py worked OK for me after I went back to the release compiler and did it > > from there, which is the case we really care about. Llvm changes themselves should > > not affect the upgrade process, since no llvm mode is used in building the distribution. > > > > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. > > > > > > > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm > > changed the debug info data structure design to make it not derived from Value. Not only > > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, > > though systematic and simple. These are in separate M3 package llvmbindings. In the > > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather > > than just what is currently used. > > > > I still am uncomfortable about how the complex C++ type hierarchies should be > > reflected in the bindings. I doesn't look feasible to reflect these into true > > M3 hierarcharies. But clients need some kind of conversions. Some functions > > return results that need to be passed back in later, but to a parameter having > > a static subtype or supertype thereof. The unwrap functions used in llvm appear > > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL > > pointers, rather than runtime errors. This will inevitably make some bugs much > > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe > > some undetected type errors can propagate into C++ and give inscrutable segfaults. > > > > I am hoping future llvm releases will not affect this so much. They do change things > > a lot. I also like to avoid chasing the new releases too often. Just as I was getting > > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. > > > > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. > > This all compiled and linked, as last I checked, but I have done very little execution > > checking. These do require getting some llvm stuff in the right places and states. > > I have had similar frustrations as Jay has with trying to use git branches, so > > for now, I just made this a separate package. They generate alternative executables > > with comparable function, so should be easy to switch between. > > > > Eventually, we will have to decide what to do about getting needed llvm stuff in, > > in a standard place and also, how much work will be done outside of the cm3 build > > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc > > would be cleanest for users, but it is very big, takes hours to compile, and maybe > > not everybody will want to use it badly enough to pay that price. Right now, you > > have to do building and installing of llvm on the side, then edit a few paths in > > m3makefiles. > > > > Keeping m3llvm as a completely separate executable means those who don't want to use > > it can be unbothered. With its still being in major development, this seems best > > for now. I originally set out on the path of having the functions of m3llvm linked > > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that > > have to be built and linked in. > > > > > > > > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. > > > > > > > Those things will take some work, but should not be conceptually difficult. > > > > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. > > > > > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. > > > > > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. > > > > > > Regards Peter > > > > > > > > > > > > > > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: > > > > > > Peter, > > > > > > How actively are you working on the llvm back end? I don't see a lot of updates to > > > the github repository. I have checked in some minor things and think I am about > > > ready to do more. But I think it best we keep things merged as much as reasonable. > > > > > > One change is adding specialized support for calls on alloca. These are very > > > recently being generated as library calls by the front end, and come through > > > the whole process as link failures. I think it best to detect these and convert > > > to llvm alloca instructions. m3cc does something like this now. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Aug 30 20:24:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 30 Aug 2015 13:24:55 -0500 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: References: Message-ID: <55E34A77.1050001@lcwb.coop> On 08/30/2015 02:45 AM, Jay K wrote: > The agenda remains seeing if Target variables can be made constants. > > The discussion in this case is more complicated and some facts are unclear. > > > Background: > > > Nested functions are a problem. > In particular, if you can take their address. > Taking the address of a nested function presents a problem. > You are presented with two or three solutions. > > > - runtime code gen > - either on the stack > > - or somewhere more expensive, possibly with garbage collection > > - possibly "templated" instead of "arbitrary"; the meaning > of this is a lot to explain -- related to libffi and mremap, which > isn't entirely portable, but is e.g. portable to Windows > > > - or instead of runtime codegen, altering how function pointers are > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > The sequence is roughly: > Check if it is a "regular" function pointer or a "closure". > If it is a "regular" function pointer, just call it. > If it is a "closure", it contains a function pointer and a "static link". > Call the function pointer, passing the static link. > > > To tell if it is "regular" function pointer or a "closure", roughly > what is done is the data at the function pointer is read and compared > against a marker value. If it equals the marker value, it is a closure. > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > The value of the marker checked for is 0 or -1, I'd have to check. > The alignment of the pointer might be a factor. In particular, on most > architectures, all instructions have a certain alignment. If the pointer has > less alignment, it can't be an instruction. Or maybe on those architectures, > the bytes are read one at a time to avoid alignment faults. > > > In particular, as far as I know, the following: > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > 4 bytes and 4 byte aligned, so functions are at least also as much > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > pointer is actually odd as well, and the low bit is removed to really find the instructions > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > I could use confirmation on much of this. > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > value that works on all. > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > then checking for two in a row reduces the odds of a false positive. > > > However, given that the closure is a marker and two pointers, it isn't like you are going > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a valid instruction or sequence thereof. So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity here. Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for certain (especially if we want to be able to call procedures produced by other compilers) but much less likely to result in a problem. Just a wild guess, but I would not be surprised if ELF and other object formats would require the machine code of a function/procedure to begin on a native word boundary, even if the hardware instruction set does not. Where so, this would obviate checking the alignment before checking for a closure, though probably target-dependently. > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > However, I want less target-variation not more. > > > Here are some my lingering questions: > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > - Is a 64bit marker value actually sufficient on IA64? > The way to help here, I think, is to ensure that a 64bit marker, > not a 128bit marker, contains the "template", and an invalid "template". > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > i believe all of these function pointers are rare. > I hope/believe the object method calls do not check for closures -- though actually > that is related to a useful language construct, that I doubt we have. > Method body procedures are required to be top-level, ensured statically, so there is no need for method call code to consider the possibility of the pointer in the object type to be a closure. > > The simplest solution is likely: > - ignore IA64, or research it further > - keep marker size at integer Pointer would be target-independent in getting the following two pointers aligned. > - for the C backend, assume no alignment of function pointers -- give up > any of the optimization, esp. x86/amd64. I think this optimization both applies to a low-frequency situation and has a very small benefit, so I would not worry about giving up on it. > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > Thoughts? > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Aug 30 22:45:18 2015 From: jay.krell at cornell.edu (Jay) Date: Sun, 30 Aug 2015 13:45:18 -0700 Subject: [M3devel] Modula-3 llvm backend In-Reply-To: <55E342FB.7060503@lcwb.coop> References: <55E0BBEC.2080905@lcwb.coop> <55E1FFF6.1070102@lcwb.coop> <55E342FB.7060503@lcwb.coop> Message-ID: <6B3E0C66-4A5A-44FF-9910-42DD01413214@gmail.com> I'm kind of impatient, sorry, that will be reflected here: - if you have a known working version, stick with it for now? - maintain multiple bindings? - patches: good & bad: it can be fun to fork, but also wasteful; maintaining diff/patch files may be an option. - Proceed without dwarf? If much else to do? You certain the problem is in LLVM? Your patches totally fix the problem. - Aomewhat interesting, m3x86 & m3cc for *some* platforms, e.g. Darwin, produce a minimum of debugging info, kinda surprising, & people don't seem to notice; I have largely fixed this with the C backend though & intend to fix it more. - Presumably I can/should expand beyond AMD_LINUX. I was going to use I386_DARWIN, but until I convince LLVM to be that, probably AMD64_DARWIN. - Jay On Aug 30, 2015, at 10:52 AM, "Rodney M. Bates" wrote: > > > On 08/29/2015 05:20 PM, Jay K wrote: >> I understand your dilemas. >> Static linking vs. dynamic. >> Cloning the source or not. >> Significantly growing our repository or not. >> Forking or not -- do we need any patches? Hopefully not but ok if we do. >> >> >> Clearly the C backend cheats -- the C compiler appears on the system >> as if by magic, or various external means. >> >> >> I am somewhat keen on seeing this LLVM stuff work. >> And somewhat interested in helping. >> >> >> First, you know, there are many sayings and their redundancies and opposites: >> >> 1a too many cooks spoil the broth >> 1b 9 women cannot have a baby in one month (but they can have 9 in 9) >> >> 2 more/many hands make less/little work >> Which is it here, 1 or 2? >> i.e. do you need/want help? > > Help is surely needed/wanted, as long as we don't start too much tripping > over each other. Just exactly where needs some thought. > >> I have built LLVM now a few times. >> Yes it is large and takes time. >> Do we have any required patches for it? > > I am *really* hoping not. Jay, you know well from gcc, and I from gdb, what a royal pain > it is to retrofit patches to new versions. But that may be unrealistic. The last I > checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be > fixed in the llvm source tree. Maybe we can get them to do that for us. > >> I'm still on the fence as to if we should "import" it, vs. depend on its presence >> "somehow". >> >> If there really is version sensitivity, that argues for importing. > > There is definitely version sensitivity. There were big changes in the > data structure for debug info and the calls to construct it, somewhere > between 3.5.0 and 3.6.1, require big rework of bindings (which they do > not provide.) They do not hesitate to change interface things. There > has been a roaring discussion about whether/how to start keeping the > "Core" interface stable--something apparently has not be a goal so far. > Core is a C interface they provide to certain things, notably not > debug info. > >> While "apt-get install llvm" might be in widespread availability, >> the exact "apt-get install llvm-3.5.0" I imagine isn't. >> >> >> The current setup obviously needs work. >> I'll likely commit something at least a little better. >> >> Do we really need 3.5.0, or can I use 3.5.2? > > It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything > pretending to be a binding for. I prefer to at least start with 3.6.1, since it is > later and significantly different, so will require just that much less retrofitting. > >> >> > So I will be upgrading asap. I am always a bit wary of upgrading seeing the >> > problems other people have had, guess I will have to trust upgrade.py. >> >> Really, I run this stuff all the time. >> On various hosts. >> What you/we/I do need though is a safe way to rebuild the compiler >> when it might be buggy. I have ways, but they aren't great/known/easy/incrmental. >> In particular, if you have an ABI change, you need two different m3cores. >> The one for the working compiler and the one for the new compiler. >> Maybe I just need to use "buildlocal" more. >> It doesn't help that the C backend isn't ABI compatible with cm3cg, due to >> nested functions and static links. cm3cg uses a target-dependent extra >> register not accessible to C as far as I know. C backend uses an extra >> last parameter. >> >> - Jay >> >> >> >> >> > Date: Sat, 29 Aug 2015 13:54:46 -0500 >> > From: rodney_bates at lcwb.coop >> > To: peter.mckinna at gmail.com; m3devel at elegosoft.com >> > Subject: Re: [M3devel] Modula-3 llvm backend >> > >> > >> > >> > On 08/29/2015 06:02 AM, Peter McKinna wrote: >> > > Hi Rodney, >> > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work. >> > > >> > >> > If nobody objects, I would like to start working on the 3.6.1 version (see below) now, >> > instead of the 3.5.0. >> > >> > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py. >> > > >> > >> > upgrade.py worked OK for me after I went back to the release compiler and did it >> > from there, which is the case we really care about. Llvm changes themselves should >> > not affect the upgrade process, since no llvm mode is used in building the distribution. >> > >> > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think. >> > > >> > >> > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm >> > changed the debug info data structure design to make it not derived from Value. Not only >> > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes, >> > though systematic and simple. These are in separate M3 package llvmbindings. In the >> > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather >> > than just what is currently used. >> > >> > I still am uncomfortable about how the complex C++ type hierarchies should be >> > reflected in the bindings. I doesn't look feasible to reflect these into true >> > M3 hierarcharies. But clients need some kind of conversions. Some functions >> > return results that need to be passed back in later, but to a parameter having >> > a static subtype or supertype thereof. The unwrap functions used in llvm appear >> > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL >> > pointers, rather than runtime errors. This will inevitably make some bugs much >> > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe >> > some undetected type errors can propagate into C++ and give inscrutable segfaults. >> > >> > I am hoping future llvm releases will not affect this so much. They do change things >> > a lot. I also like to avoid chasing the new releases too often. Just as I was getting >> > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming. >> > >> > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings. >> > This all compiled and linked, as last I checked, but I have done very little execution >> > checking. These do require getting some llvm stuff in the right places and states. >> > I have had similar frustrations as Jay has with trying to use git branches, so >> > for now, I just made this a separate package. They generate alternative executables >> > with comparable function, so should be easy to switch between. >> > >> > Eventually, we will have to decide what to do about getting needed llvm stuff in, >> > in a standard place and also, how much work will be done outside of the cm3 build >> > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc >> > would be cleanest for users, but it is very big, takes hours to compile, and maybe >> > not everybody will want to use it badly enough to pay that price. Right now, you >> > have to do building and installing of llvm on the side, then edit a few paths in >> > m3makefiles. >> > >> > Keeping m3llvm as a completely separate executable means those who don't want to use >> > it can be unbothered. With its still being in major development, this seems best >> > for now. I originally set out on the path of having the functions of m3llvm linked >> > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that >> > have to be built and linked in. >> > >> > >> > >> > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions. >> > > >> > >> > Those things will take some work, but should not be conceptually difficult. >> > >> > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder. >> > > >> > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical. >> > > >> > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun. >> > > >> > > Regards Peter >> > > >> > >> > >> > >> > > >> > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates > wrote: >> > > >> > > Peter, >> > > >> > > How actively are you working on the llvm back end? I don't see a lot of updates to >> > > the github repository. I have checked in some minor things and think I am about >> > > ready to do more. But I think it best we keep things merged as much as reasonable. >> > > >> > > One change is adding specialized support for calls on alloca. These are very >> > > recently being generated as library calls by the front end, and come through >> > > the whole process as link failures. I think it best to detect these and convert >> > > to llvm alloca instructions. m3cc does something like this now. >> > > >> > > >> > > -- >> > > Rodney Bates >> > > rodney.m.bates at acm.org >> > > >> > > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> > _______________________________________________ >> > M3devel mailing list >> > M3devel at elegosoft.com >> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 08:26:32 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 06:26:32 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <55E33CBD.307@lcwb.coop> References: , <55E33CBD.307@lcwb.coop> Message-ID: It isn't perfectly scientific, as even full paths on a machinecan have variable meaning through time. For example, I can delete some source, put other source in its place,similar or different. And on Windows there is the "net use" feature where drive lettersmap to the network. Then you delete the use, and make another one,same letter, different place. Similar to mounting on Unixy systems. Here is another story in this area.Hopefully I have my facts correct.Perhaps it is just apocryphal and I filled in details to suit my agenda.We can double check it all with a bit of time. Visual C++, has, like any other system, some value for __FILE__and some values in the debugging information. A long time ago, they were both whatever you passed on the command line. At some point, a switch was added /FC that meant "full path the valuesin the debugging information". I believe this was added around Visual C++ 5.0.Later however, it was realized, that should always be how it works. The switchno longer does anything. Or maybe /FC still serves to full path __FILE__ but no longer has anyaffect on debugging information.I'd have to go back and try every compiler I have to verify. So, I think this actually motivate my other point from years ago,which is to full path the file names in the debug info, but not necessarily here.But I want both. :) I debug a lot of stuff, and it is unavoidably difficult, but some thingsjust make it unnecessarily more difficult and tedious. We should actually be recording crcs of source files and the assertcan go off and find the source and report if it matches.Or at least the assert can report the crc and user can verify it himself.i.e. once an assert fails, you can't do much reliably. And yes, when there is no CM3_ROOT or such, unclear what partof the path to truncate.Which motivates the other option -- don't truncate anything. I believe the names have to be unique "across the link".Visible or not, I'm not sure it matters.I don't believe we use the "static" feature of C/.obj files,so even non public functions, and the globals, are in a shared globalnamespace. And with dynamic linking, you can't be sure they don't surviveinto the dynamic link search space at least on Unixy systems.. - Jay > Date: Sun, 30 Aug 2015 12:26:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] paths too truncated in assertion failures > > I too have long wished for a longer path in these messages. Although > all source files in a package are required to have unique simple names, > I doubt this is true between packages, except for visible interfaces. > Even when unique, unless you have it all memorized, it is just an extra > step to find a source file from only a simple name, when you are trying > concentrate on the bug. "../src/" usually provides no information. > > I would be happy if the paths only went back as far as, say, cm3, for > things in the repo, but how would we define where to stop for outside > code? In a way, I think a full path as on the machine where the program > was compiled might be more helpful anyway. Otherwise, that is really lost > information, if you have any reason to care about it. A partial path > that translates in an apparent way from the compiling to the executing > machine could even be misleading, as the files could be quite different. > > OTOH, this can happen even when it's all the same machine, though probably > less likely. > > On 08/29/2015 05:12 PM, Jay K wrote: > > These messages: > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file "../src/convert/Convert.m3", line 47 > > *** > > > > > > 1) It should really be a full path. > > > > I know people will disagree with me. > > You want more commonality across machines. > > I'm not sure that is worth it. > > In particular, debuggers always work more easily with full paths, for local private builds. > > Hopefully for debugging someone else's, some search path > > with "prefix replacement" is viable. > > But debugging your own build is more common and ideally > > no special setting is needed to make that work. > > > > Yes, full paths could "leak" across machines but I think that is ok. > > > > I did work on this long ago but people disagreed with at the time. > > > > > > 2) We could/should trim the root of the git checkout, so it says > > e.g. m3-sys/m3front/src/convert/Convert.m3 if that is easy. > > > > > > Some C compilers have that feature, when people are concered > > with cross machine consistency for the sake of caching and similar > > build optimizations. > > > > > > I still prefer #1. > > > > > > 3) Or at least m3front/src/convert/Convert.m3 > > 3b) or ../../m3front/src/convert/Convert.m3 > > > > > > if we really want it to be a correct relative path from the target directory. > > > > > > A big part of the problem is the build system is geared toward packages, > > not multiple packages. I'd like to solve this problem too somehow someday. > > > > > > Really -- #1 -- source paths should be full paths. > > > > C compilers vary here btw, in terms of what __FILE__ is and what is > > in debugging information. > > Sometimes whatever is passed on the command line is used. > > Sometimes full paths are computed by the compiler. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 15:31:29 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 15:31:29 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: Message-ID: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From lists at darko.org Mon Aug 31 16:26:59 2015 From: lists at darko.org (Darko Volaric) Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: > On Sat, 29 Aug 2015 22:12:37 +0000 > Jay K wrote: > > > These messages: > > ****** runtime error:*** <*ASSERT*> failed.*** file > "../src/convert/Convert.m3", line 47*** > > > > 1) It should really be a full path. > > I know people will disagree with me.You want more commonality across > machines.I'm not sure that is worth it.In particular, debuggers always work > more easily with full paths, for local private builds.Hopefully for > debugging someone else's, some search pathwith "prefix replacement" is > viable.But debugging your own build is more common and ideallyno special > setting is needed to make that work. > > Yes, full paths could "leak" across machines but I think that is ok. > > I did work on this long ago but people disagreed with at the time. > > I'd vote for full absolute path. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 31 17:06:12 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 31 Aug 2015 10:06:12 -0500 Subject: [M3devel] Oops, belated commit of llvm3.6.1 Message-ID: <55E46D64.1020806@lcwb.coop> I apparently forgot, a while back, to commit new package llvm3.6.1, which I have been writing about. It is now in the Github repo. It compiles, but segfaults, most likely because of type problems in llvmbindings. -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 31 20:02:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 18:02:07 +0000 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: , <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com>, Message-ID: And in compiler/warnings errors? i.e. do we need to preserve the "short" path in some cases? I vote for full path always. We can also provide options to trim specified prefixes, so builds at different paths get identical output. - Jay Date: Mon, 31 Aug 2015 07:26:59 -0700 Subject: Re: [M3devel] paths too truncated in assertion failures From: lists at darko.org To: wagner at elegosoft.com CC: jay.krell at cornell.edu; m3devel at elegosoft.com Absolute path is important for tools that read the output and use it. For instance my text editor jumps straight to the line in the source file when an assert occurs. On Mon, Aug 31, 2015 at 6:31 AM, Olaf Wagner wrote: On Sat, 29 Aug 2015 22:12:37 +0000 Jay K wrote: > These messages: > ****** runtime error:*** <*ASSERT*> failed.*** file "../src/convert/Convert.m3", line 47*** > > 1) It should really be a full path. > I know people will disagree with me.You want more commonality across machines.I'm not sure that is worth it.In particular, debuggers always work more easily with full paths, for local private builds.Hopefully for debugging someone else's, some search pathwith "prefix replacement" is viable.But debugging your own build is more common and ideallyno special setting is needed to make that work. > Yes, full paths could "leak" across machines but I think that is ok. > I did work on this long ago but people disagreed with at the time. I'd vote for full absolute path. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Aug 31 20:08:44 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 31 Aug 2015 20:08:44 +0200 Subject: [M3devel] paths too truncated in assertion failures In-Reply-To: References: <20150831153129.3a16e6b5a3277b84146e4399@elegosoft.com> Message-ID: <20150831200844.5b6c4c09e86238534fd1f659@elegosoft.com> On Mon, 31 Aug 2015 18:02:07 +0000 Jay K wrote: > And in compiler/warnings errors? > > i.e. do we need to preserve the "short" path in some cases? > > I vote for full path always. I second that. > > We can also provide options to trim specified prefixes, so > builds at different paths get identical output. > > - Jay -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194