From jayk123 at hotmail.com Wed Jun 1 11:28:08 2016 From: jayk123 at hotmail.com (Jay K) Date: Wed, 1 Jun 2016 09:28:08 +0000 Subject: [M3devel] [modula3/cm3] New release for Mac OS X, please? (#13) [Originally From]: notifications@github.com In-Reply-To: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> References: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> Message-ID: I kinda want to encourage ?a different approach, at least for Unix systems where binary compatibility is currently so low. So here is what I did/am doing: ?I have made assembly and C bootstraps for I386_LINUX and AMD64_LINUX.? ?I'm still testing them (waiting for boot2 to finish). ? ? ?Here is roughly how:? ? For assembly (gcc-bassed ) bootstrap:? ?cd ${CM3_ROOT}/m3-sys/m3cc? ? ?sh -c ./src/buildmany.sh? ? That actually does way more than necessary,? ? and might fail in the end. I didn't wait for it.? ? In particular, it builds a gcc-based backend for every target.? ?If you look -- buildmany.sh is trivial.? ?m3cc/src/m3makefile has always had the support. buildmany.sh is a tiny wrapper.? ? Actually the first step should be redundant with boot1.py anyway. ? ? cd ${CM3_ROOT}/scripts/python? ?# I don't think these are case sensitive even.? ?./do-cm3-all.py I386_LINUX realclean? ?./do-cm3-all.py I386_LINUX c realclean? ?./do-cm3-all.py AMD64_LINUX realclean? ?./do-cm3-all.py AMD64_LINUX c realclean? ?./boot1.py I386_LINUX c? ?./boot1.py I386_LINUX ? ?./boot1.py AMD64_LINUX c? ?./boot1.py AMD64_LINUX ? If you look, boot1.py doesn't seem obviously simple, but?it really is just a wrapper around roughly: ?cd ? ?cm3? ?mkdir? ?cp ? ?tar? ? ? ?"All else being equal", i.e. lacking broad adoption of the C or LLVM backends, ?we should make releases "this way" "and then some". ? ? ?"and then some": ? ?all supported/tested targets (Darwin, Solaris, FreeBSD, maybe NT). Enhancing it to include the entire system, installing a proper install, and supporting shared libraries. Shared libraries: either autotools or cmake or replicating the config files ? i.e. into pylib.py, or moving pylib.py logic into cm3, esp. cm3 -boot ?? Key advantages of this approach: ?1) It is cross building, we can do it all on one machine (or scale it out, but ? ? it doesn't take too long)? ?2) More importantly, the results are less machine-specific. There are ? ? no paths to dynamic linked libraries or versions thereof. This is roughly how 3.6 was released -- along with a matching source tree for the entire system and good directions to build the multiple pieces. ?The directions today are like:? ? get the right bootstrap archive ? get the source tree ? extract bootstrap ? build it (make) (again, if you look, simple stuff)? ? install it ? ? mkdir /somewhere/bin? ? ? mv cm3 /somewhere/bin ? ? ? export PATH=/somewhere/bin:$PATH ? ? ? ./boot2.py ? ? ? or ? ? ?./boot2.py c to use the C backend ? ? ? ? ? ? ? ?(mklib is only for NT targets)? ?? ?? I'd really soon like there to be fewer C variations though -- just unixc32le, unixc32be, unixc64le, posix64be, ntc32le, ntc64le. And this is almost overkill -- 64le is the vast majority. ? - Jay ---------------------------------------- Date: Tue, 31 May 2016 08:28:32 -0700 From: notifications at github.com To: cm3 at noreply.github.com CC: jay.krell at cornell.edu; comment at noreply.github.com Subject: Re: [modula3/cm3] New release for Mac OS X, please? (#13) Yes, I saw the min and all tar.gz files yesterday. I extracted the "all" tar file and the compiler worked fine. (The tar.gz files are gone from my /var directory now.) I'd be happy to help with a release. What should I do? From jay.krell at cornell.edu Wed Jun 1 11:30:16 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 09:30:16 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? Does/can it retain its incrementality? Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > Date: Tue, 31 May 2016 17:18:46 +0000 > Subject: Re: [M3devel] replace build system?? > > We should probably learn how to get the number of processors and optionally system load and make this more automatic. > Posix should standardize more of this. :( > > - Jay > > > > > > > ---------------------------------------- >> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >> To: dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 10:16:05 -0700 >> From: mika at async.caltech.edu >> Subject: Re: [M3devel] replace build system?? >> >> I mentioned this before a few times on this mailing list.... CM3 is already fairly >> parallel if you turn on the right options. Every back-end invocation can be done >> in parallel. >> >> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >> >> "Daniel Alejandro Benavides D." writes: >> ... >>>A make-based build solution would have other advantages as well like=20 >>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>four times faster. >> ... >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Wed Jun 1 13:17:43 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 11:17:43 +0000 Subject: [M3devel] IR init_int with out of range values? Message-ID: Is this reasonable? m3-libs/m3core/src/float/IEEE-be/LongRealRep.i3 INTERFACE LongRealRep; (* This interface describes the layout of IEEE double precision reals ? ?on big endian machines *) TYPE ? T = RECORD? ? ? sign ? ? ? ? : BITS ?1 FOR [0..1] ? ? ? ? ? ? ? ? ? ? ? ? ?:= 0; ? ? exponent ? ? : BITS 11 FOR [0..16_7FF] ? ? ? ? ? ? ? ? ? ? := 0; ? ? significand0 : BITS 20 FOR [0..16_FFFFF] ? ? ? ? ? ? ? ? ? := 0; ? ? significand1 : BITS 32 FOR [-16_7fffffff-1 .. 16_7fffffff] := 0; ? END; CONST ? NegInf = T { sign := 1, exponent := 16_7FF }; ? PosInf = T { sign := 0, exponent := 16_7FF }; ? Nan ? ?= T { sign := 0, exponent := 16_7FF, significand0 := 1 }; END LongRealRep. begin_init v.1 # constant NegInf # constant PosInf init_int 0 18442240474082181120 Int.64 # constant Nan init_int 8 9218868437227405312 Int.64 My C backend doesn't like 18442240474082181120 Int.64, since 18442240474082181120 does not fit in a signed 64bit integer. Should the frontend be obligated to make it Word.64? I guess I can make it work. jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ edit 1.c jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ cc 1.c 1.c:5:15: warning: integer literal is too large to be represented in a signed integer type, interpreting as unsigned ? ? ? [-Wimplicitly-unsigned-literal] long long a = 18442240474082181120ll; ? ? ? ? ? ? ? ^ 1 warning generated. jair:python jay$ ./a.out 18442240474082181120 fff0000000000000 -4503599627370496 fff0000000000000 Thank you, ?- Jay From rodney_bates at lcwb.coop Thu Jun 2 00:08:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 01 Jun 2016 17:08:34 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: <574F5CE2.8080506@lcwb.coop> On 06/01/2016 04:30 AM, Jay K wrote: > We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. It doesn't, not very well. A deep principle, if you can call it that, of C is that each compilation has no knowledge of any other compilation. Likewise, when linking them together, it's all just binary stuff, addresses, constants, etc. To get a semblance of type-safe compilation, all coders have to correctly follow some variant of a header file/#include idiom and create a matching makefile that correctly reflects the actual interdependencies. There is no checking that these are done correctly, and from direct experience, I can say they are routinely done wrong. All the modular languages have direct linguistic support for type-safe separate compilation. Implementing this correctly while using the C compilation model lies somewhere between very kludgy and impossible. > > Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? It couldn't possibly. Typical front end compilation actual work is similar to or less than executable program startup/shutdown overhead. This effect might be partially lost in the fact that m3cc is a separate executable and it has to start up for every .[im]3 file. But it would still about double the number of program executions and startup overheads. > Does/can it retain its incrementality? I don't think this is possible all. No matter what is in the makefile, Make only understands dependencies among whole source files and must rebuild if a depended-on file has been touched, even no changes to content. Our current build system works on declaration granularity. (I presume this is what you mean by incrementality." I am not aware of any language-independent build infrastructure that keeps track of dependencies on other that source file granularity. > > Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler frontend, and language-specific link-time functions in one executable. I see no benefit and almost certainly losses, in putting these in different executables. > - Jay > > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 17:18:46 +0000 >> Subject: Re: [M3devel] replace build system?? >> >> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >> Posix should standardize more of this. :( >> >> - Jay >> >> >> >> >> >> >> ---------------------------------------- >>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>> To: dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 10:16:05 -0700 >>> From: mika at async.caltech.edu >>> Subject: Re: [M3devel] replace build system?? >>> >>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>> parallel if you turn on the right options. Every back-end invocation can be done >>> in parallel. >>> >>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>> >>> "Daniel Alejandro Benavides D." writes: >>> ... >>>> A make-based build solution would have other advantages as well like=20 >>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>> four times faster. >>> ... >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From microcode at zoho.com Thu Jun 2 08:08:47 2016 From: microcode at zoho.com (microcode at zoho.com) Date: Thu, 2 Jun 2016 06:08:47 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> Message-ID: <20160602060847.GA3051@zoho.com> On Wed, Jun 01, 2016 at 05:08:34PM -0500, Rodney M. Bates wrote: > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. Agreed with everything you wrote not withstanding my total lack of M3 knowledge. Ada is another language (possibly the first?) that specifies a lot of how it needs to be built in the language spec. It deals with the issues you mentioned (and more I think) like checking parameters and types across the system, interface compliance, incremental compilation etc. There is an open source version available so it should be possible to at least get a look how they are accomplishing those things and see if it could be scrounged, adapted etc. for M3. I would think they're very Ada-centric as these kinds of things seem to always turn out to be, but who knows. The Ada compiler is part of gcc, it's called gnat or gcc-ada. The specific top-level piece that is responsible for figuring out all the dependencies is called gnatmake. From jay.krell at cornell.edu Fri Jun 3 09:22:45 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 07:22:45 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: , ,,, ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: I understand the C model very well, but I'm not certain of all the ramifications for Modula-3. I understand C++ has limited checking via name mangling. And that other systems carry names/signatures forward to runtime and check them there. Even dynamically linked C++. And Modula-3. However function names aren't everything -- i.e. they don't include structural hashes of types. It is name-based, not structure-based. The most common mistake in the loose C systems is not doing things wrong per se, but accidentally mixing old and new versions, and the result is undefined/crash. We have the same problem with our "extern" mechanism -- how many times I debugged stat corrupting stack until I put in the extra layer. C++ systems are getting better and check for "ODR violations" at link time. But checking at link time kinda contradicts my question. Goal being to fit into "generic" build systems but C++ only starts winning when the linker knows about it -- when the system is C++ specific. If C++ requires C++ linker/driver, we shouldn't feel too bad requiring a Modula-3 linker/driver. ?> Typical front end compilation actual work is similar to? ?> or less than executable program startup/shutdown overhead.? ?Ideally this is not the case. Ideally process startup is just referencing ?pages already in file system cache. Demand paging, unified file system cache ?and memory manager and all that (all systems NT, Linux, etc. are the same ?these days). But I realize in reality there is alway some amount of other CPU ?to burn at each process start. The ideal is not quite reached. ?> m3cc is a separate executable..has to start up for every .[im]3 file. B ? ?I wonder if we can fix this btw. ?If we can start it just once per library. ?Process create really is fast on typical Posix systems, so people maybe ?don't notice. ? ?There are a few methods: ? ?- alway keep the m3cg ir files ? ?- if any out of date, read them all into one large IR and write out one large assembly file ? ?- I want do that anyway, to achieve a rich/poor man's LTCG/LTO. ? ? ?poor: No need to worry about linker/driver support.? ? ? ?rich: We can afford the memory these days. Though at some point it wouldn't scale.? ? - I want do similar for C backend, as it seems noticable either the compiler startup ? ? or the few #includes I sometimes use (though C++ modules promise some relief..) ? ?? ? ? Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them? ? separately as usual, but within one m3cg invocation.? ?? ?> Does/can it retain its incrementality? ? ?I think the best but not great we could do is make would decide pessmisically ?to run the compiler, but the compiler could figure out that the old/existing ?output is still correct. The world to decide up-to-date is significant but ?perhaps somewhat less than the work to just do all the work. ? ?> ?see no benefit and almost certainly losses, in putting these in different executables. ?? ? Less code to maintain. Easier to redistribute. ? I like the model of distribution of C source. ? ? ?wget source && tar xf && configure && make && make install ? I want something like that. ? I am uncertain of "redistribution" and "active development" must/should be different. ? i.e. We do have something like this, and the two ways of working cooexist. ?? ? ? ?- Jay ---------------------------------------- > Date: Wed, 1 Jun 2016 17:08:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > > > On 06/01/2016 04:30 AM, Jay K wrote: >> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > > It doesn't, not very well. A deep principle, if you can call it that, of C is that > each compilation has no knowledge of any other compilation. Likewise, when linking > them together, it's all just binary stuff, addresses, constants, etc. To get a > semblance of type-safe compilation, all coders have to correctly follow some variant > of a header file/#include idiom and create a matching makefile that correctly reflects > the actual interdependencies. There is no checking that these are done correctly, and > from direct experience, I can say they are routinely done wrong. > > All the modular languages have direct linguistic support for type-safe separate > compilation. Implementing this correctly while using the C compilation model lies > somewhere between very kludgy and impossible. > >> >> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > > It couldn't possibly. Typical front end compilation actual work is similar to > or less than executable program startup/shutdown overhead. This effect > might be partially lost in the fact that m3cc is a separate executable and it > has to start up for every .[im]3 file. But it would still about double the > number of program executions and startup overheads. > >> Does/can it retain its incrementality? > > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. > >> >> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >> > > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > frontend, and language-specific link-time functions in one executable. I see no > benefit and almost certainly losses, in putting these in different executables. > >> - Jay >> >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 17:18:46 +0000 >>> Subject: Re: [M3devel] replace build system?? >>> >>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>> Posix should standardize more of this. :( >>> >>> - Jay >>> >>> >>> >>> >>> >>> >>> ---------------------------------------- >>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>> To: dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>> From: mika at async.caltech.edu >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>> parallel if you turn on the right options. Every back-end invocation can be done >>>> in parallel. >>>> >>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>> >>>> "Daniel Alejandro Benavides D." writes: >>>> ... >>>>> A make-based build solution would have other advantages as well like=20 >>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>> four times faster. >>>> ... >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Fri Jun 3 16:44:12 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:44:12 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575197BC.1030100@lcwb.coop> BTW, 'though the current build system is considerably sophisticated, it does have some bugs, which we should fix. I have the exact setup & symptom for one of them written down somewhere. I think it's when adding a procedure signature to an interface when a procedure body for it was already present in a compiled exporting module, things don't get recompiled right. Also, I occasionally get recompile failures that I don't think are valid, and when I clean the package and recompile that, all is well. I haven't ever tracked down how to reproduce this or noted its exact symptom. We also don't keep track of compile options that affect generated code, so as to recompile when only the options change. E.g., optimization level, debug info presence and format. I did do this for the size of WIDECHAR, since mismatches here would be disastrous. On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>> in parallel. >>>>> >>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>> >>>>> "Daniel Alejandro Benavides D." writes: >>>>> ... >>>>>> A make-based build solution would have other advantages as well like=20 >>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>> four times faster. >>>>> ... >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 3 16:53:16 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:53:16 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575199DC.4060305@lcwb.coop> On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > I like this idea, but we don't want to force serialization where we now do parallel runs of m3cg. OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands seem to scroll by slower than the unit compiles when running cm3, despite the fact that the latter include a main program invocation of m3cc. Surely it's not merely smaller units? And we are also running separately too. > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > We could just wrap what we have in makefile that, almost trivially invokes cm3 with a few odd options. Or is that cheating? > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>> in parallel. >>>>> >>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>> >>>>> "Daniel Alejandro Benavides D." writes: >>>>> ... >>>>>> A make-based build solution would have other advantages as well like=20 >>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>> four times faster. >>>>> ... >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Fri Jun 3 19:18:07 2016 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 3 Jun 2016 19:18:07 +0200 Subject: [M3devel] replace build system?? In-Reply-To: <575199DC.4060305@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> Message-ID: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs >> and process them >> separately as usual, but within one m3cg invocation. >> > > I like this idea, but we don't want to force serialization where we now > do parallel > runs of m3cg. > > OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile > commands > seem to scroll by slower than the unit compiles when running cm3, > despite the fact > that the latter include a main program invocation of m3cc. Surely it's > not merely > smaller units? And we are also running separately too. > Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' that the backend would generate code in parallel? - This would not be right what I want because actually obtaining deterministic build results is of high value for security reasons. It can be necessary to verify whether a given compiler toolkit / build environment must have been infected at the stage of build time with hindsight. I know deterministic builds are quite hard to realize even for some C/C++ applications; nonetheless I had never been thinking about Modula-3 or CM3 in specific. At least it was a well known problem with Python that builds were non-deterministic due to parallel code generation. However that should not happen if just different source files are translated at the same time. Moreover that could also parallelize the activity of the front (and middle)-end. From rodney_bates at lcwb.coop Fri Jun 3 20:31:40 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 13:31:40 -0500 Subject: [M3devel] replace build system?? In-Reply-To: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> Message-ID: <5751CD0C.1070903@lcwb.coop> On 06/03/2016 12:18 PM, Elmar Stellnberger wrote: > >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs >>> and process them >>> separately as usual, but within one m3cg invocation. >>> >> >> I like this idea, but we don't want to force serialization where we now >> do parallel >> runs of m3cg. >> >> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile >> commands >> seem to scroll by slower than the unit compiles when running cm3, >> despite the fact >> that the latter include a main program invocation of m3cc. Surely it's >> not merely >> smaller units? And we are also running separately too. >> > > Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' that the backend would generate code in parallel? > - This would not be right what I want because actually obtaining deterministic build results is of high value for security reasons. It can be necessary to verify whether a given compiler toolkit / build environment must have been infected at the stage of build time with hindsight. > I know deterministic builds are quite hard to realize even for some C/C++ applications; nonetheless I had never been thinking about Modula-3 or CM3 in specific. At least it was a well known problem with Python that builds were non-deterministic due to parallel code generation. > However that should not happen if just different source files are translated at the same time. Moreover that could also parallelize the activity of the front (and middle)-end. > ___ I don't believe the builder bugs I was talking about have anything to do with parallel compilation. In fact, they happen with no parallelism. Mika can weigh in, but I am sure the parallelism is carefully synchronized so the resulting compiled code is entirely deterministic, even when the cpu usage getting there is not. ____________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Jun 3 21:48:25 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 19:48:25 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <575199DC.4060305@lcwb.coop> References: , , ,,, , ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , ,,<20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> ,<575199DC.4060305@lcwb.coop> Message-ID: > like this idea, but we don't want to force serialization where we now do parallel > runs of m3cg. Good point. We could use pthreads in there though, maybe. Except I'm not sure we own enough of the code..and..er...oh..nevermind. I believe gcc is replete with globals. As is the frontend. We could still partition the work. If we have say 40 files to compile, we could run 4 m3cg's in parallel that each compile 10 files... ?> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands ?> seem to scroll by slower than the unit compiles when running cm3, despite the fact I believe the C compiler is effectively slow. The C compiler is actually very fast, but it has to do the same or almost the same work very many times. Compiling a thousand or so lines is very quick, but #include windows.h or stdio.h or vector and you have really a ton of code, and it redoes the work for every source files. Precompiled headers can help. Though at least with the Microsoft compiler, they change then language very slightly. And you have to work through each compiler's command line options (maybe cmake/automake??) C++ modules will be the salvation, maybe soon. ?- Jay ---------------------------------------- > Date: Fri, 3 Jun 2016 09:53:16 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > > > On 06/03/2016 02:22 AM, Jay K wrote: >> I understand the C model very well, but I'm not certain of all >> the ramifications for Modula-3. >> >> >> I understand C++ has limited checking via name mangling. >> >> >> And that other systems carry names/signatures forward to runtime >> and check them there. Even dynamically linked C++. And Modula-3. >> >> >> However function names aren't everything -- i.e. they don't include >> structural hashes of types. It is name-based, not structure-based. >> >> >> The most common mistake in the loose C systems is not doing things >> wrong per se, but accidentally mixing old and new versions, >> and the result is undefined/crash. >> >> >> We have the same problem with our "extern" mechanism -- how many times >> I debugged stat corrupting stack until I put in the extra layer. >> >> >> C++ systems are getting better and check for "ODR violations" at link time. >> But checking at link time kinda contradicts my question. >> Goal being to fit into "generic" build systems but C++ only starts >> winning when the linker knows about it -- when the system is C++ specific. >> If C++ requires C++ linker/driver, we shouldn't feel too bad >> requiring a Modula-3 linker/driver. >> >> >>> Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. >> >> >> Ideally this is not the case. Ideally process startup is just referencing >> pages already in file system cache. Demand paging, unified file system cache >> and memory manager and all that (all systems NT, Linux, etc. are the same >> these days). But I realize in reality there is alway some amount of other CPU >> to burn at each process start. The ideal is not quite reached. >> >>> m3cc is a separate executable..has to start up for every .[im]3 file. B >> >> I wonder if we can fix this btw. >> If we can start it just once per library. >> Process create really is fast on typical Posix systems, so people maybe >> don't notice. >> >> There are a few methods: >> - alway keep the m3cg ir files >> - if any out of date, read them all into one large IR and write out one large assembly file >> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >> poor: No need to worry about linker/driver support. >> rich: We can afford the memory these days. Though at some point it wouldn't scale. >> - I want do similar for C backend, as it seems noticable either the compiler startup >> or the few #includes I sometimes use (though C++ modules promise some relief..) >> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >> separately as usual, but within one m3cg invocation. >> > > I like this idea, but we don't want to force serialization where we now do parallel > runs of m3cg. > > OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands > seem to scroll by slower than the unit compiles when running cm3, despite the fact > that the latter include a main program invocation of m3cc. Surely it's not merely > smaller units? And we are also running separately too. > >> >>> Does/can it retain its incrementality? >> >> I think the best but not great we could do is make would decide pessmisically >> to run the compiler, but the compiler could figure out that the old/existing >> output is still correct. The world to decide up-to-date is significant but >> perhaps somewhat less than the work to just do all the work. >> >>> see no benefit and almost certainly losses, in putting these in different executables. >> >> Less code to maintain. Easier to redistribute. >> I like the model of distribution of C source. >> wget source && tar xf && configure && make && make install >> > > We could just wrap what we have in makefile that, almost trivially invokes cm3 with > a few odd options. Or is that cheating? > >> I want something like that. >> I am uncertain of "redistribution" and "active development" must/should be different. >> i.e. We do have something like this, and the two ways of working cooexist. >> >> >> - Jay >> >> >> >> >> >> ---------------------------------------- >>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] replace build system?? >>> >>> >>> >>> On 06/01/2016 04:30 AM, Jay K wrote: >>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>> >>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>> each compilation has no knowledge of any other compilation. Likewise, when linking >>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>> semblance of type-safe compilation, all coders have to correctly follow some variant >>> of a header file/#include idiom and create a matching makefile that correctly reflects >>> the actual interdependencies. There is no checking that these are done correctly, and >>> from direct experience, I can say they are routinely done wrong. >>> >>> All the modular languages have direct linguistic support for type-safe separate >>> compilation. Implementing this correctly while using the C compilation model lies >>> somewhere between very kludgy and impossible. >>> >>>> >>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>> >>> It couldn't possibly. Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. This effect >>> might be partially lost in the fact that m3cc is a separate executable and it >>> has to start up for every .[im]3 file. But it would still about double the >>> number of program executions and startup overheads. >>> >>>> Does/can it retain its incrementality? >>> >>> I don't think this is possible all. No matter what is in the makefile, Make >>> only understands dependencies among whole source files and must rebuild if >>> a depended-on file has been touched, even no changes to content. Our current >>> build system works on declaration granularity. (I presume this is what you >>> mean by incrementality." I am not aware of any language-independent build >>> infrastructure that keeps track of dependencies on other that source file >>> granularity. >>> >>>> >>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>> >>> >>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>> frontend, and language-specific link-time functions in one executable. I see no >>> benefit and almost certainly losses, in putting these in different executables. >>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>> Posix should standardize more of this. :( >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>> To: dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>> in parallel. >>>>>> >>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>> >>>>>> "Daniel Alejandro Benavides D." writes: >>>>>> ... >>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>> four times faster. >>>>>> ... >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Fri Jun 3 21:51:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 19:51:07 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <575197BC.1030100@lcwb.coop> References: , , ,,, , ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , ,,<20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> ,<575197BC.1030100@lcwb.coop> Message-ID: Moving a module from one package to another is a problem I have hit. I moved some path manipulation module a few years ago and incremental builds failed. Not tracking optimization can be seen as a feature, or a bug. ? I might just want to not optimize a few files that I'm debugging. WIDECHAR certainly. Choice of C backend is another. :) Nested functions have a different interface. But appending "c" to BUILD_DIR is maybe a good workaround. Maybe. I wish I could be link compatible but I don't see how. gcc puts the static link often in a special register dedicated to the purpose -- ABIs actually designed and optimized with this in mind! I put it as the last parameter. ?- Jay ---------------------------------------- > Date: Fri, 3 Jun 2016 09:44:12 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > BTW, 'though the current build system is considerably sophisticated, > it does have some bugs, which we should fix. I have the exact setup & > symptom for one of them written down somewhere. I think it's when adding > a procedure signature to an interface when a procedure body for it was > already present in a compiled exporting module, things don't get recompiled > right. > > Also, I occasionally get recompile failures that I don't think are valid, > and when I clean the package and recompile that, all is well. I haven't > ever tracked down how to reproduce this or noted its exact symptom. > > We also don't keep track of compile options that affect generated code, > so as to recompile when only the options change. E.g., optimization level, > debug info presence and format. I did do this for the size of WIDECHAR, > since mismatches here would be disastrous. > > On 06/03/2016 02:22 AM, Jay K wrote: >> I understand the C model very well, but I'm not certain of all >> the ramifications for Modula-3. >> >> >> I understand C++ has limited checking via name mangling. >> >> >> And that other systems carry names/signatures forward to runtime >> and check them there. Even dynamically linked C++. And Modula-3. >> >> >> However function names aren't everything -- i.e. they don't include >> structural hashes of types. It is name-based, not structure-based. >> >> >> The most common mistake in the loose C systems is not doing things >> wrong per se, but accidentally mixing old and new versions, >> and the result is undefined/crash. >> >> >> We have the same problem with our "extern" mechanism -- how many times >> I debugged stat corrupting stack until I put in the extra layer. >> >> >> C++ systems are getting better and check for "ODR violations" at link time. >> But checking at link time kinda contradicts my question. >> Goal being to fit into "generic" build systems but C++ only starts >> winning when the linker knows about it -- when the system is C++ specific. >> If C++ requires C++ linker/driver, we shouldn't feel too bad >> requiring a Modula-3 linker/driver. >> >> >>> Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. >> >> >> Ideally this is not the case. Ideally process startup is just referencing >> pages already in file system cache. Demand paging, unified file system cache >> and memory manager and all that (all systems NT, Linux, etc. are the same >> these days). But I realize in reality there is alway some amount of other CPU >> to burn at each process start. The ideal is not quite reached. >> >>> m3cc is a separate executable..has to start up for every .[im]3 file. B >> >> I wonder if we can fix this btw. >> If we can start it just once per library. >> Process create really is fast on typical Posix systems, so people maybe >> don't notice. >> >> There are a few methods: >> - alway keep the m3cg ir files >> - if any out of date, read them all into one large IR and write out one large assembly file >> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >> poor: No need to worry about linker/driver support. >> rich: We can afford the memory these days. Though at some point it wouldn't scale. >> - I want do similar for C backend, as it seems noticable either the compiler startup >> or the few #includes I sometimes use (though C++ modules promise some relief..) >> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >> separately as usual, but within one m3cg invocation. >> >> >>> Does/can it retain its incrementality? >> >> I think the best but not great we could do is make would decide pessmisically >> to run the compiler, but the compiler could figure out that the old/existing >> output is still correct. The world to decide up-to-date is significant but >> perhaps somewhat less than the work to just do all the work. >> >>> see no benefit and almost certainly losses, in putting these in different executables. >> >> Less code to maintain. Easier to redistribute. >> I like the model of distribution of C source. >> wget source && tar xf && configure && make && make install >> >> I want something like that. >> I am uncertain of "redistribution" and "active development" must/should be different. >> i.e. We do have something like this, and the two ways of working cooexist. >> >> >> - Jay >> >> >> >> >> >> ---------------------------------------- >>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] replace build system?? >>> >>> >>> >>> On 06/01/2016 04:30 AM, Jay K wrote: >>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>> >>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>> each compilation has no knowledge of any other compilation. Likewise, when linking >>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>> semblance of type-safe compilation, all coders have to correctly follow some variant >>> of a header file/#include idiom and create a matching makefile that correctly reflects >>> the actual interdependencies. There is no checking that these are done correctly, and >>> from direct experience, I can say they are routinely done wrong. >>> >>> All the modular languages have direct linguistic support for type-safe separate >>> compilation. Implementing this correctly while using the C compilation model lies >>> somewhere between very kludgy and impossible. >>> >>>> >>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>> >>> It couldn't possibly. Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. This effect >>> might be partially lost in the fact that m3cc is a separate executable and it >>> has to start up for every .[im]3 file. But it would still about double the >>> number of program executions and startup overheads. >>> >>>> Does/can it retain its incrementality? >>> >>> I don't think this is possible all. No matter what is in the makefile, Make >>> only understands dependencies among whole source files and must rebuild if >>> a depended-on file has been touched, even no changes to content. Our current >>> build system works on declaration granularity. (I presume this is what you >>> mean by incrementality." I am not aware of any language-independent build >>> infrastructure that keeps track of dependencies on other that source file >>> granularity. >>> >>>> >>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>> >>> >>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>> frontend, and language-specific link-time functions in one executable. I see no >>> benefit and almost certainly losses, in putting these in different executables. >>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>> Posix should standardize more of this. :( >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>> To: dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>> in parallel. >>>>>> >>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>> >>>>>> "Daniel Alejandro Benavides D." writes: >>>>>> ... >>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>> four times faster. >>>>>> ... >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Fri Jun 3 23:08:23 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 16:08:23 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> , <575197BC.1030100@lcwb.coop> Message-ID: <5751F1C7.5050806@lcwb.coop> On 06/03/2016 02:51 PM, Jay K wrote: > Moving a module from one package to another is a problem I have hit. > I moved some path manipulation module a few years ago and incremental builds failed. > Yes, intra-package dependencies are dealt with very poorly. There is no automatic rebuilding at all (do we really want it, given that a library may have multiple client packages?) Detection of problems doesn't happen until link time, and then with virtually useless error messages that contain only hash codes of language constructs. I'm not all confident of how complete the detection is either. I fixed just one such message to be slightly more helpful a couple of years ago, but several IR operators need additional operands to do anything decent. > Not tracking optimization can be seen as a feature, or a bug. > I might just want to not optimize a few files that I'm debugging. > > WIDECHAR certainly. > > Choice of C backend is another. :) > Nested functions have a different interface. > But appending "c" to BUILD_DIR is maybe a good workaround. Maybe. > I wish I could be link compatible but I don't see how. > gcc puts the static link often in a special register dedicated to the purpose -- ABIs actually designed and optimized with this in mind! > I put it as the last parameter. > > - Jay > > > ---------------------------------------- >> Date: Fri, 3 Jun 2016 09:44:12 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> BTW, 'though the current build system is considerably sophisticated, >> it does have some bugs, which we should fix. I have the exact setup & >> symptom for one of them written down somewhere. I think it's when adding >> a procedure signature to an interface when a procedure body for it was >> already present in a compiled exporting module, things don't get recompiled >> right. >> >> Also, I occasionally get recompile failures that I don't think are valid, >> and when I clean the package and recompile that, all is well. I haven't >> ever tracked down how to reproduce this or noted its exact symptom. >> >> We also don't keep track of compile options that affect generated code, >> so as to recompile when only the options change. E.g., optimization level, >> debug info presence and format. I did do this for the size of WIDECHAR, >> since mismatches here would be disastrous. >> >> On 06/03/2016 02:22 AM, Jay K wrote: >>> I understand the C model very well, but I'm not certain of all >>> the ramifications for Modula-3. >>> >>> >>> I understand C++ has limited checking via name mangling. >>> >>> >>> And that other systems carry names/signatures forward to runtime >>> and check them there. Even dynamically linked C++. And Modula-3. >>> >>> >>> However function names aren't everything -- i.e. they don't include >>> structural hashes of types. It is name-based, not structure-based. >>> >>> >>> The most common mistake in the loose C systems is not doing things >>> wrong per se, but accidentally mixing old and new versions, >>> and the result is undefined/crash. >>> >>> >>> We have the same problem with our "extern" mechanism -- how many times >>> I debugged stat corrupting stack until I put in the extra layer. >>> >>> >>> C++ systems are getting better and check for "ODR violations" at link time. >>> But checking at link time kinda contradicts my question. >>> Goal being to fit into "generic" build systems but C++ only starts >>> winning when the linker knows about it -- when the system is C++ specific. >>> If C++ requires C++ linker/driver, we shouldn't feel too bad >>> requiring a Modula-3 linker/driver. >>> >>> >>>> Typical front end compilation actual work is similar to >>>> or less than executable program startup/shutdown overhead. >>> >>> >>> Ideally this is not the case. Ideally process startup is just referencing >>> pages already in file system cache. Demand paging, unified file system cache >>> and memory manager and all that (all systems NT, Linux, etc. are the same >>> these days). But I realize in reality there is alway some amount of other CPU >>> to burn at each process start. The ideal is not quite reached. >>> >>>> m3cc is a separate executable..has to start up for every .[im]3 file. B >>> >>> I wonder if we can fix this btw. >>> If we can start it just once per library. >>> Process create really is fast on typical Posix systems, so people maybe >>> don't notice. >>> >>> There are a few methods: >>> - alway keep the m3cg ir files >>> - if any out of date, read them all into one large IR and write out one large assembly file >>> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >>> poor: No need to worry about linker/driver support. >>> rich: We can afford the memory these days. Though at some point it wouldn't scale. >>> - I want do similar for C backend, as it seems noticable either the compiler startup >>> or the few #includes I sometimes use (though C++ modules promise some relief..) >>> >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >>> separately as usual, but within one m3cg invocation. >>> >>> >>>> Does/can it retain its incrementality? >>> >>> I think the best but not great we could do is make would decide pessmisically >>> to run the compiler, but the compiler could figure out that the old/existing >>> output is still correct. The world to decide up-to-date is significant but >>> perhaps somewhat less than the work to just do all the work. >>> >>>> see no benefit and almost certainly losses, in putting these in different executables. >>> >>> Less code to maintain. Easier to redistribute. >>> I like the model of distribution of C source. >>> wget source && tar xf && configure && make && make install >>> >>> I want something like that. >>> I am uncertain of "redistribution" and "active development" must/should be different. >>> i.e. We do have something like this, and the two ways of working cooexist. >>> >>> >>> - Jay >>> >>> >>> >>> >>> >>> ---------------------------------------- >>>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> >>>> >>>> On 06/01/2016 04:30 AM, Jay K wrote: >>>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>>> >>>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>>> each compilation has no knowledge of any other compilation. Likewise, when linking >>>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>>> semblance of type-safe compilation, all coders have to correctly follow some variant >>>> of a header file/#include idiom and create a matching makefile that correctly reflects >>>> the actual interdependencies. There is no checking that these are done correctly, and >>>> from direct experience, I can say they are routinely done wrong. >>>> >>>> All the modular languages have direct linguistic support for type-safe separate >>>> compilation. Implementing this correctly while using the C compilation model lies >>>> somewhere between very kludgy and impossible. >>>> >>>>> >>>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>>> >>>> It couldn't possibly. Typical front end compilation actual work is similar to >>>> or less than executable program startup/shutdown overhead. This effect >>>> might be partially lost in the fact that m3cc is a separate executable and it >>>> has to start up for every .[im]3 file. But it would still about double the >>>> number of program executions and startup overheads. >>>> >>>>> Does/can it retain its incrementality? >>>> >>>> I don't think this is possible all. No matter what is in the makefile, Make >>>> only understands dependencies among whole source files and must rebuild if >>>> a depended-on file has been touched, even no changes to content. Our current >>>> build system works on declaration granularity. (I presume this is what you >>>> mean by incrementality." I am not aware of any language-independent build >>>> infrastructure that keeps track of dependencies on other that source file >>>> granularity. >>>> >>>>> >>>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>>> >>>> >>>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>>> frontend, and language-specific link-time functions in one executable. I see no >>>> benefit and almost certainly losses, in putting these in different executables. >>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> From: jay.krell at cornell.edu >>>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>>> Posix should standardize more of this. :( >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------- >>>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>>> To: dabenavidesd at yahoo.es >>>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>>> From: mika at async.caltech.edu >>>>>>> Subject: Re: [M3devel] replace build system?? >>>>>>> >>>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>>> in parallel. >>>>>>> >>>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>>> >>>>>>> "Daniel Alejandro Benavides D." writes: >>>>>>> ... >>>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>>> four times faster. >>>>>>> ... >>>>>>> _______________________________________________ >>>>>>> M3devel mailing list >>>>>>> M3devel at elegosoft.com >>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From mika at async.caltech.edu Sat Jun 4 00:16:34 2016 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jun 2016 15:16:34 -0700 Subject: [M3devel] replace build system?? In-Reply-To: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> Message-ID: <20160603221635.019231A206A@async.async.caltech.edu> If you set M3_PARALLEL_BACK, the different individual invocations of m3cgc (or whatever the back end is called) are done in parallel, as separate Unix processes. Nothing tricky, not nondeterministic either. Nothing depends on these jobs except the linker. Mika Elmar Stellnberger writes: > >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs >>> and process them >>> separately as usual, but within one m3cg invocation. >>> >> >> I like this idea, but we don't want to force serialization where we now >> do parallel >> runs of m3cg. >> >> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile >> commands >> seem to scroll by slower than the unit compiles when running cm3, >> despite the fact >> that the latter include a main program invocation of m3cc. Surely it's >> not merely >> smaller units? And we are also running separately too. >> > >Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' >that the backend would generate code in parallel? > - This would not be right what I want because actually obtaining >deterministic build results is of high value for security reasons. It >can be necessary to verify whether a given compiler toolkit / build >environment must have been infected at the stage of build time with >hindsight. > I know deterministic builds are quite hard to realize even for some >C/C++ applications; nonetheless I had never been thinking about Modula-3 >or CM3 in specific. At least it was a well known problem with Python >that builds were non-deterministic due to parallel code generation. > However that should not happen if just different source files are >translated at the same time. Moreover that could also parallelize the >activity of the front (and middle)-end. >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Sat Jun 4 02:23:37 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 3 Jun 2016 20:23:37 -0400 Subject: [M3devel] replace build system?? In-Reply-To: <575197BC.1030100@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575197BC.1030100@lcwb.coop> Message-ID: <20160604002337.GA30109@topoi.pooq.com> On Fri, Jun 03, 2016 at 09:44:12AM -0500, Rodney M. Bates wrote: > BTW, 'though the current build system is considerably sophisticated, > it does have some bugs, which we should fix. I have the exact setup & > symptom for one of them written down somewhere. I think it's when adding > a procedure signature to an interface when a procedure body for it was > already present in a compiled exporting module, things don't get recompiled > right. > > Also, I occasionally get recompile failures that I don't think are valid, > and when I clean the package and recompile that, all is well. I haven't > ever tracked down how to reproduce this or noted its exact symptom. I get them too, and I too have not managed to reproduce this. I have the impression that sonetimes something that should have been recompiled isn't, but I don't really know. Deleting the LINIXLIBC directory usually makes everything work. -- hendrik > > We also don't keep track of compile options that affect generated code, > so as to recompile when only the options change. E.g., optimization level, > debug info presence and format. I did do this for the size of WIDECHAR, > since mismatches here would be disastrous. > > On 06/03/2016 02:22 AM, Jay K wrote: > >I understand the C model very well, but I'm not certain of all > >the ramifications for Modula-3. > > > > > >I understand C++ has limited checking via name mangling. > > > > > >And that other systems carry names/signatures forward to runtime > >and check them there. Even dynamically linked C++. And Modula-3. > > > > > >However function names aren't everything -- i.e. they don't include > >structural hashes of types. It is name-based, not structure-based. > > > > > >The most common mistake in the loose C systems is not doing things > >wrong per se, but accidentally mixing old and new versions, > >and the result is undefined/crash. > > > > > >We have the same problem with our "extern" mechanism -- how many times > >I debugged stat corrupting stack until I put in the extra layer. > > > > > >C++ systems are getting better and check for "ODR violations" at link time. > >But checking at link time kinda contradicts my question. > >Goal being to fit into "generic" build systems but C++ only starts > >winning when the linker knows about it -- when the system is C++ specific. > >If C++ requires C++ linker/driver, we shouldn't feel too bad > >requiring a Modula-3 linker/driver. > > > > > > > Typical front end compilation actual work is similar to > > > or less than executable program startup/shutdown overhead. > > > > > > Ideally this is not the case. Ideally process startup is just referencing > > pages already in file system cache. Demand paging, unified file system cache > > and memory manager and all that (all systems NT, Linux, etc. are the same > > these days). But I realize in reality there is alway some amount of other CPU > > to burn at each process start. The ideal is not quite reached. > > > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > > > I wonder if we can fix this btw. > > If we can start it just once per library. > > Process create really is fast on typical Posix systems, so people maybe > > don't notice. > > > > There are a few methods: > > - alway keep the m3cg ir files > > - if any out of date, read them all into one large IR and write out one large assembly file > > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > > poor: No need to worry about linker/driver support. > > rich: We can afford the memory these days. Though at some point it wouldn't scale. > > - I want do similar for C backend, as it seems noticable either the compiler startup > > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > > separately as usual, but within one m3cg invocation. > > > > > > > Does/can it retain its incrementality? > > > > I think the best but not great we could do is make would decide pessmisically > > to run the compiler, but the compiler could figure out that the old/existing > > output is still correct. The world to decide up-to-date is significant but > > perhaps somewhat less than the work to just do all the work. > > > > > see no benefit and almost certainly losses, in putting these in different executables. > > > > Less code to maintain. Easier to redistribute. > > I like the model of distribution of C source. > > wget source && tar xf && configure && make && make install > > > > I want something like that. > > I am uncertain of "redistribution" and "active development" must/should be different. > > i.e. We do have something like this, and the two ways of working cooexist. > > > > > > - Jay > > > > > > > > > > > >---------------------------------------- > >>Date: Wed, 1 Jun 2016 17:08:34 -0500 > >>From: rodney_bates at lcwb.coop > >>To: m3devel at elegosoft.com > >>Subject: Re: [M3devel] replace build system?? > >> > >> > >> > >>On 06/01/2016 04:30 AM, Jay K wrote: > >>>We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > >> > >>It doesn't, not very well. A deep principle, if you can call it that, of C is that > >>each compilation has no knowledge of any other compilation. Likewise, when linking > >>them together, it's all just binary stuff, addresses, constants, etc. To get a > >>semblance of type-safe compilation, all coders have to correctly follow some variant > >>of a header file/#include idiom and create a matching makefile that correctly reflects > >>the actual interdependencies. There is no checking that these are done correctly, and > >>from direct experience, I can say they are routinely done wrong. > >> > >>All the modular languages have direct linguistic support for type-safe separate > >>compilation. Implementing this correctly while using the C compilation model lies > >>somewhere between very kludgy and impossible. > >> > >>> > >>>Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > >> > >>It couldn't possibly. Typical front end compilation actual work is similar to > >>or less than executable program startup/shutdown overhead. This effect > >>might be partially lost in the fact that m3cc is a separate executable and it > >>has to start up for every .[im]3 file. But it would still about double the > >>number of program executions and startup overheads. > >> > >>>Does/can it retain its incrementality? > >> > >>I don't think this is possible all. No matter what is in the makefile, Make > >>only understands dependencies among whole source files and must rebuild if > >>a depended-on file has been touched, even no changes to content. Our current > >>build system works on declaration granularity. (I presume this is what you > >>mean by incrementality." I am not aware of any language-independent build > >>infrastructure that keeps track of dependencies on other that source file > >>granularity. > >> > >>> > >>>Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > >>> > >> > >>In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > >>frontend, and language-specific link-time functions in one executable. I see no > >>benefit and almost certainly losses, in putting these in different executables. > >> > >>>- Jay > >>> > >>> > >>> > >>> > >>>---------------------------------------- > >>>>From: jay.krell at cornell.edu > >>>>To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > >>>>Date: Tue, 31 May 2016 17:18:46 +0000 > >>>>Subject: Re: [M3devel] replace build system?? > >>>> > >>>>We should probably learn how to get the number of processors and optionally system load and make this more automatic. > >>>>Posix should standardize more of this. :( > >>>> > >>>>- Jay > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>---------------------------------------- > >>>>>To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com > >>>>>To: dabenavidesd at yahoo.es > >>>>>Date: Tue, 31 May 2016 10:16:05 -0700 > >>>>>From: mika at async.caltech.edu > >>>>>Subject: Re: [M3devel] replace build system?? > >>>>> > >>>>>I mentioned this before a few times on this mailing list.... CM3 is already fairly > >>>>>parallel if you turn on the right options. Every back-end invocation can be done > >>>>>in parallel. > >>>>> > >>>>>Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... > >>>>> > >>>>>"Daniel Alejandro Benavides D." writes: > >>>>>... > >>>>>>A make-based build solution would have other advantages as well like=20 > >>>>>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 > >>>>>>four times faster. > >>>>>... > >>>>>_______________________________________________ > >>>>>M3devel mailing list > >>>>>M3devel at elegosoft.com > >>>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>>_______________________________________________ > >>>>M3devel mailing list > >>>>M3devel at elegosoft.com > >>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>>_______________________________________________ > >>>M3devel mailing list > >>>M3devel at elegosoft.com > >>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >> > >>-- > >>Rodney Bates > >>rodney.m.bates at acm.org > >>_______________________________________________ > >>M3devel mailing list > >>M3devel at elegosoft.com > >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Sat Jun 4 03:08:23 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 4 Jun 2016 01:08:23 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <20160604002337.GA30109@topoi.pooq.com> References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, <20160531171605.13AA61A2068@async.async.caltech.edu>, , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com> Message-ID: Aside: > LINIXLIBC Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD SPARC32_SOLARIS (which internally lets you chose C compiler) ? They have all been present and working for years. These other names are a warty legacy. The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) I wouldn't mind lowercase or omitting the underscore or using a dash though -- possibly short forms recognized by "config.guess". - Jay > Date: Fri, 3 Jun 2016 20:23:37 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Fri, Jun 03, 2016 at 09:44:12AM -0500, Rodney M. Bates wrote: > > BTW, 'though the current build system is considerably sophisticated, > > it does have some bugs, which we should fix. I have the exact setup & > > symptom for one of them written down somewhere. I think it's when adding > > a procedure signature to an interface when a procedure body for it was > > already present in a compiled exporting module, things don't get recompiled > > right. > > > > Also, I occasionally get recompile failures that I don't think are valid, > > and when I clean the package and recompile that, all is well. I haven't > > ever tracked down how to reproduce this or noted its exact symptom. > > I get them too, and I too have not managed to reproduce this. I have > the impression that sonetimes something that should have been > recompiled isn't, but I don't really know. Deleting the LINIXLIBC > directory usually makes everything work. > > -- hendrik > > > > > We also don't keep track of compile options that affect generated code, > > so as to recompile when only the options change. E.g., optimization level, > > debug info presence and format. I did do this for the size of WIDECHAR, > > since mismatches here would be disastrous. > > > > On 06/03/2016 02:22 AM, Jay K wrote: > > >I understand the C model very well, but I'm not certain of all > > >the ramifications for Modula-3. > > > > > > > > >I understand C++ has limited checking via name mangling. > > > > > > > > >And that other systems carry names/signatures forward to runtime > > >and check them there. Even dynamically linked C++. And Modula-3. > > > > > > > > >However function names aren't everything -- i.e. they don't include > > >structural hashes of types. It is name-based, not structure-based. > > > > > > > > >The most common mistake in the loose C systems is not doing things > > >wrong per se, but accidentally mixing old and new versions, > > >and the result is undefined/crash. > > > > > > > > >We have the same problem with our "extern" mechanism -- how many times > > >I debugged stat corrupting stack until I put in the extra layer. > > > > > > > > >C++ systems are getting better and check for "ODR violations" at link time. > > >But checking at link time kinda contradicts my question. > > >Goal being to fit into "generic" build systems but C++ only starts > > >winning when the linker knows about it -- when the system is C++ specific. > > >If C++ requires C++ linker/driver, we shouldn't feel too bad > > >requiring a Modula-3 linker/driver. > > > > > > > > > > Typical front end compilation actual work is similar to > > > > or less than executable program startup/shutdown overhead. > > > > > > > > > Ideally this is not the case. Ideally process startup is just referencing > > > pages already in file system cache. Demand paging, unified file system cache > > > and memory manager and all that (all systems NT, Linux, etc. are the same > > > these days). But I realize in reality there is alway some amount of other CPU > > > to burn at each process start. The ideal is not quite reached. > > > > > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > > > > > I wonder if we can fix this btw. > > > If we can start it just once per library. > > > Process create really is fast on typical Posix systems, so people maybe > > > don't notice. > > > > > > There are a few methods: > > > - alway keep the m3cg ir files > > > - if any out of date, read them all into one large IR and write out one large assembly file > > > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > > > poor: No need to worry about linker/driver support. > > > rich: We can afford the memory these days. Though at some point it wouldn't scale. > > > - I want do similar for C backend, as it seems noticable either the compiler startup > > > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > > > > > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > > > separately as usual, but within one m3cg invocation. > > > > > > > > > > Does/can it retain its incrementality? > > > > > > I think the best but not great we could do is make would decide pessmisically > > > to run the compiler, but the compiler could figure out that the old/existing > > > output is still correct. The world to decide up-to-date is significant but > > > perhaps somewhat less than the work to just do all the work. > > > > > > > see no benefit and almost certainly losses, in putting these in different executables. > > > > > > Less code to maintain. Easier to redistribute. > > > I like the model of distribution of C source. > > > wget source && tar xf && configure && make && make install > > > > > > I want something like that. > > > I am uncertain of "redistribution" and "active development" must/should be different. > > > i.e. We do have something like this, and the two ways of working cooexist. > > > > > > > > > - Jay > > > > > > > > > > > > > > > > > >---------------------------------------- > > >>Date: Wed, 1 Jun 2016 17:08:34 -0500 > > >>From: rodney_bates at lcwb.coop > > >>To: m3devel at elegosoft.com > > >>Subject: Re: [M3devel] replace build system?? > > >> > > >> > > >> > > >>On 06/01/2016 04:30 AM, Jay K wrote: > > >>>We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > > >> > > >>It doesn't, not very well. A deep principle, if you can call it that, of C is that > > >>each compilation has no knowledge of any other compilation. Likewise, when linking > > >>them together, it's all just binary stuff, addresses, constants, etc. To get a > > >>semblance of type-safe compilation, all coders have to correctly follow some variant > > >>of a header file/#include idiom and create a matching makefile that correctly reflects > > >>the actual interdependencies. There is no checking that these are done correctly, and > > >>from direct experience, I can say they are routinely done wrong. > > >> > > >>All the modular languages have direct linguistic support for type-safe separate > > >>compilation. Implementing this correctly while using the C compilation model lies > > >>somewhere between very kludgy and impossible. > > >> > > >>> > > >>>Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > > >> > > >>It couldn't possibly. Typical front end compilation actual work is similar to > > >>or less than executable program startup/shutdown overhead. This effect > > >>might be partially lost in the fact that m3cc is a separate executable and it > > >>has to start up for every .[im]3 file. But it would still about double the > > >>number of program executions and startup overheads. > > >> > > >>>Does/can it retain its incrementality? > > >> > > >>I don't think this is possible all. No matter what is in the makefile, Make > > >>only understands dependencies among whole source files and must rebuild if > > >>a depended-on file has been touched, even no changes to content. Our current > > >>build system works on declaration granularity. (I presume this is what you > > >>mean by incrementality." I am not aware of any language-independent build > > >>infrastructure that keeps track of dependencies on other that source file > > >>granularity. > > >> > > >>> > > >>>Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > > >>> > > >> > > >>In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > > >>frontend, and language-specific link-time functions in one executable. I see no > > >>benefit and almost certainly losses, in putting these in different executables. > > >> > > >>>- Jay > > >>> > > >>> > > >>> > > >>> > > >>>---------------------------------------- > > >>>>From: jay.krell at cornell.edu > > >>>>To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > > >>>>Date: Tue, 31 May 2016 17:18:46 +0000 > > >>>>Subject: Re: [M3devel] replace build system?? > > >>>> > > >>>>We should probably learn how to get the number of processors and optionally system load and make this more automatic. > > >>>>Posix should standardize more of this. :( > > >>>> > > >>>>- Jay > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>---------------------------------------- > > >>>>>To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com > > >>>>>To: dabenavidesd at yahoo.es > > >>>>>Date: Tue, 31 May 2016 10:16:05 -0700 > > >>>>>From: mika at async.caltech.edu > > >>>>>Subject: Re: [M3devel] replace build system?? > > >>>>> > > >>>>>I mentioned this before a few times on this mailing list.... CM3 is already fairly > > >>>>>parallel if you turn on the right options. Every back-end invocation can be done > > >>>>>in parallel. > > >>>>> > > >>>>>Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... > > >>>>> > > >>>>>"Daniel Alejandro Benavides D." writes: > > >>>>>... > > >>>>>>A make-based build solution would have other advantages as well like=20 > > >>>>>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 > > >>>>>>four times faster. > > >>>>>... > > >>>>>_______________________________________________ > > >>>>>M3devel mailing list > > >>>>>M3devel at elegosoft.com > > >>>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>>_______________________________________________ > > >>>>M3devel mailing list > > >>>>M3devel at elegosoft.com > > >>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>> > > >>>_______________________________________________ > > >>>M3devel mailing list > > >>>M3devel at elegosoft.com > > >>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>> > > >> > > >>-- > > >>Rodney Bates > > >>rodney.m.bates at acm.org > > >>_______________________________________________ > > >>M3devel mailing list > > >>M3devel at elegosoft.com > > >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > >_______________________________________________ > > >M3devel mailing list > > >M3devel at elegosoft.com > > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jun 4 15:32:10 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 4 Jun 2016 09:32:10 -0400 Subject: [M3devel] replace build system?? In-Reply-To: References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575197BC.1030100@lcwb.coop> <20160604002337.GA30109@topoi.pooq.com> Message-ID: <20160604133210.GA19285@topoi.pooq.com> On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > Aside: > > LINIXLIBC > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > SPARC32_SOLARIS (which internally lets you chose C compiler) > ? How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is there a way of telling it otherwise? Do I have to reinstall from scratch? Is there an installer that knows I386_LINUX? Last time I looked there didn't seem to be. > They have all been present and working for years. > These other names are a warty legacy. > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) And how do I ask for the C code generator? -- hendrik From jay.krell at cornell.edu Sun Jun 5 09:46:56 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 5 Jun 2016 07:46:56 +0000 Subject: [M3devel] some quake proposals for more string functions? Message-ID: A new abstract type that represents a set of characters: ? It might actually be a string underneath with 0 or 1 at each position, to aid ease of implementation. ? or a 2-array of 64-bit integers (implied that only characters 0-127 can be in a set) And then functions that indicate how many characters from a set a string contains, or starts with, or ends with? a = CharSet("a") fo = CharSet("fo") foo = CharSet("foo") % redundant but ok fo == foo StringCountSet("foo", a) :0 % how many characters in set a are in string "foo" StringStartsSet("foo", fo) : 3 % how many characters in set fo does string "foo" start with StringEndSet("food", fo) : 0 % how many characters in set fo does string "food" end with FindFirstCharInSet("foo", fo): 0 FindFirstCharInSet("foo", CharSet("d"): -1 or length("foo") ? FindNextCharInSet("foo", fo, 0): 1 ? Might want to combine first/next functions, and require initialOffset and maximumOffset parameters? StringOnlyContainsCharsInSet(str, set): ? return?StringStartsSet(str, set) = length(str) StringContainsNoCharsInSet(str, set): ? return?StringCountSet(str, set) = 0 StringRemoveCharsInSetFromStart(str, set): ? return sub(str,?StringStartsSet(str, set), length(str)) StringRemoveCharsInSetFromEnd(str, set): ? return sub(str, lenth(str) -?StringStartsSet(str, set), length(str)) and then, really, we might add regexps find and find+replace, another day/year? There is some overlap here with the stuff Olaf put in, granted. Thank you, ?- Jay From lists at darko.org Mon Jun 6 01:09:35 2016 From: lists at darko.org (Darko Volaric) Date: Mon, 6 Jun 2016 01:09:35 +0200 Subject: [M3devel] [M3commit] [modula3/cm3] 137373: Add grisu support (little dragon) which allows fas... In-Reply-To: <5754aab547142_6cbe3fe7add0b2c0187858@hookshot-fe3-cp1-prd.iad.github.net.mail> References: <5754aab547142_6cbe3fe7add0b2c0187858@hookshot-fe3-cp1-prd.iad.github.net.mail> Message-ID: For those wondering: http://www.ryanjuckett.com/programming/printing-floating-point-numbers/ On Mon, Jun 6, 2016 at 12:41 AM, GitHub wrote: > Branch: refs/heads/master > Home: https://github.com/modula3/cm3 > Commit: 137373312f6dc35115b5070fcd477a376fa007ea > > https://github.com/modula3/cm3/commit/137373312f6dc35115b5070fcd477a376fa007ea > Author: peter mckinna > Date: 2016-06-03 (Fri, 03 Jun 2016) > > Changed paths: > A m3-libs/m3core/src/float/Common/grisu/CachedPowers.i3 > A m3-libs/m3core/src/float/Common/grisu/CachedPowers.m3 > A m3-libs/m3core/src/float/Common/grisu/Grisu.i3 > A m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > A m3-libs/m3core/src/float/Common/grisu/IEEE.i3 > A m3-libs/m3core/src/float/Common/grisu/IEEE.m3 > A m3-libs/m3core/src/float/Common/grisu/SimFP.i3 > A m3-libs/m3core/src/float/Common/grisu/SimFP.m3 > A m3-libs/m3core/src/float/Common/grisu/m3makefile > M m3-libs/m3core/src/float/Common/m3makefile > M m3-libs/m3core/src/float/IEEE/LongFloat.m3 > M m3-libs/m3core/src/float/IEEE/RealFloat.m3 > > Log Message: > ----------- > Add grisu support (little dragon) which allows faster conversion > from longreal (and real) to ascii in 99.4% of cases. In 0.6% one has > to fall back on dragon4. The algorithm uses native integers instead of > bignums to convert up to 4 times faster than dragon4. > > > Commit: 5703fac32dd7f7718c3f8ce70c15eb9af73499b5 > > https://github.com/modula3/cm3/commit/5703fac32dd7f7718c3f8ce70c15eb9af73499b5 > Author: peter mckinna > Date: 2016-06-03 (Fri, 03 Jun 2016) > > Changed paths: > M m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > M m3-libs/m3core/src/float/Common/grisu/IEEE.m3 > M m3-libs/m3core/src/float/Common/grisu/SimFP.m3 > > Log Message: > ----------- > Cosmetic clean. > > > Commit: 72fe0cb894af1eb605350424733d7bf78520dd43 > > https://github.com/modula3/cm3/commit/72fe0cb894af1eb605350424733d7bf78520dd43 > Author: peter mckinna > Date: 2016-06-04 (Sat, 04 Jun 2016) > > Changed paths: > M m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > > Log Message: > ----------- > Fix exponent to agree with dragon > > > Commit: 2dd875488ae9f37fee4473b39802e82e6b9068f3 > > https://github.com/modula3/cm3/commit/2dd875488ae9f37fee4473b39802e82e6b9068f3 > Author: peter mckinna > Date: 2016-06-06 (Mon, 06 Jun 2016) > > Changed paths: > M m3-sys/m3back/src/M3C.m3 > M scripts/python/capture-boot.py > > Log Message: > ----------- > Merge branch 'master' of https://github.com/modula3/cm3 > > > Compare: > https://github.com/modula3/cm3/compare/0f85470335a7...2dd875488ae9 > _______________________________________________ > M3commit mailing list > M3commit at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 6 19:39:49 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Jun 2016 17:39:49 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <20160604133210.GA19285@topoi.pooq.com> References: , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, <20160531171605.13AA61A2068@async.async.caltech.edu>, , <574F5CE2.8080506@lcwb.coop>, <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , <20160604133210.GA19285@topoi.pooq.com> Message-ID: Fair questions. LINUXLIBC6 and I386_LINUX are the same really. It should work to just rename a few files and change the string in the config file. Overkill but should also work, *something like*: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 ./do-cm3-all.py realclean I386_LINUX ./do-cm3-all.py buildship I386_LINUX # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX ./make-dist-cfg.py I386_LINUX # does very little, you could instead edit your config I have to run through this still myself. With the last two steps, it should use I386_LINUX by default you don't have to keep saying it. This is less overkill if e.g. you want to switch from LINUXLIBC6 to AMD64_LINUX. To use the C compiler, similarly: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using -- no need to a second time if did previous ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX c ./make-dist-cfg.py I386_LINUX Now, the last step, if you aren't going to be using the scripts all the time...you have to edit your bin/config/cm3cfg.common to enable the C backend. Where it says: if not defined ("M3_BACKEND_MODE") M3_BACKEND_MODE = "3" % -- defines how the frontend, backend, and assembler interact % "0" -- don't call m3_backend, M3CG produces object code % "1" -- don't call m3_backend, M3CG produces assembly code % "2" -- call m3_backend, it produces object code % "3" -- call m3_backend, it produces assembly code % "C" -- use C backend, and then compile with compile_c() end change: M3_BACKEND_MODE = "3" to: M3_BACKEND_MODE = "C" I haven't tried these *exact* steps though. We should nail the exact minimal steps and check them in to doc or scripts/python. I'm sure these are not minimal -- we should split it up as minimal to get a compiler, and then separate to build the rest of the system. And we might want to try rename or copy instead of rebuild. Something not in-place will also be good, more like make-dist.py. Sorry, I wanted to send more confident directions earlier but got hung up on some stuff. - Jay > Date: Sat, 4 Jun 2016 09:32:10 -0400 > From: hendrik at topoi.pooq.com > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > > Aside: > > > LINIXLIBC > > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > > SPARC32_SOLARIS (which internally lets you chose C compiler) > > ? > > How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is > there a way of telling it otherwise? Do I have to reinstall from > scratch? Is there an installer that knows I386_LINUX? Last time I > looked there didn't seem to be. > > > They have all been present and working for years. > > These other names are a warty legacy. > > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) > > And how do I ask for the C code generator? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jun 11 09:45:01 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 11 Jun 2016 07:45:01 +0000 Subject: [M3devel] cm3 -boot? Message-ID: Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. They seem useless currently. ?- Jay From rodney_bates at lcwb.coop Sat Jun 11 21:07:26 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 11 Jun 2016 14:07:26 -0500 Subject: [M3devel] cm3 -boot? In-Reply-To: References: Message-ID: <575C616E.4030608@lcwb.coop> Not I. All I did was try to make some wild guesses what to expand it to do in conjunction with the llvm backend. On 06/11/2016 02:45 AM, Jay K wrote: > Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? > I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. > They seem useless currently. > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Jun 12 09:35:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 12 Jun 2016 07:35:50 +0000 Subject: [M3devel] cm3 -boot? In-Reply-To: <575C616E.4030608@lcwb.coop> References: , <575C616E.4030608@lcwb.coop> Message-ID: There are multiple parts of "boot". I haven't actually looked into all of them, but might soon. There are: 1) "pushing the compilation along per source file". Compilation is a few steps, and boot deliberately stops earlier. Where it stops, I don't view as a scientific thing, but it is based kind of what tools people tend to have where. Our system kind of assumes that C cross compilers -- and the C runtime headers/libraries and the assembler and linker -- are not particularly available. So "boot" stops at C source or assembly source, assuming the target system has a native assembler, C compiler, linker, etc. Cross compilation systems are more common these days, but I don't yet view as adequately easy to produce or acquire. The C compiler part is actually pretty easy with gcc, but constructing the actually required adequate system -- linker, assembler, "crt0.o", headers, libraries (sysroot) is much more/worse. So I think the assumption is true enough and useful. 2) Production of some sort of makefiles or makeflle fragments to drive the later steps. One small dilema though, is to decide if the second part of bootstrap builds only cm3, then uses it to build the rest, or builds everything. If it only builds cm3, it has smaller requirements. For example, dynamic libs don't really matter. Also incrementality at this point matters less. This is what I have been doing, but I'm torn on the right choice. At the very least, boot1.py should tar.gz-up the source tree, so it is sure to match. Currently boot1.py does some makefile construction but I think maybe instead to repair/extend the cm3 -boot makefile production. Basically to output an autoconf/automake/libtool system. Or least to output something more complete here, even if not autoconf/make/libtool. (I finally am using these systems successfully, and they are ok, but e.g. libtool compiles everything twice, either with unnecessary variation or no variation at all, quite disappointing -- basically everyone using libtool can/should cut their build times in half.) ?- Jay ---------------------------------------- > Date: Sat, 11 Jun 2016 14:07:26 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 -boot? > > > Not I. All I did was try to make some wild guesses what to expand it to > do in conjunction with the llvm backend. > > > On 06/11/2016 02:45 AM, Jay K wrote: >> Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? >> I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. >> They seem useless currently. >> >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Jun 21 17:54:14 2016 From: jay.krell at cornell.edu (Jay) Date: Tue, 21 Jun 2016 08:54:14 -0700 Subject: [M3devel] Commits mails missing? Message-ID: <960DBCE3-8C99-423F-A8BE-D80796F05BA3@gmail.com> Github m3 commits mails missing? Just me? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 19 09:00:31 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jun 2016 07:00:31 +0000 Subject: [M3devel] m3 cvsweb? Message-ID: Is it possible to bring back www.elegosoft.com/cgi-bin/cvsweb.cgi? The history on github is sometimes unclear e.g. of m3-sys/cm3/src/Builder.m3. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 07:12:13 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 05:12:13 +0000 Subject: [M3devel] cm3 -boot Builder.GenObjectList etc. Message-ID: Does anyone have a full story around cm3 -boot? I "fixed" Builder.GenObjectList to produce valid make code,which is what I thought it was doing, but now I realize, I guess,it was generating quake code.The chunking was probably to workaround historical fixed size buffersand buffer overflows in quake, that we have since fixed. Anyway, I could fix this all up in one of a few ways: - do nothing; do everything in python - generate cmake stuff - generate autotools/libtool stuff - generate quake stuff - other Python nearly suffices, but the current direction bugsme in multiple ways: - I duplicate things like cflags/compiler/asssembler between config and python - The python hardcodes knowing what is a program vs. a library In going to try to complete the cm3 code here..like putting in linking,it strikes me that one problem is when to do probing for libraries,the logic around shared vs. static, prefering shared by default,prefering static if build_standalone, and using whatever is found if only one. Running that logic in cm3 -boot is slightly challenging.We'd either assume and hardcode file presence, possibly just assuming static,or defer the logic to the next phase. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 19 10:45:48 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jun 2016 08:45:48 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , , , <20160604133210.GA19285@topoi.pooq.com>, Message-ID: I'm pretty convinced the thing to do here, is still run cm3 in its "bulk" way, to output a bunch of .mc/.ic or .c files, and then use autotools/libtool, or possibly cmake to run the C compiler, the assembler, and lib/linker, and probably for ship/install as well. It should output configure.ac/Makefile.am files. This would give us both much increased portability and much decreased maintenance. And we'd use make -j for parallelism. The config files would go away. Quake might even go away. I oscillate between, on one side: There exists Posix cc, so one can portably run a C compiler. And ar+ranlib are very well standardized. But we go well beyond that -- we run the assembler, we produce shared libraries. We support many systems, with fairly minimal and efficient layering, but autotools/libtool and cmake go much further out of their way and support more. We basically only have gcc-based, gcc-like, and Solaris. gcc-like, I mean, Apple's clang accepts gcc options. When using the C backend, we could do another trick -- generate 4+ variations all in one file each with #if's around them: #if 32bit and little endian and posix // x86, arm32 ... #elif 64bit and little endian and posix // amd64, arm64 ... #elif 32bit and big endian and posix // sparc32, ppc_darwin ... #elif 64bit and big endian and posix // sparc64, ppc64_darwin ... #endif The #if's would be controlled by autoconf. We'd have one slightly bloated source distribution for all Posix systems. I know the autotools have a very mixed reputation, and it is deserved. Instead of sh/m4/sed/awk/make/python, they should just us C/C++, maybe Python. Their portability is difficult to reject though. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com Date: Mon, 6 Jun 2016 17:39:49 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] replace build system?? Fair questions. LINUXLIBC6 and I386_LINUX are the same really. It should work to just rename a few files and change the string in the config file. Overkill but should also work, *something like*: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 ./do-cm3-all.py realclean I386_LINUX ./do-cm3-all.py buildship I386_LINUX # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX ./make-dist-cfg.py I386_LINUX # does very little, you could instead edit your config I have to run through this still myself. With the last two steps, it should use I386_LINUX by default you don't have to keep saying it. This is less overkill if e.g. you want to switch from LINUXLIBC6 to AMD64_LINUX. To use the C compiler, similarly: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using -- no need to a second time if did previous ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX c ./make-dist-cfg.py I386_LINUX Now, the last step, if you aren't going to be using the scripts all the time...you have to edit your bin/config/cm3cfg.common to enable the C backend. Where it says: if not defined ("M3_BACKEND_MODE") M3_BACKEND_MODE = "3" % -- defines how the frontend, backend, and assembler interact % "0" -- don't call m3_backend, M3CG produces object code % "1" -- don't call m3_backend, M3CG produces assembly code % "2" -- call m3_backend, it produces object code % "3" -- call m3_backend, it produces assembly code % "C" -- use C backend, and then compile with compile_c() end change: M3_BACKEND_MODE = "3" to: M3_BACKEND_MODE = "C" I haven't tried these *exact* steps though. We should nail the exact minimal steps and check them in to doc or scripts/python. I'm sure these are not minimal -- we should split it up as minimal to get a compiler, and then separate to build the rest of the system. And we might want to try rename or copy instead of rebuild. Something not in-place will also be good, more like make-dist.py. Sorry, I wanted to send more confident directions earlier but got hung up on some stuff. - Jay > Date: Sat, 4 Jun 2016 09:32:10 -0400 > From: hendrik at topoi.pooq.com > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > > Aside: > > > LINIXLIBC > > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > > SPARC32_SOLARIS (which internally lets you chose C compiler) > > ? > > How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is > there a way of telling it otherwise? Do I have to reinstall from > scratch? Is there an installer that knows I386_LINUX? Last time I > looked there didn't seem to be. > > > They have all been present and working for years. > > These other names are a warty legacy. > > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) > > And how do I ask for the C code generator? > > -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 07:01:33 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 05:01:33 +0000 Subject: [M3devel] m3 cvsweb? In-Reply-To: References: Message-ID: I found it. https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cm3/src/Builder.m3 very useful because e.g. I put comments sometimes in code, e.g.: configure_c_compiler() % why? http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/Darwin.common.diff?r1=1.41;r2=1.42 - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: m3 cvsweb? Date: Sun, 19 Jun 2016 07:00:31 +0000 Is it possible to bring back www.elegosoft.com/cgi-bin/cvsweb.cgi? The history on github is sometimes unclear e.g. of m3-sys/cm3/src/Builder.m3. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 06:44:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 04:44:51 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: Message-ID: i.e.: diff --git a/m3-sys/m3quake/src/M3Path.m3 b/m3-sys/m3quake/src/M3Path.m3 index b3e0f27..1b3ac2c 100644 --- a/m3-sys/m3quake/src/M3Path.m3 +++ b/m3-sys/m3quake/src/M3Path.m3 @@ -22,8 +22,8 @@ TYPE CONST Suffix = ARRAY OSKind OF SMap { - (* Unix *) SMap { "", ".i3", ".ib", ".ic", ".is", ".io", - ".m3", ".mb", ".mc", ".ms", ".mo", + (* Unix *) SMap { "", ".i3", ".ib", ".ic", "_i.s", "_i.o", + ".m3", ".mb", ".mc", "_m.s", "_m.o", ".ig", ".mg", ".c", ".h", ".bc", ".s", ".o", ".a", ".a", ".m3x", "", ".mx", ".tmpl" }, (* GrumpyUnix *) SMap { "", ".i3", ".ib", ".ic", "_i.s", "_i.o", full system built on Darwin no problem as I expected. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: naming convention unix vs. grumpyunix? Date: Mon, 20 Jun 2016 01:40:21 +0000 I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 03:40:21 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 01:40:21 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? Message-ID: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 22 22:45:56 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 22 Jun 2016 15:45:56 -0500 Subject: [M3devel] Fwd: Re: Commits mails missing? In-Reply-To: <576AB04C.8050602@lcwb.coop> References: <576AB04C.8050602@lcwb.coop> Message-ID: <576AF904.40906@lcwb.coop> FWIW: For a long time, I have been getting three copies of these, two starting with [m3commit][modula3/cm3] in the subject, and one without the [m3commit]. For a few days, I was getting only the shorter one. But now the other two have shown up, for each such commit, 3 days later arriving, but showing the identical date and time of origination. On 06/21/2016 10:54 AM, Jay wrote: > Github m3 commits mails missing? Just me? > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at m3lists.elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Wed Jun 22 22:53:12 2016 From: lists at darko.org (Darko Volaric) Date: Wed, 22 Jun 2016 22:53:12 +0200 Subject: [M3devel] Fwd: Re: Commits mails missing? In-Reply-To: <576AF904.40906@lcwb.coop> References: <576AB04C.8050602@lcwb.coop> <576AF904.40906@lcwb.coop> Message-ID: When I changed it some time ago to fix an issue related to people using emails on github that were not subscribed to the list, your email was in there along with the list address, so I left it in there. You might also be personally subscribed to the commits, which would explain the third copy. On Wed, Jun 22, 2016 at 10:45 PM, Rodney M. Bates wrote: > > FWIW: > > For a long time, I have been getting three copies of these, two starting > with > [m3commit][modula3/cm3] in the subject, and one without the [m3commit]. > For a few days, I was getting only the shorter one. But now the other two > have shown up, for each such commit, 3 days later arriving, but showing the > identical date and time of origination. > > On 06/21/2016 10:54 AM, Jay wrote: > >> Github m3 commits mails missing? Just me? >> >> >> - Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at m3lists.elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> > -- > Rodney Bates > rodney.m.bates at acm.org > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 23 02:38:18 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jun 2016 00:38:18 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Thu Jun 23 08:33:00 2016 From: dmuysers at hotmail.com (dirk muysers) Date: Thu, 23 Jun 2016 08:33:00 +0200 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay -------------------------------------------------------------------------------- From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------------------------------------------------------------------------- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From nafkhami at elegosoft.com Thu Jun 23 11:01:04 2016 From: nafkhami at elegosoft.com (navid afkhami) Date: Thu, 23 Jun 2016 11:01:04 +0200 Subject: [M3devel] New Link for M3 Mailing lists and Archives Message-ID: <576BA550.9000403@elegosoft.com> Hi, We would like to inform you that the Mailing lists and Archives links are available again and you can find them in the mentioned link below. https://m3lists.elegosoft.com Thank you elego Software Solutions -- Navid Afkhami IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 navid.afkhami at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From jay.krell at cornell.edu Thu Jun 23 20:33:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jun 2016 18:33:47 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu>, , Message-ID: Not really. My goal is to be close as possible to: tar xf foo.tar.gz mkdir bar # optional cd bar # or foo ../foo/configure make sudo make install Nagging questions: Is there one foo.tar.gz for everyone and autoconf picks the right part of it, or people pick the "correct" one for their system. Is there a few such files -- target.tar.gz, the-rest-m3.tar.gz, maybe m3cc.tar.gz -- this is how 3.6 was structured Back then quake was written in C however, not sure it matters. Do we first build cm3 and then the rest of the system using it, or do we just use "make" to build everything. I can imagine how to build everything from assembly using autoconf/libtool/make, but I kinda only want to do that only if cm3 also reuses the same infrastructure. Sometimes I also think of giving up on dynamic linking, since that is one of the bigger thorns. - Jay From: dmuysers at hotmail.com To: jay.krell at cornell.edu; hosking at purdue.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Thu, 23 Jun 2016 08:33:00 +0200 Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Jun 24 19:34:16 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Jun 2016 17:34:16 +0000 (UTC) Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: <359115250.1721526.1466789656512.JavaMail.yahoo@mail.yahoo.com> Hi all: Although I have proposed moving to Java or encouraged al least (both build system and runtime), you have to take for granted that there are sources of security leaks like [1] (Protection in Programming-Language Translations)explains in Java to JVML translation. Specially fault isolation properties defined by Modula-3 UNSAFE modules. Thanks in advance [1] ?Digital Systems Research Center: Report 154?. [Online]. Available on: ftp://ftp.hpl.hp.com/gatekeeper/pub/compaq/SRC/research-reports/abstracts/src-rr-154.html. [Accesed: 24-jun-2016]. El Jueves 23 de junio de 2016 13:33, Jay K escribi?: Not really. My goal is to be close as possible to: tar xf foo.tar.gz mkdir bar # optional cd bar # or foo ../foo/configure make sudo make install Nagging questions: Is there one foo.tar.gz for everyone and autoconf picks the right part of it, or people pick the "correct" one for their system. Is there a few such files -- target.tar.gz, the-rest-m3.tar.gz, maybe m3cc.tar.gz -- this is how 3.6 was structured Back then quake was written in C however, not sure it matters. Do we first build cm3 and then the rest of the system using it, or do we just use "make" to build everything. I can imagine how to build everything from assembly using autoconf/libtool/make, but I kinda only want to do that only if cm3 also reuses the same infrastructure. Sometimes I also think of giving up on dynamic linking, since that is one of the bigger thorns. - Jay ________________________________ From: dmuysers at hotmail.com To: jay.krell at cornell.edu; hosking at purdue.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Thu, 23 Jun 2016 08:33:00 +0200 Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay ________________________________ From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: >I propose making unix match grumpyunix and removing grumpyunix. > >There is slight upside and should be no downside. > >The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. > >Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? > >I expect everything will just work. > >- Jay > > > > > _______________________________________________ >M3devel mailing list >M3devel at m3lists.elegosoft.com >https://m3lists.elegosoft.com/mailman/listinfo/m3devel > ________________________________ _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel From wagner at elegosoft.com Sat Jun 25 14:24:59 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 25 Jun 2016 14:24:59 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? Message-ID: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Lurking in the shadows and reading all the recent mails about improving the build system and seeing more and more steps 'backwards' to adapt to C, autoconf, makefiles and more overcome tools and concepts, a heretical thought comes to mind: Have you ever considered using docker as a means to provide easy access and distribution of Modula-3? Some historic background: in the eighties and nineties, standardization took place on the language level with structured programming, data types, modules and interfaces, classes etc. I know of several attempts to extend the new concepts and abstractions to operating system level: Smalltalk, ELAN (extended Algol-68) (EUMEL OS ;-) and even Modula-3 come to mind (SPIN OS). All these attempts were partially successful in their implementation, but never succeeded in gaining even 0.01% of acceptance in the OS market. On the OS level, however, all standardization attempts sadly failed (IMHO). POSIX is only a crutch, and all vendors and even the free Unix derivates and lookalikes went their own way wrt. libraries, linking and packaging. As Greg Nelson (IIRC) mentioned to me some years ago, the aspects of linking have never been analyzed in the scope of Modula-3, and so have all the other OS-specific tasks like package building and distribution. When the Modula-3 package system has been designed, there were no Redhat or Debian packages, and perhaps only some first attempts at SunOS packages. In the Java realm at least, packages, repositories and build systems like maven have been largely successful, and they look very much like a redesigned M3 build system. Recently however, a pragmatical approach at distributing easy-to-run software services and applications has been and still is very successful: bundle all the needed stuff together and put it into one 'container'. The free and most successful implementation of this is the Linux docker system. While the implementation is ABI-centric and depends on Linux, nontheless docker containers can be run easily on MacOS and Windows too, which should cover more than 95% of the installation base, I think. At Elego we've been using docker successfully for more than a year now to automate and standardize both our development and system operation procedures. I haven't thought this through, but there could be several ways of using docker together with the CM3 system. For beginners, there could be a standalone docker image with the compiler, all packages installed, and an HTTP-service as GUI, which would be started automatically when running the docker image. Local directories may be mounted at predefined locations to provide access to local files. We already have the HTTP-service GUI in our sources! As a development system, I could envisage a docker image with the compiler and build system in one container image and the package libraries -- perhaps even separated for different target architectures -- in persistent data containers. Initial versions of the latter could also be distributed via existing docker registries in the internet. We would have to define something like a standard M3 docker interface for M3 docker containers, which could be done with container entrypoints and some additional scripts. We could use docker-compose specifications to define complex setups. Finally, the M3 docker system could be extended not only to run new binaries inside its own container, but to build standalone docker containers to run the applications and distribute them via the existing registry system in the internet. It could all be as easy as typing docker run m3-juno to start the Juno-2 constrained based editor on your local system without even installing one Modula-3 package or dependency. You only need to have installed docker or one of its emulation packages. CAVEATS This is not for everybody and every purpose though. First, it does not help in the realm of efficient systems programming for dedicated target platforms. As this was one of the original goals of Modula-3 development, it is somehow at cross-purposes with its intention. Second, it doesn't solve the building and linking problems, but works around them. Third, it adds another level of complexity and probably hundreds of MBs to running an M3 application. This seems to be more and more acceptable, looking at other development though. ;-) PROS It would completely resolve the access and distribution aspect of Modula-3 IMO. No questions anymore about how to install the compiler and how to get it running on system X. At least, if the system X is able to run docker containers, but as I said before, most systems today are. What do you think? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dmuysers at hotmail.com Sat Jun 25 15:46:20 2016 From: dmuysers at hotmail.com (dirk muysers) Date: Sat, 25 Jun 2016 15:46:20 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Message-ID: Another --less heavyweight-- package distribution system is Zero Install. https://0install.de/ -----Original Message----- From: Olaf Wagner Sent: Saturday, June 25, 2016 2:24 PM To: m3devel Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? Lurking in the shadows and reading all the recent mails about improving the build system and seeing more and more steps 'backwards' to adapt to C, autoconf, makefiles and more overcome tools and concepts, a heretical thought comes to mind: Have you ever considered using docker as a means to provide easy access and distribution of Modula-3? Some historic background: in the eighties and nineties, standardization took place on the language level with structured programming, data types, modules and interfaces, classes etc. I know of several attempts to extend the new concepts and abstractions to operating system level: Smalltalk, ELAN (extended Algol-68) (EUMEL OS ;-) and even Modula-3 come to mind (SPIN OS). All these attempts were partially successful in their implementation, but never succeeded in gaining even 0.01% of acceptance in the OS market. On the OS level, however, all standardization attempts sadly failed (IMHO). POSIX is only a crutch, and all vendors and even the free Unix derivates and lookalikes went their own way wrt. libraries, linking and packaging. As Greg Nelson (IIRC) mentioned to me some years ago, the aspects of linking have never been analyzed in the scope of Modula-3, and so have all the other OS-specific tasks like package building and distribution. When the Modula-3 package system has been designed, there were no Redhat or Debian packages, and perhaps only some first attempts at SunOS packages. In the Java realm at least, packages, repositories and build systems like maven have been largely successful, and they look very much like a redesigned M3 build system. Recently however, a pragmatical approach at distributing easy-to-run software services and applications has been and still is very successful: bundle all the needed stuff together and put it into one 'container'. The free and most successful implementation of this is the Linux docker system. While the implementation is ABI-centric and depends on Linux, nontheless docker containers can be run easily on MacOS and Windows too, which should cover more than 95% of the installation base, I think. At Elego we've been using docker successfully for more than a year now to automate and standardize both our development and system operation procedures. I haven't thought this through, but there could be several ways of using docker together with the CM3 system. For beginners, there could be a standalone docker image with the compiler, all packages installed, and an HTTP-service as GUI, which would be started automatically when running the docker image. Local directories may be mounted at predefined locations to provide access to local files. We already have the HTTP-service GUI in our sources! As a development system, I could envisage a docker image with the compiler and build system in one container image and the package libraries -- perhaps even separated for different target architectures -- in persistent data containers. Initial versions of the latter could also be distributed via existing docker registries in the internet. We would have to define something like a standard M3 docker interface for M3 docker containers, which could be done with container entrypoints and some additional scripts. We could use docker-compose specifications to define complex setups. Finally, the M3 docker system could be extended not only to run new binaries inside its own container, but to build standalone docker containers to run the applications and distribute them via the existing registry system in the internet. It could all be as easy as typing docker run m3-juno to start the Juno-2 constrained based editor on your local system without even installing one Modula-3 package or dependency. You only need to have installed docker or one of its emulation packages. CAVEATS This is not for everybody and every purpose though. First, it does not help in the realm of efficient systems programming for dedicated target platforms. As this was one of the original goals of Modula-3 development, it is somehow at cross-purposes with its intention. Second, it doesn't solve the building and linking problems, but works around them. Third, it adds another level of complexity and probably hundreds of MBs to running an M3 application. This seems to be more and more acceptable, looking at other development though. ;-) PROS It would completely resolve the access and distribution aspect of Modula-3 IMO. No questions anymore about how to install the compiler and how to get it running on system X. At least, if the system X is able to run docker containers, but as I said before, most systems today are. What do you think? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel From adacore at marino.st Sat Jun 25 16:06:12 2016 From: adacore at marino.st (John Marino) Date: Sat, 25 Jun 2016 16:06:12 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Message-ID: <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> On 6/25/2016 2:24 PM, Olaf Wagner wrote: > Lurking in the shadows and reading all the recent mails about improving > the build system and seeing more and more steps 'backwards' to adapt > to C, autoconf, makefiles and more overcome tools and concepts, a > heretical thought comes to mind: Have you ever considered using docker > as a means to provide easy access and distribution of Modula-3? You mean Linux-only Docker? In general this concept would not be compatible with typical s/w build repositories used by various OS. If you went to a primary Docker distribution system, you're basically saying A) M3 is limited to Linux and B) It's distributed by third parties (you) only. At first glance, I'd say it's a bad idea. John --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus From jay.krell at cornell.edu Sat Jun 25 19:43:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Jun 2016 17:43:07 +0000 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation: automake "compiles" to make cm3 "compiles" to quake Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem. Not the entire problem -- cloud app declaration, software composition. Though it still low level -- how do I describe and join multiple containers? I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work. I think they are great for portabilitity. Even better, generate C++ and likely gain efficient exception handling. Delegating the work to the C++ implementers, which generally do much better than us here (basically inheriting "stack walkers" wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could: - accept our current platforms/ports (i.e. don't fret over AIX, etc.) - accept that the config files aren't bad asis - bootstrap can just automate slightly more and build cm3 - use cm3 to build and install everything else, including cm3cg, dynamic linking, etc. That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core The system it is building would have to be matched with the cm3 itself. It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! - Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 20:17:11 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 18:17:11 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv0826315534 #yiv0826315534 --.yiv0826315534hmmessage P{margin:0px;padding:0px;}#yiv0826315534 body.yiv0826315534hmmessage{font-size:12pt;font-family:Calibri;}#yiv0826315534 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 20:20:51 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 18:20:51 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> hi:besides that cm3 v4 could compile pm3 anyway. El S?bado 25 de junio de 2016 13:17, Daniel Alejandro Benavides D. escribi?: Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv3270809950 --.yiv3270809950hmmessage P{margin:0px;padding:0px;}#yiv3270809950 body.yiv3270809950hmmessage{font-size:12pt;font-family:Calibri;}#yiv3270809950 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 21:41:44 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 19:41:44 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> Message-ID: <285148656.2323402.1466883704854.JavaMail.yahoo@mail.yahoo.com> Hi all:anyway whether C or assembly should be available as options for bootstrapping, now we just have gcc and not all platforms are gcc ready.Thanks in advance El S?bado 25 de junio de 2016 13:20, Daniel Alejandro Benavides D. escribi?: hi:besides that cm3 v4 could compile pm3 anyway. El S?bado 25 de junio de 2016 13:17, Daniel Alejandro Benavides D. escribi?: Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv4790279345 --.yiv4790279345hmmessage P{margin:0px;padding:0px;}#yiv4790279345 body.yiv4790279345hmmessage{font-size:12pt;font-family:Calibri;}#yiv4790279345 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 26 08:34:39 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jun 2016 06:34:39 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu>, Message-ID: Slight clarification from earlier: "Dependency on gcc" is gcc or cc is a good way to run the correct assembler, which they decide to do for .s suffix, but not .ms/.is And, it turns out, when faced with assembly source, automake likes to run the C compiler. If you imagine a strategy where we write automake intput, then .s helps. Though the next possible step is automake being "native" -- and shiping some form of it for Windows, which I know doesn't go over well, but it really is impressive the extent that GNU build tools (gcc/ld) can target Windows -- it would be a good direction now for an AMD64 Windows port (besides the C backend...and ignoring LLVM...which maybe I should move my focus too...) ?- Jay ________________________________ > From: jay.krell at cornell.edu > To: hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] naming convention unix vs. grumpyunix? > Date: Thu, 23 Jun 2016 00:38:18 +0000 > > This is a bit long and out of order, sorry. > Simple story is for us to get out of the platform-specific build system > maintenance business, and reuse larger portability from other projects. > > > > I've been wrestling with this in my head a long while. > > > - I don't like maintaining the config files. > It is hard to be an expert on dynamic linking across "many" operating > systems, linkers, versions. > > > - I don't like that for example an AIX port remains absent. > And now I see AIX doesn't have $ORIGIN. > > > - It bothers me just slightly that we aren't portable > to the older systems that lack $ORIGIN. > > > $ORIGIN is nice if you are redistributing binaries, > that will be moved around, but it was never needed > for self-built software, or software installed to > an agreed upon place, and it isn't supported in setuid or such > programs. > > (Aside -- and remember how bad it used to be? > We used to distribute binaries with random hardcoded paths, > and advise people to set LD_LIBRARY_PATH. Even for stuff people > self-built, it wasn't good. So I did improve things > but I don't think it is worth us doing ourselves.) > > > - Our current bootstrap/cross-build story isn't automated enough. > And then, what should it look like? > > > - Generating cmake or autoconf/automake/libtool input provides some > potential answers. > > I'd really like to delegate to folks that did and will continue to > port pretty much everywhere. > Sometimes I think, hey, we can just do what we need ourselves, but > then I see how > much gnarly system-specific knowledge autotools/cmake deliver nicely > to their users. > > > I had a mental stumbling block for years with cmake/autotools but finally > got over it. I have prototyped some simple uses, both with recursive > make and non-recursive make. > > > configure is a bit slow, but we'd have a very minimal one. > The resulting make invocations are ok. > > > I can almost just generate makefiles myself, but then for example > I don't know much about "install". cmake/automake provide me "install" > with me knowing nothing. > > > I don't really want to be an expert in make, compiler flags, linker flags, > Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but > not so much as the make/sed/awk/sh level. > > > There are a few details of autoconf/cmake/libtool I don't like, where > the Modula-3 > build system is clearly and simply superior. And other areas where I'm not > sure what is ideal. > > > Where Modula-3 is clearly superior in that in producing static and dynamic > libraries, it only ever compiles once. cmake and libtool are pretty keen > on compiling everything twice -- even with identical command lines. > > > Where I'm not sure is our probing for libraries and the > build_standalone feature. > I think if we did things a little different/better, we wouldn't even > have cm3 > be standalone. > > > I very much want to offer to users the: > tar xf cm3... > cd cm3... > configure > make > make install > > > sort of experience. > > > There are slight difficulties. > configure probes the C compiler for what it produces. > Let's ignore C-backend and LLVM for now and consider cm3cg. > > > The likely best bootstrap format is assembly source. Like the 3.6 release. > For just cm3/m3core/libm3, or the entire system? > > > So configure probing vs. having on hand possibly just one assembly > source is a bit of a misfit. > > > Perhaps configure would be tailored to hardcode what the distribution > contains. > > > Or perhaps the distribution would contain "everything" and configure > would pick the right one. > It is obviously wasteful, but these days maybe ok, and the result > easier for people to install. > > > The C generating backend doesn't fix this much or entirely, since the > C is still target-specific. > Maybe we can fold the C down to just a few platforms, and then the > idea of one cross-platform distribution > might work. Maybe eventually the generated C can speak in "integer" > and array/struct references, instead > of front-end computed offsets, but that is a ways off. > > > autotools/libtool also solve that problem where non-shipped binaries > don't run. > Something we have hacked on by sprinkling build_standalone around. > I'm not sure if cmake fixes this. > > > I'm not sure they solve it the way I want though -- I'd like to have > the uninstalled > paths hardcoded, then relink or otherwise binary edit in install. > > > One thing I need to study a bit more is how to install all the extra > stuff to the pkg directories. > > As well,...so many things... we have this structure: > bin/foo > lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) > pkg/foo/TARGET/foo.so > > > I have always found it a little suspicious that binaries have implicit > TARGET > but pkgs have explicit TARGET. I somewhat pine for a layout that can > accomodiate > all targets including the bin directory. > > > I suppose if bin and lib are what run, and pkg is only for building, > this accomodates > unshipped cross builds nicely. But ideally you could have a runnable > PPC_DARWIN/I386_DARWIN/AMD664_DARWIN > system all in structure (caveat that PPC_DARWIN doesn't work in > Rosetta because of our > preemptive suspend -- cooperative suspend would fix that.) > > > - Jay > > > > > ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] naming convention unix vs. grumpyunix? > Date: Wed, 22 Jun 2016 21:19:12 +0000 > > Why import dependencies on make and automake? > > Sent from my iPad > > On Jun 22, 2016, at 9:41 PM, Jay K > > wrote: > > I propose making unix match grumpyunix and removing grumpyunix. > > There is slight upside and should be no downside. > > The upside is that various tools -- make and automake -- know that .s > is assembly and will assemble it. > > Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. > foo.m3 => foo.ms/foo.mo? > > I expect everything will just work. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at m3lists.elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Sun Jun 26 09:15:37 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jun 2016 07:15:37 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , , , <20160604133210.GA19285@topoi.pooq.com>, Message-ID: >> How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is? >> there a way of telling it otherwise? Do I have to reinstall from? Ok I went through this exercise. Finally getting at least some cross tools on the Linux/amd64 system to target x86. gcc and libc but not libxaw. I had some trial/error and hits/misses. I started from no x86 tools (not even gcc) and I never got 32bit Xaw working on AMD64 Linux working. So I did remove all the gui stuff from pkginfo.txt. But a native system won't have these troubles. But anyway, based on my experience: ?vi $HOME/cm3.l6/bin/cm3.cfg? ?Change HOST and TARGET to I386_LINUX unconditionally:? ? ? ?jay at debamd64:~/dev2/cm3/m3-libs/m3core$ cat ?/home/jay/cm3.l6/bin/cm3.cfg ? ?if not defined("SL") SL = "/" end? ?HOST = "I386_LINUX"? ?TARGET = "I386_LINUX"? ?INSTALL_ROOT = (path() & SL & "..")? ?include(path() & SL & "config" & SL & TARGET)? ? ? ?You have to change them both for the existing cm3cg to be used.? ?There are ways other than changing HOST, lik3 cm3 -Dm3back=cm3cg.? ?Your tree probably needs to be all in sync first for this to work, due to the reuse of existing cm3 and cm3cg?with a fresh m3core/libm3. ?i.e. first do an upgrade.py.? ? Manually build m3core/libm3. ? ? ? ? cd $ROOT/m3-libs/m3core ? ? cm3 ? ? cm3 -ship ?? ? cd ../libm3 ? ? cm3 ? ? cm3 -ship ? ? ? ? ? ?and then scripts/python/upgrade-full.sh ? That's it. Pretty easy. If you want to skip the upgrade, you can probably first copy the LINUXLIBC6 pkg/m3core, libm3 to I386_LINUX. ?- Jay ? ________________________________ > From: jay.krell at cornell.edu > To: hendrik at topoi.pooq.com > Date: Mon, 6 Jun 2016 17:39:49 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > Fair questions. > > > LINUXLIBC6 and I386_LINUX are the same really. > > It should work to just rename a few files and change the string in the > config file. > > Overkill but should also work, *something like*: > > cd scripts/python > edit ../pkginfo.txt > remove these two lines: > odbc database > postgres95 database > rm ../PKGS # might not be needed but doesn't hurt > ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 > ./do-cm3-all.py realclean I386_LINUX > ./do-cm3-all.py buildship I386_LINUX > # Quick test of compiler, should run and error: > ../../m3-sys/cm3/I386_LINUX/cm3 > If it fails though, we have already shipped most stuff, so not good. > I need to rewrite these steps to be more friendly if they fail. > > # and then some final small steps > ./install-cm3-compiler.py I386_LINUX > ./make-dist-cfg.py I386_LINUX # does very little, you could instead > edit your config > > > > I have to run through this still myself. > With the last two steps, it should use I386_LINUX by default you don't > have to keep saying it. > > > This is less overkill if e.g. you want to switch from LINUXLIBC6 to > AMD64_LINUX. > > > To use the C compiler, similarly: > cd scripts/python > edit ../pkginfo.txt > remove these two lines: > odbc database > postgres95 database > rm ../PKGS # might not be needed but doesn't hurt > ./upgrade.py # be current in whatever you are using -- no need to a > second time if did previous > ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there > ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there > # Quick test of compiler, should run and error: > ../../m3-sys/cm3/I386_LINUX/cm3 > If it fails though, we have already shipped most stuff, so not good. > I need to rewrite these steps to be more friendly if they fail. > > # and then some final small steps > ./install-cm3-compiler.py I386_LINUX c > ./make-dist-cfg.py I386_LINUX > > Now, the last step, if you aren't going to be using the scripts all > the time...you have to edit your bin/config/cm3cfg.common to enable the > C backend. > Where it says: > > if not defined ("M3_BACKEND_MODE") > M3_BACKEND_MODE = "3" > % -- defines how the frontend, backend, and assembler interact > % "0" -- don't call m3_backend, M3CG produces object code > % "1" -- don't call m3_backend, M3CG produces assembly code > % "2" -- call m3_backend, it produces object code > % "3" -- call m3_backend, it produces assembly code > % "C" -- use C backend, and then compile with compile_c() > end > > > change: > M3_BACKEND_MODE = "3" > > to: > M3_BACKEND_MODE = "C" > > I haven't tried these *exact* steps though. > > > We should nail the exact minimal steps and check them in to doc or > scripts/python. > I'm sure these are not minimal -- we should split it up as minimal to > get a compiler, > and then separate to build the rest of the system. And we might want > to try rename or copy instead > of rebuild. > > > Something not in-place will also be good, more like make-dist.py. > Sorry, I wanted to send more confident directions earlier but got hung > up on some stuff. > > > > - Jay > > > >> Date: Sat, 4 Jun 2016 09:32:10 -0400 >> From: hendrik at topoi.pooq.com >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: >>> Aside: >>>> LINIXLIBC >>> Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 >>> SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD >>> SPARC32_SOLARIS (which internally lets you chose C compiler) >>> ? >> >> How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is >> there a way of telling it otherwise? Do I have to reinstall from >> scratch? Is there an installer that knows I386_LINUX? Last time I >> looked there didn't seem to be. >> >>> They have all been present and working for years. >>> These other names are a warty legacy. >>> The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. > 1) The assumption that Linux is x86 only. 2) The fact that libc5 back > in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible > with libc6. There is more motivation imho for pthreads vs. userthreads > targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should > be link compatible with one of them though?) >> >> And how do I ask for the C code generator? >> >> -- hendrik > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at elegosoft.com Mon Jun 27 08:10:09 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 27 Jun 2016 08:10:09 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> On Sat, 25 Jun 2016 17:43:07 +0000 Jay K wrote: > I agree and disagree. > > Regarding Posix -- I think it did well for low level C. > > I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. > > There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation: automake "compiles" to make cm3 "compiles" to quake > Quake was written in C. > > The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. > > The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! > > cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. > > I'm aware of Docker. > > I think it is solving a problem. Not the entire problem -- cloud app declaration, software composition. Though it still low level -- how do I describe and join multiple containers? They invented docker-compose for that. > I'm not sure it is relevant here. > > However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I wouldn't like to give us dynamic linking, too, nor anything we have now. > I don't think C and autoconf/automake are clearly a step backward. See below. > I want more portability and, I admit, less work. I think they are great for portabilitity. Even better, generate C++ and likely gain efficient exception handling. Delegating the work to the C++ implementers, which generally do much better than us here (basically inheriting "stack walkers" wherever they exist.) > > I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) > > I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. > > If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. > > There are middle grounds that don't "step backward".We could: - accept our current platforms/ports (i.e. don't fret over AIX, etc.) - accept that the config files aren't bad asis - bootstrap can just automate slightly more and build cm3 - use cm3 to build and install everything else, including cm3cg, dynamic linking, etc. That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. > > One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. > We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. > configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core The system it is building would have to be matched with the cm3 itself. It might turn around at the cm3 point and use that cm3 to rebuild. > > This is only a minor layering/scripting over what we already have. > This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. > Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. > Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! > - Jay > > > To: wagner at elegosoft.com; m3devel at elegosoft.com > > From: adacore at marino.st > > Date: Sat, 25 Jun 2016 16:06:12 +0200 > > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > > Lurking in the shadows and reading all the recent mails about improving > > > the build system and seeing more and more steps 'backwards' to adapt > > > to C, autoconf, makefiles and more overcome tools and concepts, a > > > heretical thought comes to mind: Have you ever considered using docker > > > as a means to provide easy access and distribution of Modula-3? > > > > You mean Linux-only Docker? > > > > In general this concept would not be compatible with typical s/w build > > repositories used by various OS. If you went to a primary Docker > > distribution system, you're basically saying A) M3 is limited to Linux > > and B) It's distributed by third parties (you) only. > > > > At first glance, I'd say it's a bad idea. > > > > John Hi Jay and everybody else, I think I've been slightly misunderstood; but that was to be expected. I agree with most of the things you say. For now I'd just like to add some more or less random comments: * "backwards" didn't mean a step back for cm3, but only that I think that the approach it takes is just pragmatical: testing for everything that might be there or not. And it was certainly not intended to criticise anybody's recent work on cm3. * I didn't really like cminstall, too, though I understand it might seem so. Replacing it with a more sophisticated configure/automake at installation time would probably help the installation very much. * I doubt that generating C or C++ will be the right way to go, though I'm not against it as an alternative way/backend. The first versions of M3 used C as intermediate language, and after some years it was said to have been "a mediocre choice at best". Been there, done that. * I think build systems need to provide a purely declarative interface for the user; m3makefiles/quake is a rather good implementation. * I don't think we should try to make the M3 compiler behave more like a C compiler; rather than stepping from handling one package at a time to compiling single sources we need to look at building complete systems at a time. * I don't think we should try to unify/combine the m3 package concept with OS installation packages. * I don't think that using docker will help with providing a better build system, it might just help to make M3 more easily available to more users and thus help to keep the community alive. * I don't think that the M3 community currently has the ressources to really solve the issues of a better build system and easy distribution with all existing OS platforms. There are just too few people and the tasks are too huge. So I thought it might be worth thinking of somehing completely different to attract more people. * I guess that to come up with a reasonable declarative new or better build system, we'd need one common stable platform for it, and separate this task from those of achieving portability for all platforms. The platform might be an interpreter (like UCSD Pascal used or the JVM) or a really stable system and library interface (like an extended POSIX layer) or something completely new ;-) LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctly. Using an interpreter would also give up one of the big advantages of current CM3, that it can be compiled and linked natively on OSs. Finally: There are too many sentences starting with "I". Please excuse me. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 27 08:31:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 06:31:47 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: redirecting... > Olaf > LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl I would like to take this up, maybe soon. I do have a bit of an agenda. Maybe my priorities are mixed up. 1 Provide a very portable system. 2 Provide an easy to install and use system. 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least for their supported backends). 4 Maybe write our own backends. Where is the LLVM support at? Mostly working? Barely working? I know LLVM is big and changing, and maybe they don't value compatibility of bitcode. But look at what we have with the gcc backend. Even if we didn't have to patch it at all, I expect we'd still have to keep and build a local copy. Perhaps we should just do that? With LLVM, with its different licensing, perhaps we could get our "frontend" merged upstream, but this would then give us a compatibility burden in the persisted m3cg. Is that ok? It is hypothetical at this point. I know everyone here doesn't really like C/C++ (except me). And, more significantly, I know the system written in itself is a great test case, but I wonder if we shouldn't write a new "real" frontend in C or C++, and see if we can't merge that upstream with gcc and/or clang. It also worth mentioning that I believe gcc's Ada front end is written in Ada -- you don't actually have to write the frontend in C/C++ to merge upstream. But there might remain licensing concern. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 27 08:39:12 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 06:39:12 +0000 Subject: [M3devel] parse.c licensing question, dual? Message-ID: - I basically understand licensing. - I understand GPL - I understand more liberal BSD license - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. What I do not understand: - dual licensing - who owns parse.c - can parse.c be dual licensed? In particular: jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ m3-parse.h parse.c m3-def.h m3cg.h Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly in-process ones, either call-based or "linearized IR in memory". In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. m3cg.h is output by m3cggen. m3-def.h I wrote. These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. Can we also BSD license them or such? (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Jun 27 08:53:33 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 27 Jun 2016 08:53:33 +0200 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: Message-ID: <20160627085333.916aae7db6aa078d5300c739@elegosoft.com> On Mon, 27 Jun 2016 06:39:12 +0000 Jay K wrote: > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. > > > What I do not understand: > - dual licensing > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? > > (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) My understand is that you can put any license on things you wrote yourself. I'm not really sure, but I doubt that there is any legal entity left that cares for the M3 sources from DEC SRC (if it is that old). So I _think_ that we (you) might change the copyright for those. To be more compatible with the GNU stuff, it might be better to use LGPL together with the gcc backend. I am not a lawyer though. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 27 10:16:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 08:16:06 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> Message-ID: I'm going to read Olaf's assertion as 1 "go from GPL to LGPL", not 2 "go from DEC to LGPL". Imho we should use the OpenBSD or FreeBSD or NetBSD licenses. The OpenBSD folks are..funny but right-seeming. Their take for example on the Apache 2.0 license is -- why another long license? They'd need to pay a lawyer to allow for it. So just don't allow for it. They have stayed back with Apache 1.x for this reason. So you should reuse an existing short license. So, sorry, another question I forgot to ask -- who owns parse.c? Can we relicense it? And, ok, I own m3-def.h. I can just paste two license into it? I'll research the old Qt story here I guess or otherwise research. There is also a claim that the FreeBSD license is GPL-compatible, which implies we can use it on parse.c/m3-def.h -- just a single license. That is clearer to me. I just understand what it means to have two licences, unless, e.g. the license is context-dependent -- different people get different license depending on situation, like if they paid, or if they are getting paid. I think I was going to use m3-def.h/parse.c in the C backend, writing it in C or C++, and the intervening layer that allows easily writing multiple passes over the IR. The result instead was incredibly tedious and makes changing the IR more difficult/tedious. If I embark on an LLVM backend, I'll again be tempted to do that. It would be nice if we could relicense all the DEC SRC stuff as slightly more liberal BSD. I see DEC SRC seems to have an optional give back clause -- if you give your changes back to DEC, then you also license it to them liberally. But give back doesn't seem mandatory. 4. Improvements. LICENSEE hereby grants to DIGITAL a non-exclusive, non-transferable, royalty free right to use, modify, reproduce and distribute with the right to sublicense at any tier, any improvements, enhancements, extensions, or modifications that LICENSEE make to SOFTWARE, provided such are returned to DIGITAL by LICENSEE. - Jay From: hosking at purdue.edu To: wagner at elegosoft.com CC: jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] parse.c licensing question, dual? Date: Mon, 27 Jun 2016 06:55:17 +0000 Please be careful here. Going from the current license to LGPL is probably not the best route for CM3! On 27 Jun 2016, at 4:53 PM, Olaf Wagner wrote: On Mon, 27 Jun 2016 06:39:12 +0000 Jay K wrote: - I basically understand licensing. - I understand GPL - I understand more liberal BSD license - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. What I do not understand: - dual licensing - who owns parse.c - can parse.c be dual licensed? In particular: jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ m3-parse.h parse.c m3-def.h m3cg.h Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly in-process ones, either call-based or "linearized IR in memory". In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. m3cg.h is output by m3cggen. m3-def.h I wrote. These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. Can we also BSD license them or such? (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) My understand is that you can put any license on things you wrote yourself. I'm not really sure, but I doubt that there is any legal entity left that cares for the M3 sources from DEC SRC (if it is that old). So I _think_ that we (you) might change the copyright for those. To be more compatible with the GNU stuff, it might be better to use LGPL together with the gcc backend. I am not a lawyer though. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 27 10:37:12 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 08:37:12 +0000 Subject: [M3devel] getting all registers in gc? Message-ID: I just noticed this in the Boehm GC documentation: ?- Changed the alpha port to use the generic register scanning code instead ? ?of alpha_mach_dep.s. ?Alpha_mach_dep.s doesn't look for pointers in fp ? ?registers, but gcc sometimes spills pointers there. ?(Thanks to Manuel ? ?Serrano for helping me debug this by email.) ?Changed the IA64 code to ? ?do something similar for similar reasons. This would seem like a hazard for us too. And not convincingly Alpha/IA64-specific. We basically assume setjmp stores a context, or at least all live pointers. In hindsight I see two problems: ?- one alluded to -- jmpbuf might not have floating point registers, ? ?and floating point registers might have pointers. ?- Same thing but more general: jmpbuf might not even have all integer ? ?registers? ? ? So that leaves the question "What is generic register scanning code"? I don't know yet but..thinking... ?Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? ? ?I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, ?as I think that is a lingering thing we need to handle to finish our portability. Ignoring IA64 for now, maybe here: void __cdecl ThreadPThread__sigsuspend(void) { ? struct { ? ? sigjmp_buf jb; ? } s; ? ZERO_MEMORY(s); ? if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ #ifdef M3_REGISTER_WINDOWS ? ? siglongjmp(s.jb, 1); /* flush register windows */ ? else #endif ? ? sigsuspend(&mask); } ? ? ?and here: ? ?void __cdecl ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) { ? struct { ? ? sigjmp_buf jb; ? } s; ? ZERO_MEMORY(s); ? if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ we should use getcontext/RtlCaptureContext/GetThreadContext? I'll look more at the Boehm code. ?- Jay ? From jay.krell at cornell.edu Mon Jun 27 11:14:13 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 09:14:13 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: References: Message-ID: ok, the Boehm code: For the current live thread, merely: ? ? ? ? ? ? ? ? ? ? ? ? /* Push enough of the current stack eagerly to ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* ensure that callee-save registers saved in ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* GC frames are scanned. ? ? ? ? ? ? ? ? ? ? ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* In the non-threads case, schedule entire ? ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* stack for scanning. ? ? ? ? ? ? ? ? ? ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* The second argument is a pointer to the ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* (possibly null) thread context, for ? ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* (currently hypothetical) more precise ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* stack scanning. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ /* ?* In the absence of threads, push the stack contents. ?* In the presence of threads, push enough of the current stack ?* to ensure that callee-save registers saved in collector frames have been ?* seen. ?* FIXME: Merge with per-thread stuff. ?*/ /*ARGSUSED*/ STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) { # ? if defined(THREADS) ? ? ? ? if (0 == cold_gc_frame) return; # ? ? ? ifdef STACK_GROWS_DOWN ? ? ? ? ? GC_push_all_eager(GC_approx_sp(), cold_gc_frame); ? ? ? ? ? /* For IA64, the register stack backing store is handled ? ? ?*/ ? ? ? ? ? /* in the thread-specific code. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? */ # ? ? ? else ? ? ? ? ? GC_push_all_eager(cold_gc_frame, GC_approx_sp()); # ? ? ? endif # ? else ... # ? endif /* !THREADS */ GC_INNER ptr_t GC_approx_sp(void) { ? ? volatile word sp; ? ? sp = (word)&sp; ? ? ? ? ? ? ? ? /* Also force stack to grow if necessary. Otherwise the */ ? ? ? ? ? ? ? ? /* later accesses might cause the kernel to think we're */ ? ? ? ? ? ? ? ? /* doing something wrong. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? */ ? ? return((ptr_t)sp); ? ? ? ? ? ? ? ? /* GNU C: alternatively, we may return the value of ? ? */ ? ? ? ? ? ? ? ? /*__builtin_frame_address(0). ? ? ? ? ? ? ? ? ? ? ? ? ? */ } Notice that it doesn't even do what it says -- no attempt to save registers to stack. but for suspended threads it is more convincing: /* Ensure that either registers are pushed, or callee-save registers ? ?*/ /* are somewhere on the stack, and then call fn(arg, ctxt). ? ? ? ? ? ? */ /* ctxt is either a pointer to a ucontext_t we generated, or NULL. ? ? ?*/ GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ptr_t arg) { ? ? volatile int dummy; ? ? void * context = 0; .. .... ?a mix of methods:? ? - sometimes processor specific assembly? ? - sometimes getcontext, and then workaround a bug on Linux/amd64? ? - and then _setjmp on Unix? ? - setjmp on Windows? ?getcontext to me seems more promising than setjmp, ?and you can use both ? ? for Win32, I suggest RtlCaptureContext (for live thread too) ?? ?? ? We maybe should copy getcontext from various BSDs? ? i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need ? to worry about the glibc getcontext bug). or maybe just getcontext. The gradually expanding register set on x86 makes me nervous that this isn't a maintenance problem, but I'm guessing you never get pointers spilled to the ymm/zmm registers. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Mon, 27 Jun 2016 08:37:12 +0000 > Subject: [M3devel] getting all registers in gc? > > I just noticed this in the Boehm GC documentation: > > - Changed the alpha port to use the generic register scanning code instead > of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp > registers, but gcc sometimes spills pointers there. (Thanks to Manuel > Serrano for helping me debug this by email.) Changed the IA64 code to > do something similar for similar reasons. > > > This would seem like a hazard for us too. > > And not convincingly Alpha/IA64-specific. > > We basically assume setjmp stores a context, or at least all live pointers. > > > In hindsight I see two problems: > - one alluded to -- jmpbuf might not have floating point registers, > and floating point registers might have pointers. > > > - Same thing but more general: jmpbuf might not even have all integer > registers? > > > > So that leaves the question "What is generic register scanning code"? > > I don't know yet but..thinking... > > Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? > > I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, > as I think that is a lingering thing we need to handle to finish our portability. > > Ignoring IA64 for now, maybe here: > > void > __cdecl > ThreadPThread__sigsuspend(void) > { > struct { > sigjmp_buf jb; > } s; > > ZERO_MEMORY(s); > > if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ > #ifdef M3_REGISTER_WINDOWS > siglongjmp(s.jb, 1); /* flush register windows */ > else > #endif > sigsuspend(&mask); > } > > > and here: > > void > __cdecl > ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) > { > struct { > sigjmp_buf jb; > } s; > > ZERO_MEMORY(s); > > if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ > > > we should use getcontext/RtlCaptureContext/GetThreadContext? > > > I'll look more at the Boehm code. > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Mon Jun 27 16:59:20 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 10:59:20 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: <20160627145920.GA10803@topoi.pooq.com> On Mon, Jun 27, 2016 at 06:31:47AM +0000, Jay K wrote: > I know everyone here doesn't really like C/C++ (except me). > And, more significantly, I know the system written in itself > is a great test case, but I wonder if we shouldn't write a new > "real" frontend in C or C++, and see if we can't merge that > upstream with gcc and/or clang. Replacing the entire front end with a new one, written from scratch, and not just a machine-translation or a hand-translation of the old one, would enable us to deal with the licincing issues about the M3 compiler. Then only the libraries will be stuck with the wrong licence. OCaml might also be a good language in which to write a new compiler. > > It also worth mentioning that I believe gcc's Ada front end > is written in Ada -- you don't actually have to write > the frontend in C/C++ to merge upstream. > > But there might remain licensing concern. -- hendrik From hendrik at topoi.pooq.com Mon Jun 27 17:35:12 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 11:35:12 -0400 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: Message-ID: <20160627153512.GB10803@topoi.pooq.com> On Mon, Jun 27, 2016 at 06:39:12AM +0000, Jay K wrote: > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. > > > What I do not understand: > - dual licensing Here's how dual licensing works. If you release a work and grant several public licences to it, the public can use it under any of tthe licences that have been granted to it. So if you've granted rights under license A and under license B, someone for who doesn't like th rerms of licence A can if she chooses, use the rights granted under license B instead. Don't know what's clear about that. nIn particular, I believe any new components of CM3 should be licensed under both the existing licence and the GPL or LGPL, and perhaps under MIT as well. Then gradually, as we write new components, the entire ecosystem will become more and more compatible with the commonly used free licenses. It isi the copyright owner who determines what license(s) to release stuff under. If you write it, you own the copyright, unless it is work-for-hire, in which case your employer does. We do not own the copyright of the SRC Modula 3 system, so we cannot relicence it. What we can do, given enough time, is write a new one. -- hendrik > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? The copyright owner can do this. The copyright owner can license them under as many licences as he wishes. No one else can. -- hendrik From jay.krell at cornell.edu Mon Jun 27 22:00:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:00:06 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: Can the owner relicense parse.c, or it is stuck with GPL because it links to gcc? And then -- who owns it? Digital and Critical Mass? ?- Jay ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: wagner at elegosoft.com; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 09:51:31 +0000 > > parse.c is contaminated by GPL. > > On 27 Jun 2016, at 6:16 PM, Jay K > > wrote: > > I'm going to read Olaf's assertion as 1 "go from GPL to LGPL", not 2 > "go from DEC to LGPL". > > > Imho we should use the OpenBSD or FreeBSD or NetBSD licenses. > > > The OpenBSD folks are..funny but right-seeming. > Their take for example on the Apache 2.0 license is -- why another long > license? They'd need to pay a lawyer to allow for it. So just don't > allow for it. They have stayed back with Apache 1.x for this reason. > > > So you should reuse an existing short license. > > > So, sorry, another question I forgot to ask -- who owns parse.c? > Can we relicense it? > > > And, ok, I own m3-def.h. I can just paste two license into it? > I'll research the old Qt story here I guess or otherwise research. > > There is also a claim that the FreeBSD license is GPL-compatible, which > implies we can use it on parse.c/m3-def.h -- just a single license. > That is clearer to me. I just understand what it means to have two > licences, unless, e.g. the license is context-dependent -- different > people get different license depending on situation, like if they paid, > or if they are getting paid. > > > I think I was going to use m3-def.h/parse.c in the C backend, writing > it in C or C++, and the intervening layer that allows easily writing > multiple passes over the IR. The result instead was incredibly tedious > and makes changing the IR more difficult/tedious. > If I embark on an LLVM backend, I'll again be tempted to do that. > > > It would be nice if we could relicense all the DEC SRC stuff as > slightly more liberal BSD. > I see DEC SRC seems to have an optional give back clause -- if you give > your changes back to DEC, then you also license it to them liberally. > But give back doesn't seem mandatory. > > 4. Improvements. LICENSEE hereby grants to DIGITAL a non-exclusive, > non-transferable, royalty free right to use, modify, reproduce > and distribute with the right to sublicense at any tier, any > improvements, enhancements, extensions, or modifications that > LICENSEE make to SOFTWARE, provided such are returned to DIGITAL > by LICENSEE. > > > - Jay > > > ________________________________ > From: hosking at purdue.edu > To: wagner at elegosoft.com > CC: jay.krell at cornell.edu; > m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 06:55:17 +0000 > > Please be careful here. > Going from the current license to LGPL is probably not the best route > for CM3! > > On 27 Jun 2016, at 4:53 PM, Olaf Wagner > > wrote: > > On Mon, 27 Jun 2016 06:39:12 +0000 > Jay K > wrote: > > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but > everyone seems to define it "like how static libraries work", "maybe > with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL > licensed, linked to other GPL licensed code, and does not "link" to > cm3, so does not infect the cm3 runtime, so does not infect all the > Modula-3 code. > > > What I do not understand: > - dual licensing > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like > the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in > non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by > parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? > > (and broken record, but m3-def.h...we could really use some sort of > preprocessor for Modula-3, maybe...this form of C/C++ is super > useful...) > > My understand is that you can put any license on things you wrote yourself. > I'm not really sure, but I doubt that there is any legal entity left that > cares for the M3 sources from DEC SRC (if it is that old). So I _think_ > that we (you) might change the copyright for those. > > To be more compatible with the GNU stuff, it might be better to use > LGPL together with the gcc backend. > > I am not a lawyer though. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From jay.krell at cornell.edu Mon Jun 27 22:04:27 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:04:27 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu> References: , , <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu> Message-ID: How do we ensure that? What about other backends, e.g. C or LLVM? The Boehm collector code implies the situation is under control, if we are like it, at least. I think getcontext will go far toward providing peace of mind on many platforms (and RtlCaptureContext on NT). I might still be convinced that setjmp suffices. In that, compiler will spill volatiles ahead of it anyway. I wonder if that suffices, but I haven't convinced myself. ?- Jay ---------------------------------------- > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] getting all registers in gc? > Date: Mon, 27 Jun 2016 09:50:06 +0000 > > A thought: can we simply ensure that the gcc-based backend never spills pointers to FP regs? > >> On 27 Jun 2016, at 7:14 PM, Jay K wrote: >> >> ok, the Boehm code: >> >> For the current live thread, merely: >> >> /* Push enough of the current stack eagerly to */ >> /* ensure that callee-save registers saved in */ >> /* GC frames are scanned. */ >> /* In the non-threads case, schedule entire */ >> /* stack for scanning. */ >> /* The second argument is a pointer to the */ >> /* (possibly null) thread context, for */ >> /* (currently hypothetical) more precise */ >> /* stack scanning. */ >> /* >> * In the absence of threads, push the stack contents. >> * In the presence of threads, push enough of the current stack >> * to ensure that callee-save registers saved in collector frames have been >> * seen. >> * FIXME: Merge with per-thread stuff. >> */ >> /*ARGSUSED*/ >> STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) >> { >> # if defined(THREADS) >> if (0 == cold_gc_frame) return; >> # ifdef STACK_GROWS_DOWN >> GC_push_all_eager(GC_approx_sp(), cold_gc_frame); >> /* For IA64, the register stack backing store is handled */ >> /* in the thread-specific code. */ >> # else >> GC_push_all_eager(cold_gc_frame, GC_approx_sp()); >> # endif >> # else >> ... >> # endif /* !THREADS */ >> >> >> GC_INNER ptr_t GC_approx_sp(void) >> { >> volatile word sp; >> sp = (word)&sp; >> /* Also force stack to grow if necessary. Otherwise the */ >> /* later accesses might cause the kernel to think we're */ >> /* doing something wrong. */ >> return((ptr_t)sp); >> /* GNU C: alternatively, we may return the value of */ >> /*__builtin_frame_address(0). */ >> } >> >> >> Notice that it doesn't even do what it says -- no attempt >> to save registers to stack. >> >> >> but for suspended threads it is more convincing: >> >> >> /* Ensure that either registers are pushed, or callee-save registers */ >> /* are somewhere on the stack, and then call fn(arg, ctxt). */ >> /* ctxt is either a pointer to a ucontext_t we generated, or NULL. */ >> GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), >> ptr_t arg) >> { >> volatile int dummy; >> void * context = 0; >> >> >> .. >> .... >> >> >> a mix of methods: >> - sometimes processor specific assembly >> - sometimes getcontext, and then workaround a bug on Linux/amd64 >> - and then _setjmp on Unix >> - setjmp on Windows >> >> >> getcontext to me seems more promising than setjmp, >> and you can use both >> >> for Win32, I suggest RtlCaptureContext (for live thread too) >> >> >> We maybe should copy getcontext from various BSDs? >> i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need >> to worry about the glibc getcontext bug). >> >> or maybe just getcontext. The gradually expanding register set on x86 makes me nervous >> that this isn't a maintenance problem, but I'm guessing you never get pointers >> spilled to the ymm/zmm registers. >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Date: Mon, 27 Jun 2016 08:37:12 +0000 >>> Subject: [M3devel] getting all registers in gc? >>> >>> I just noticed this in the Boehm GC documentation: >>> >>> - Changed the alpha port to use the generic register scanning code instead >>> of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp >>> registers, but gcc sometimes spills pointers there. (Thanks to Manuel >>> Serrano for helping me debug this by email.) Changed the IA64 code to >>> do something similar for similar reasons. >>> >>> >>> This would seem like a hazard for us too. >>> >>> And not convincingly Alpha/IA64-specific. >>> >>> We basically assume setjmp stores a context, or at least all live pointers. >>> >>> >>> In hindsight I see two problems: >>> - one alluded to -- jmpbuf might not have floating point registers, >>> and floating point registers might have pointers. >>> >>> >>> - Same thing but more general: jmpbuf might not even have all integer >>> registers? >>> >>> >>> >>> So that leaves the question "What is generic register scanning code"? >>> >>> I don't know yet but..thinking... >>> >>> Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? >>> >>> I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, >>> as I think that is a lingering thing we need to handle to finish our portability. >>> >>> Ignoring IA64 for now, maybe here: >>> >>> void >>> __cdecl >>> ThreadPThread__sigsuspend(void) >>> { >>> struct { >>> sigjmp_buf jb; >>> } s; >>> >>> ZERO_MEMORY(s); >>> >>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>> #ifdef M3_REGISTER_WINDOWS >>> siglongjmp(s.jb, 1); /* flush register windows */ >>> else >>> #endif >>> sigsuspend(&mask); >>> } >>> >>> >>> and here: >>> >>> void >>> __cdecl >>> ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) >>> { >>> struct { >>> sigjmp_buf jb; >>> } s; >>> >>> ZERO_MEMORY(s); >>> >>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>> >>> >>> we should use getcontext/RtlCaptureContext/GetThreadContext? >>> >>> >>> I'll look more at the Boehm code. >>> >>> >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From lemming at henning-thielemann.de Mon Jun 27 22:17:38 2016 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 27 Jun 2016 22:17:38 +0200 (CEST) Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: On Mon, 27 Jun 2016, Jay K wrote: > Can the owner relicense parse.c, Sure, the owner can always relicense. > or it is stuck with GPL because it links to gcc? He can choose any license that is compatible with GPL. From jay.krell at cornell.edu Mon Jun 27 22:19:35 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:19:35 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , Message-ID: Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). But parse.c might be ownerless and stuck? ? - Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 22:17:38 +0200 > From: lemming at henning-thielemann.de > To: jay.krell at cornell.edu > CC: hosking at purdue.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > > > On Mon, 27 Jun 2016, Jay K wrote: > >> Can the owner relicense parse.c, > > Sure, the owner can always relicense. > >> or it is stuck with GPL because it links to gcc? > > He can choose any license that is compatible with GPL. From rodney_bates at lcwb.coop Mon Jun 27 22:19:30 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 15:19:30 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: <57718A52.3030609@lcwb.coop> On 06/27/2016 01:31 AM, Jay K wrote: > redirecting... > > > Olaf > >> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl > My big lament about llvm regards producing Dwarf debug info. Having debug info in this vastly superior format is one of my main reasons for wanting an llvm back end. From the llvm documentation, it sounded like it would do this, including altering debug info as needed to match any optimizations done. Then I hit a brick wall, finding out llvm only handles a severe subset of Dwarf, apparently what they felt was needed for C & C++. More below. > > I would like to take this up, maybe soon. > > > I do have a bit of an agenda. > Maybe my priorities are mixed up. > > > 1 Provide a very portable system. > 2 Provide an easy to install and use system. > 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least > for their supported backends). > 4 Maybe write our own backends. > > > Where is the LLVM support at? > Mostly working? Barely working? I could swear I remember getting the llvm back end to recompile everything in the m3front group twice and converge to identical-sized compiled code, and that I said so in either a commit message or an m3devel list post. But I couldn't find any such statement. This is on AMD64_LINUX, and if you are careful not to ask m3llvm for debug info at all. Otherwise, there are crashes in m3llvm and/or llc, trying to provide Dwarf. This is what I was working on when I made the disappointing discoveries. If my memory is correct here, this means the llvm back end is close to usable for building. Also, again from memory, I think the llvm back end has no failures of test cases that do not also fail using m3cg. There are still other questions, though. So far, there are no changes required to the stock llvm version being used (3.6.1) But it still requires a build of that on the machine where the M3 compilation is done, and it is big and slow to build. Lots of llvm libraries have to be linked in to m3llvm, and separate executable llc needs to be available. The llvm folk are constantly making major API changes, with no explanation other than diffing successive versions of header files, so there is no practical chance of using whatever llvm version may be already installed on a particular host machine. In light of that, I suppose we could fork and modify a specific version of llvm and put its source code in our own repository, similar to m3cg. But maintenance work on llvm is, to my standards, a real nightmare. Just about every single identifier and operator you see involves a needle-in-haystack search to locate its declaration, which could be just about anywhere, in order to know what it is. And no, the names and operator spellings are not close to adequate to clue you in. They have gone to every length possible to use every clever new C++ "feature" that comes out in the latest C++- standard, which always just increases the complexity of the search to a declaration. So I don't fancy doing any of this. (BTW, =17 in recent discussions.) As for the debug info problem, there has been talk in the llvm list of a rework that allows a front end to directly produce true Dwarf (presumably not a subset) for type info only, which would not be changed by optimizations and so llvm would not need to pass it through in its own, different, representation. I don't know how far this has gone. If/when it happens, utilizing it would entail constructing new bindings to altered llvm APIs. This in itself is an extremely tedious and error-prone task that I hoped, after doing it for llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, there is no type checking possible, so nitty little errors will only show up as hard-to-diagnose runtime errors. That is made even worse by the lack of a debugger that handles both languages. Or, we could use Dragisha's rtinfo library to get type info for a debugger from the .M3WEB files, and Dwarf for the rest. Off hand, this sounds like the best approach to me. Stick with llvm 3.6.1 as long a possible, and avoid more pain. > > > I know LLVM is big and changing, and maybe they don't value > compatibility of bitcode. Actually, keeping their bitcode stable across llvm releases is one place they do talk about compatibility. But m3llvm uses calls to llvm APIs to construct llvm IR as in-memory data, then another call to get llvm to convert it to bitcode. So bitcode's stability is irrelevant to us. I once thought about producing llvm bitcode directly, but that seems like a pretty big job. It would, however, obviate creating most of those wretched bindings. > > > But look at what we have with the gcc backend. > Even if we didn't have to patch it at all, I expect > we'd still have to keep and build a local copy. > > Perhaps we should just do that? > > With LLVM, with its different licensing, perhaps we could > get our "frontend" merged upstream, but this would > then give us a compatibility burden in the persisted m3cg. > Is that ok? > It is hypothetical at this point. > > > I know everyone here doesn't really like C/C++ (except me). > And, more significantly, I know the system written in itself > is a great test case, but I wonder if we shouldn't write a new > "real" frontend in C or C++, and see if we can't merge that > upstream with gcc and/or clang. > > > It also worth mentioning that I believe gcc's Ada front end > is written in Ada -- you don't actually have to write > the frontend in C/C++ to merge upstream. Yes, I once worked on this front end. As I understood, SRC M3 was not merged into gcc only because RMS was annoyed that SRC made it a separate executable, so its license was not poisoned by GPL. > > But there might remain licensing concern. > > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 27 22:26:39 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 15:26:39 -0500 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , Message-ID: <57718BFF.1070501@lcwb.coop> On 06/27/2016 03:19 PM, Jay K wrote: > Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). > > But parse.c might be ownerless and stuck? Isn't there some law that if more than 25% of a work is changed, it ceases to be a derived work but becomes a new work by the changer? If so, I would guess the current parse.c would meet this criterion, and I think Jay would own it, since I think he has done the majority of the changes. IANAL. > > - Jay > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 22:17:38 +0200 >> From: lemming at henning-thielemann.de >> To: jay.krell at cornell.edu >> CC: hosking at purdue.edu; m3devel at elegosoft.com >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> >> On Mon, 27 Jun 2016, Jay K wrote: >> >>> Can the owner relicense parse.c, >> >> Sure, the owner can always relicense. >> >>> or it is stuck with GPL because it links to gcc? >> >> He can choose any license that is compatible with GPL. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 27 22:28:57 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:28:57 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> Message-ID: Regarding debug info. I have thought about this. Various optimizing compilers make some effort to produce some debug info. And as much as they don't optimize, they should do that. However I believe in general, optimization and describability with any debug format, is not a solvable problem. There are many points in a program where things aren't particularly transformed/lost by the optimizer. Things tend to be clearly defined at the start of a function. Variable locations can be described over ranges of code as either being in a particular register, or register + offset (frame), or nowhere. But line numbers, as critical as they are, I think are the most untenable aspect. Compilers can move code around arbitrarily, including removing it. The C backend also has good debugging as a goal. We should be able to have structs with fields, instead of just byte arrays. And all the parameters/locals should have good names. ? This I messed up slightly and will try to fix. Every identifier ? gets an appended number for uniqueness, which is sometimes but rarely needed. ?? Does the current LLVM backend produce any debug info or none? I probably avoid any approach that requires writing bindings. Unless it is to our own code, like I did for Posix. Otherwise keeping things up to date is tedious and error prone and errors are fatal. To our own code is just slightly better. Maybe I'll write bitcode. Or go through the persisted cm3cg IR that m3cc uses, but it sounds like you are super far along and we should build on that. I have other things to do first though and I can't promise that I won't reinvent eventually. ?- Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 15:19:30 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 01:31 AM, Jay K wrote: >> redirecting... >> >>> Olaf >> >>> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl >> > > My big lament about llvm regards producing Dwarf debug info. Having > debug info in this vastly superior format is one of my main reasons > for wanting an llvm back end. From the llvm documentation, it sounded > like it would do this, including altering debug info as needed to > match any optimizations done. Then I hit a brick wall, finding out > llvm only handles a severe subset of Dwarf, apparently what they felt > was needed for C & C++. More below. > >> >> I would like to take this up, maybe soon. >> >> >> I do have a bit of an agenda. >> Maybe my priorities are mixed up. >> >> >> 1 Provide a very portable system. >> 2 Provide an easy to install and use system. >> 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least >> for their supported backends). >> 4 Maybe write our own backends. >> >> >> Where is the LLVM support at? >> Mostly working? Barely working? > > I could swear I remember getting the llvm back end to recompile > everything in the m3front group twice and converge to identical-sized > compiled code, and that I said so in either a commit message or > an m3devel list post. But I couldn't find any such statement. > > This is on AMD64_LINUX, and if you are careful not to ask m3llvm > for debug info at all. Otherwise, there are crashes in m3llvm > and/or llc, trying to provide Dwarf. This is what I was working > on when I made the disappointing discoveries. If my memory is > correct here, this means the llvm back end is close to usable > for building. > > Also, again from memory, I think the llvm back end has no failures > of test cases that do not also fail using m3cg. > > There are still other questions, though. So far, there are no > changes required to the stock llvm version being used (3.6.1) > But it still requires a build of that on the machine where the > M3 compilation is done, and it is big and slow to build. Lots > of llvm libraries have to be linked in to m3llvm, and separate > executable llc needs to be available. > > The llvm folk are constantly making major API changes, with no > explanation other than diffing successive versions of header > files, so there is no practical chance of using whatever llvm > version may be already installed on a particular host machine. > > In light of that, I suppose we could fork and modify a specific > version of llvm and put its source code in our own repository, > similar to m3cg. But maintenance work on llvm is, to my > standards, a real nightmare. Just about every single identifier > and operator you see involves a needle-in-haystack search to > locate its declaration, which could be just about anywhere, > in order to know what it is. > > And no, the names and operator spellings are not close to adequate > to clue you in. They have gone to every length possible to use > every clever new C++ "feature" that comes out in the latest > C++- standard, which always just increases the complexity > of the search to a declaration. So I don't fancy doing any of > this. (BTW, =17 in recent discussions.) > > As for the debug info problem, there has been talk in the llvm > list of a rework that allows a front end to directly produce > true Dwarf (presumably not a subset) for type info only, which > would not be changed by optimizations and so llvm would not > need to pass it through in its own, different, representation. > I don't know how far this has gone. > > If/when it happens, utilizing it would entail constructing new > bindings to altered llvm APIs. This in itself is an extremely > tedious and error-prone task that I hoped, after doing it for > llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, > there is no type checking possible, so nitty little errors will > only show up as hard-to-diagnose runtime errors. That is made > even worse by the lack of a debugger that handles both languages. > > Or, we could use Dragisha's rtinfo library to get type info for > a debugger from the .M3WEB files, and Dwarf for the rest. Off > hand, this sounds like the best approach to me. Stick with > llvm 3.6.1 as long a possible, and avoid more pain. > >> >> >> I know LLVM is big and changing, and maybe they don't value >> compatibility of bitcode. > > Actually, keeping their bitcode stable across llvm releases is > one place they do talk about compatibility. But m3llvm uses calls > to llvm APIs to construct llvm IR as in-memory data, then another > call to get llvm to convert it to bitcode. So bitcode's stability > is irrelevant to us. I once thought about producing llvm bitcode > directly, but that seems like a pretty big job. It would, however, > obviate creating most of those wretched bindings. > >> >> >> But look at what we have with the gcc backend. >> Even if we didn't have to patch it at all, I expect >> we'd still have to keep and build a local copy. >> >> Perhaps we should just do that? >> >> With LLVM, with its different licensing, perhaps we could >> get our "frontend" merged upstream, but this would >> then give us a compatibility burden in the persisted m3cg. >> Is that ok? >> It is hypothetical at this point. >> >> >> I know everyone here doesn't really like C/C++ (except me). >> And, more significantly, I know the system written in itself >> is a great test case, but I wonder if we shouldn't write a new >> "real" frontend in C or C++, and see if we can't merge that >> upstream with gcc and/or clang. >> >> >> It also worth mentioning that I believe gcc's Ada front end >> is written in Ada -- you don't actually have to write >> the frontend in C/C++ to merge upstream. > > Yes, I once worked on this front end. As I understood, SRC M3 > was not merged into gcc only because RMS was annoyed that SRC made > it a separate executable, so its license was not poisoned by GPL. > >> >> But there might remain licensing concern. >> >> >> >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From lemming at henning-thielemann.de Mon Jun 27 22:29:08 2016 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 27 Jun 2016 22:29:08 +0200 (CEST) Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: On Mon, 27 Jun 2016, Rodney M. Bates wrote: > And no, the names and operator spellings are not close to adequate > to clue you in. They have gone to every length possible to use > every clever new C++ "feature" that comes out in the latest > C++- standard, which always just increases the complexity > of the search to a declaration. So I don't fancy doing any of > this. (BTW, =17 in recent discussions.) A teacher of mine called this behavior "version junkie". > Actually, keeping their bitcode stable across llvm releases is > one place they do talk about compatibility. But m3llvm uses calls > to llvm APIs to construct llvm IR as in-memory data, then another > call to get llvm to convert it to bitcode. So bitcode's stability > is irrelevant to us. I once thought about producing llvm bitcode > directly, but that seems like a pretty big job. It would, however, > obviate creating most of those wretched bindings. An alternative would be to create .ll text files. But its format changes, too. From jay.krell at cornell.edu Mon Jun 27 22:30:57 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:30:57 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <57718BFF.1070501@lcwb.coop> References: , ,,<20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, ,,<653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, ,,, , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , , , , <57718BFF.1070501@lcwb.coop> Message-ID: Some number should be reasonable yes, but I think you exaggerate how much I've done. :) I'll look later. ?- Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 15:26:39 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > > > > On 06/27/2016 03:19 PM, Jay K wrote: >> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >> >> But parse.c might be ownerless and stuck? > > Isn't there some law that if more than 25% of a work is changed, it ceases > to be a derived work but becomes a new work by the changer? If so, I would > guess the current parse.c would meet this criterion, and I think Jay would > own it, since I think he has done the majority of the changes. > > IANAL. > >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>> From: lemming at henning-thielemann.de >>> To: jay.krell at cornell.edu >>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>> Subject: Re: [M3devel] parse.c licensing question, dual? >>> >>> >>> On Mon, 27 Jun 2016, Jay K wrote: >>> >>>> Can the owner relicense parse.c, >>> >>> Sure, the owner can always relicense. >>> >>>> or it is stuck with GPL because it links to gcc? >>> >>> He can choose any license that is compatible with GPL. >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Mon Jun 27 22:32:29 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:32:29 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: References: , , <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu>, Message-ID: ?> I think getcontext will go far toward providing peace of mind on many platform That is surprisingly a syscall where I've now looked. Surprising and slightly bad. In the name of thread suspension, it might be affordable. Still might copy that underlying code. We need to solve this even with cooperative suspend btw -- which I really want... ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] getting all registers in gc? > Date: Mon, 27 Jun 2016 20:04:27 +0000 > > How do we ensure that? > > What about other backends, e.g. C or LLVM? > > The Boehm collector code implies the situation is under control, if we are like it, at least. > > I think getcontext will go far toward providing peace of mind on many platforms (and RtlCaptureContext > on NT). > > I might still be convinced that setjmp suffices. > In that, compiler will spill volatiles ahead of it anyway. > I wonder if that suffices, but I haven't convinced myself. > > - Jay > > ---------------------------------------- >> From: hosking at purdue.edu >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] getting all registers in gc? >> Date: Mon, 27 Jun 2016 09:50:06 +0000 >> >> A thought: can we simply ensure that the gcc-based backend never spills pointers to FP regs? >> >>> On 27 Jun 2016, at 7:14 PM, Jay K wrote: >>> >>> ok, the Boehm code: >>> >>> For the current live thread, merely: >>> >>> /* Push enough of the current stack eagerly to */ >>> /* ensure that callee-save registers saved in */ >>> /* GC frames are scanned. */ >>> /* In the non-threads case, schedule entire */ >>> /* stack for scanning. */ >>> /* The second argument is a pointer to the */ >>> /* (possibly null) thread context, for */ >>> /* (currently hypothetical) more precise */ >>> /* stack scanning. */ >>> /* >>> * In the absence of threads, push the stack contents. >>> * In the presence of threads, push enough of the current stack >>> * to ensure that callee-save registers saved in collector frames have been >>> * seen. >>> * FIXME: Merge with per-thread stuff. >>> */ >>> /*ARGSUSED*/ >>> STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) >>> { >>> # if defined(THREADS) >>> if (0 == cold_gc_frame) return; >>> # ifdef STACK_GROWS_DOWN >>> GC_push_all_eager(GC_approx_sp(), cold_gc_frame); >>> /* For IA64, the register stack backing store is handled */ >>> /* in the thread-specific code. */ >>> # else >>> GC_push_all_eager(cold_gc_frame, GC_approx_sp()); >>> # endif >>> # else >>> ... >>> # endif /* !THREADS */ >>> >>> >>> GC_INNER ptr_t GC_approx_sp(void) >>> { >>> volatile word sp; >>> sp = (word)&sp; >>> /* Also force stack to grow if necessary. Otherwise the */ >>> /* later accesses might cause the kernel to think we're */ >>> /* doing something wrong. */ >>> return((ptr_t)sp); >>> /* GNU C: alternatively, we may return the value of */ >>> /*__builtin_frame_address(0). */ >>> } >>> >>> >>> Notice that it doesn't even do what it says -- no attempt >>> to save registers to stack. >>> >>> >>> but for suspended threads it is more convincing: >>> >>> >>> /* Ensure that either registers are pushed, or callee-save registers */ >>> /* are somewhere on the stack, and then call fn(arg, ctxt). */ >>> /* ctxt is either a pointer to a ucontext_t we generated, or NULL. */ >>> GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), >>> ptr_t arg) >>> { >>> volatile int dummy; >>> void * context = 0; >>> >>> >>> .. >>> .... >>> >>> >>> a mix of methods: >>> - sometimes processor specific assembly >>> - sometimes getcontext, and then workaround a bug on Linux/amd64 >>> - and then _setjmp on Unix >>> - setjmp on Windows >>> >>> >>> getcontext to me seems more promising than setjmp, >>> and you can use both >>> >>> for Win32, I suggest RtlCaptureContext (for live thread too) >>> >>> >>> We maybe should copy getcontext from various BSDs? >>> i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need >>> to worry about the glibc getcontext bug). >>> >>> or maybe just getcontext. The gradually expanding register set on x86 makes me nervous >>> that this isn't a maintenance problem, but I'm guessing you never get pointers >>> spilled to the ymm/zmm registers. >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Date: Mon, 27 Jun 2016 08:37:12 +0000 >>>> Subject: [M3devel] getting all registers in gc? >>>> >>>> I just noticed this in the Boehm GC documentation: >>>> >>>> - Changed the alpha port to use the generic register scanning code instead >>>> of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp >>>> registers, but gcc sometimes spills pointers there. (Thanks to Manuel >>>> Serrano for helping me debug this by email.) Changed the IA64 code to >>>> do something similar for similar reasons. >>>> >>>> >>>> This would seem like a hazard for us too. >>>> >>>> And not convincingly Alpha/IA64-specific. >>>> >>>> We basically assume setjmp stores a context, or at least all live pointers. >>>> >>>> >>>> In hindsight I see two problems: >>>> - one alluded to -- jmpbuf might not have floating point registers, >>>> and floating point registers might have pointers. >>>> >>>> >>>> - Same thing but more general: jmpbuf might not even have all integer >>>> registers? >>>> >>>> >>>> >>>> So that leaves the question "What is generic register scanning code"? >>>> >>>> I don't know yet but..thinking... >>>> >>>> Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? >>>> >>>> I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, >>>> as I think that is a lingering thing we need to handle to finish our portability. >>>> >>>> Ignoring IA64 for now, maybe here: >>>> >>>> void >>>> __cdecl >>>> ThreadPThread__sigsuspend(void) >>>> { >>>> struct { >>>> sigjmp_buf jb; >>>> } s; >>>> >>>> ZERO_MEMORY(s); >>>> >>>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>>> #ifdef M3_REGISTER_WINDOWS >>>> siglongjmp(s.jb, 1); /* flush register windows */ >>>> else >>>> #endif >>>> sigsuspend(&mask); >>>> } >>>> >>>> >>>> and here: >>>> >>>> void >>>> __cdecl >>>> ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) >>>> { >>>> struct { >>>> sigjmp_buf jb; >>>> } s; >>>> >>>> ZERO_MEMORY(s); >>>> >>>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>>> >>>> >>>> we should use getcontext/RtlCaptureContext/GetThreadContext? >>>> >>>> >>>> I'll look more at the Boehm code. >>>> >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > From jay.krell at cornell.edu Mon Jun 27 22:42:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:42:51 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , , , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , , , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, ,,, , , , , , <57718BFF.1070501@lcwb.coop>, Message-ID: Tony changed the license from DEC to FSF here (possibly someone else did in another branch) https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 There is this growth: jair:~ jay$ wc -l parse.c.current? ? ? 6516 parse.c.current jair:~ jay$ wc -l parse.c.old? ? ? 4190 parse.c.old though derivation is still hard to argue with. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Date: Mon, 27 Jun 2016 20:30:57 +0000 > Subject: Re: [M3devel] parse.c licensing question, dual? > > Some number should be reasonable yes, but I think you exaggerate how much I've done. :) > I'll look later. > > - Jay > > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 15:26:39 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> >> >> On 06/27/2016 03:19 PM, Jay K wrote: >>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>> >>> But parse.c might be ownerless and stuck? >> >> Isn't there some law that if more than 25% of a work is changed, it ceases >> to be a derived work but becomes a new work by the changer? If so, I would >> guess the current parse.c would meet this criterion, and I think Jay would >> own it, since I think he has done the majority of the changes. >> >> IANAL. >> >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>> From: lemming at henning-thielemann.de >>>> To: jay.krell at cornell.edu >>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>> >>>> >>>> On Mon, 27 Jun 2016, Jay K wrote: >>>> >>>>> Can the owner relicense parse.c, >>>> >>>> Sure, the owner can always relicense. >>>> >>>>> or it is stuck with GPL because it links to gcc? >>>> >>>> He can choose any license that is compatible with GPL. >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Mon Jun 27 23:44:13 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 17:44:13 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: <20160627214413.GA25148@topoi.pooq.com> On Mon, Jun 27, 2016 at 03:19:30PM -0500, Rodney M. Bates wrote: > > My big lament about llvm regards producing Dwarf debug info. Having > debug info in this vastly superior format is one of my main reasons > for wanting an llvm back end. From the llvm documentation, it sounded > like it would do this, including altering debug info as needed to > match any optimizations done. Then I hit a brick wall, finding out > llvm only handles a severe subset of Dwarf, apparently what they felt > was needed for C & C++. More below. llvm is an intermediate code for C and C++. Its address arithmetic is C/C++ arithmetic. Any usability for other languages is a happy accident, whatever they may say. -- hendrik From hendrik at topoi.pooq.com Mon Jun 27 23:48:49 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 17:48:49 -0400 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: <20160627085333.916aae7db6aa078d5300c739@elegosoft.com> <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: <20160627214849.GB25148@topoi.pooq.com> On Mon, Jun 27, 2016 at 10:17:38PM +0200, Henning Thielemann wrote: > > On Mon, 27 Jun 2016, Jay K wrote: > > >Can the owner relicense parse.c, > > Sure, the owner can always relicense. > > >or it is stuck with GPL because it links to gcc? > > He can choose any license that is compatible with GPL. And any other licence at al. But it would ony be directly useful if one of the licences is conpatible with GPL. -- hendrik k From jay.krell at cornell.edu Tue Jun 28 01:44:19 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 23:44:19 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160627214413.GA25148@topoi.pooq.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop>, <20160627214413.GA25148@topoi.pooq.com> Message-ID: gcc has these other frontends -- ada and java at least.If they are much used -- and I don't know if they are, but if they are, LLVM should expand its goals to replace them.Just my opinion.But either way, other people are using LLVM, like for OCaml and Rust I'm pretty sure.And GPU stuff.So it isn't just C/C++, but may be a happy accident, indeed. How much of gcc/dwarf is that way?I understand gcc does actually cater to Pascal/Modula-3?Ada some -- the multiple forms of divide, the nested functions (eventhough we have to patch gcc for nested functions, it does have some nested function support already, includinga certain optimization you won't see any any narrow C/C++ compiler -- ok, granted, they have nested functions in their C frontend.., but still, the divide stuff..) - Jay > Date: Mon, 27 Jun 2016 17:44:13 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Mon, Jun 27, 2016 at 03:19:30PM -0500, Rodney M. Bates wrote: > > > > My big lament about llvm regards producing Dwarf debug info. Having > > debug info in this vastly superior format is one of my main reasons > > for wanting an llvm back end. From the llvm documentation, it sounded > > like it would do this, including altering debug info as needed to > > match any optimizations done. Then I hit a brick wall, finding out > > llvm only handles a severe subset of Dwarf, apparently what they felt > > was needed for C & C++. More below. > > llvm is an intermediate code for C and C++. Its address arithmetic is > C/C++ arithmetic. Any usability for other languages is a happy > accident, whatever they may say. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 28 02:29:47 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 19:29:47 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> Message-ID: <5771C4FB.8040009@lcwb.coop> On 06/27/2016 03:28 PM, Jay K wrote: > Regarding debug info. > > > I have thought about this. > > > Various optimizing compilers make some effort to produce > some debug info. > > > And as much as they don't optimize, they should do that. > > > However I believe in general, optimization and describability > with any debug format, is not a solvable problem. > > > There are many points in a program where things aren't particularly > transformed/lost by the optimizer. > > > Things tend to be clearly defined at the start of a function. > Variable locations can be described over ranges of code as either > being in a particular register, or register + offset (frame), or nowhere. > > > But line numbers, as critical as they are, I think are the most > untenable aspect. Compilers can move code around arbitrarily, including removing it. > Yes, this is a bundle of hard problems. llvm's official goal is that, in the face of optimizations, it should still be possible for a debugger to observe all program state of the original program, but not necessarily alter any state. Without optimizations, it should also be possible to alter any state as well. How well they are achieving this, I don't know, but I would guess it comes closer than gcc, especially an old gcc. > > The C backend also has good debugging as a goal. > We should be able to have structs with fields, instead of just byte arrays. > > And all the parameters/locals should have good names. > This I messed up slightly and will try to fix. Every identifier > gets an appended number for uniqueness, which is sometimes but rarely needed. > > > Does the current LLVM backend produce any debug info or none? It has lots of code for producing debug info, probably as much code as for producing compilable output, maybe slightly more. I think it is pretty complete. But it has not had a lot of testing, and there are known cases where it has crashes. The tests expose several of these. I was working on this when I hit the problem I have complained about. If you don't ask for debug output, these don't happen. If I could decide what to do when llvm's debug info is inadequate, I would work on fixing them--probably not too big a job. > > I probably avoid any approach that requires writing bindings. > Unless it is to our own code, like I did for Posix. > Otherwise keeping things up to date is tedious and error prone and errors are fatal. > To our own code is just slightly better. > > Maybe I'll write bitcode. Or go through the persisted cm3cg IR that m3cc uses, but it sounds > like you are super far along and we should build on that. > I have other things to do first though and I can't promise that I won't reinvent eventually. > At this point, I think this might be nice to have. The existing m3llvm would be a good starting point. I think mostly, it would just need the calls on llvm APIs replaced by bitcode-emitting calls, plus the low-level things for them to call. > - Jay > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 15:19:30 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 llvm backend? >> >> >> >> On 06/27/2016 01:31 AM, Jay K wrote: >>> redirecting... >>> >>>> Olaf >>> >>>> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl >>> >> >> My big lament about llvm regards producing Dwarf debug info. Having >> debug info in this vastly superior format is one of my main reasons >> for wanting an llvm back end. From the llvm documentation, it sounded >> like it would do this, including altering debug info as needed to >> match any optimizations done. Then I hit a brick wall, finding out >> llvm only handles a severe subset of Dwarf, apparently what they felt >> was needed for C & C++. More below. >> >>> >>> I would like to take this up, maybe soon. >>> >>> >>> I do have a bit of an agenda. >>> Maybe my priorities are mixed up. >>> >>> >>> 1 Provide a very portable system. >>> 2 Provide an easy to install and use system. >>> 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least >>> for their supported backends). >>> 4 Maybe write our own backends. >>> >>> >>> Where is the LLVM support at? >>> Mostly working? Barely working? >> >> I could swear I remember getting the llvm back end to recompile >> everything in the m3front group twice and converge to identical-sized >> compiled code, and that I said so in either a commit message or >> an m3devel list post. But I couldn't find any such statement. >> >> This is on AMD64_LINUX, and if you are careful not to ask m3llvm >> for debug info at all. Otherwise, there are crashes in m3llvm >> and/or llc, trying to provide Dwarf. This is what I was working >> on when I made the disappointing discoveries. If my memory is >> correct here, this means the llvm back end is close to usable >> for building. >> >> Also, again from memory, I think the llvm back end has no failures >> of test cases that do not also fail using m3cg. >> >> There are still other questions, though. So far, there are no >> changes required to the stock llvm version being used (3.6.1) >> But it still requires a build of that on the machine where the >> M3 compilation is done, and it is big and slow to build. Lots >> of llvm libraries have to be linked in to m3llvm, and separate >> executable llc needs to be available. >> >> The llvm folk are constantly making major API changes, with no >> explanation other than diffing successive versions of header >> files, so there is no practical chance of using whatever llvm >> version may be already installed on a particular host machine. >> >> In light of that, I suppose we could fork and modify a specific >> version of llvm and put its source code in our own repository, >> similar to m3cg. But maintenance work on llvm is, to my >> standards, a real nightmare. Just about every single identifier >> and operator you see involves a needle-in-haystack search to >> locate its declaration, which could be just about anywhere, >> in order to know what it is. >> >> And no, the names and operator spellings are not close to adequate >> to clue you in. They have gone to every length possible to use >> every clever new C++ "feature" that comes out in the latest >> C++- standard, which always just increases the complexity >> of the search to a declaration. So I don't fancy doing any of >> this. (BTW, =17 in recent discussions.) >> >> As for the debug info problem, there has been talk in the llvm >> list of a rework that allows a front end to directly produce >> true Dwarf (presumably not a subset) for type info only, which >> would not be changed by optimizations and so llvm would not >> need to pass it through in its own, different, representation. >> I don't know how far this has gone. >> >> If/when it happens, utilizing it would entail constructing new >> bindings to altered llvm APIs. This in itself is an extremely >> tedious and error-prone task that I hoped, after doing it for >> llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, >> there is no type checking possible, so nitty little errors will >> only show up as hard-to-diagnose runtime errors. That is made >> even worse by the lack of a debugger that handles both languages. >> >> Or, we could use Dragisha's rtinfo library to get type info for >> a debugger from the .M3WEB files, and Dwarf for the rest. Off >> hand, this sounds like the best approach to me. Stick with >> llvm 3.6.1 as long a possible, and avoid more pain. >> >>> >>> >>> I know LLVM is big and changing, and maybe they don't value >>> compatibility of bitcode. >> >> Actually, keeping their bitcode stable across llvm releases is >> one place they do talk about compatibility. But m3llvm uses calls >> to llvm APIs to construct llvm IR as in-memory data, then another >> call to get llvm to convert it to bitcode. So bitcode's stability >> is irrelevant to us. I once thought about producing llvm bitcode >> directly, but that seems like a pretty big job. It would, however, >> obviate creating most of those wretched bindings. >> >>> >>> >>> But look at what we have with the gcc backend. >>> Even if we didn't have to patch it at all, I expect >>> we'd still have to keep and build a local copy. >>> >>> Perhaps we should just do that? >>> >>> With LLVM, with its different licensing, perhaps we could >>> get our "frontend" merged upstream, but this would >>> then give us a compatibility burden in the persisted m3cg. >>> Is that ok? >>> It is hypothetical at this point. >>> >>> >>> I know everyone here doesn't really like C/C++ (except me). >>> And, more significantly, I know the system written in itself >>> is a great test case, but I wonder if we shouldn't write a new >>> "real" frontend in C or C++, and see if we can't merge that >>> upstream with gcc and/or clang. >>> >>> >>> It also worth mentioning that I believe gcc's Ada front end >>> is written in Ada -- you don't actually have to write >>> the frontend in C/C++ to merge upstream. >> >> Yes, I once worked on this front end. As I understood, SRC M3 >> was not merged into gcc only because RMS was annoyed that SRC made >> it a separate executable, so its license was not poisoned by GPL. >> >>> >>> But there might remain licensing concern. >>> >>> >>> >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 28 02:31:53 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 19:31:53 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: <5771C579.2000607@lcwb.coop> On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > >> And no, the names and operator spellings are not close to adequate >> to clue you in. They have gone to every length possible to use >> every clever new C++ "feature" that comes out in the latest >> C++- standard, which always just increases the complexity >> of the search to a declaration. So I don't fancy doing any of >> this. (BTW, =17 in recent discussions.) > > A teacher of mine called this behavior "version junkie". > Yes, yes. > >> Actually, keeping their bitcode stable across llvm releases is >> one place they do talk about compatibility. But m3llvm uses calls >> to llvm APIs to construct llvm IR as in-memory data, then another >> call to get llvm to convert it to bitcode. So bitcode's stability >> is irrelevant to us. I once thought about producing llvm bitcode >> directly, but that seems like a pretty big job. It would, however, >> obviate creating most of those wretched bindings. > > An alternative would be to create .ll text files. But its format changes, too. Yes. But, according to the list talk, they don't have the intention to make it as stable as the bitcode format. > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 28 03:03:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 01:03:47 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5771C579.2000607@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: So..rambling a bit..but I have discussed some with people and considered it. "the interface to the compiler backend" and my half serious answer: All of the compilers have a well documented very stable interface to their backends, and it is in fact the same, roughly, interface to all the backends: source code. It is true it isn't the most efficient representation. Maybe the least efficient. It might not expose all the internals at least portable (e.g. tail call). But it works, is heavily used, is well known, documented, has high compatibility requirements, somewhat readable with standard tools, etc. I would advocate that C and C++ be evolved a little bit for these purposes. In particular, C needs exception handling. C and C++ need a tail call notion. _alloca should maybe be standardized. I should be able to generate image-relative pointers/offsets. (trivial in Microsoft assembly with "imagerel"), to help me layout position indepenent data structures. C-- is kind of this, and there was some C-resembling assembly,' but I think really C should be the starting point, as it is pretty close. Probably need some extensions like non-int bitfields, and rotate, and shift right with both sign copying and zero fill. > A teacher of mine called this behavior "version junkie". There are at least two big reasons for this. - The language really is improving. Programs written in the newer language are easier to read and often easier to optimize and sometimes easier to implement a compiler for. - Dogfooding -- using the language helps inform the language implementer where they have done things right, or wrong, and what to improve. I believe in fact that under-dogfooding of C++ led to some early omissions. The need for auto for example. Granted, Stroustrup put it in in the 1980s and had to remove it. But with more dogfooding by more implementers, it would have been added earlier. Similarly "template typedefs". Are obviously needed once you use templates for about a day. Modula-3 has similar failings imho. For example, the fact that with/var imply a nesting that "needs" indentation and needs "end". This is something that C++ and much later even C fixed. Perhaps though, perhaps the Modula-3 designers were balancing the specification and implementation against user convenience, as the current design is obviously simpler to specify and implement. But tedious to use. So the binary form of LLVM bit code is more stable than the text form? - Jay > Date: Mon, 27 Jun 2016 19:31:53 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > >> And no, the names and operator spellings are not close to adequate > >> to clue you in. They have gone to every length possible to use > >> every clever new C++ "feature" that comes out in the latest > >> C++- standard, which always just increases the complexity > >> of the search to a declaration. So I don't fancy doing any of > >> this. (BTW, =17 in recent discussions.) > > > > A teacher of mine called this behavior "version junkie". > > > > Yes, yes. > > > > >> Actually, keeping their bitcode stable across llvm releases is > >> one place they do talk about compatibility. But m3llvm uses calls > >> to llvm APIs to construct llvm IR as in-memory data, then another > >> call to get llvm to convert it to bitcode. So bitcode's stability > >> is irrelevant to us. I once thought about producing llvm bitcode > >> directly, but that seems like a pretty big job. It would, however, > >> obviate creating most of those wretched bindings. > > > > An alternative would be to create .ll text files. But its format changes, too. > > Yes. But, according to the list talk, they don't have the intention to > make it as stable as the bitcode format. > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 28 07:00:49 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 05:00:49 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , , , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , , , , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , , , , , , , , , , <57718BFF.1070501@lcwb.coop>, , Message-ID: Links paste funny. I'll try again: 1.17 https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.17;content-type=text%2Fplain vs. v1.18 https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.18;content-type=text%2Fplain Tony, you changed from..er...hold on.. that was not really a change. here vs. 1.12: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.12;content-type=text%2Fplain /* Copyright (C) 1993, Digital Equipment Corporation ? ? ? ? ? ? ? ? ? ? ? ? */ /* All rights reserved. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ /* See the file COPYRIGHT for a full description. ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ v 1.13: ? ? Copyright (C) 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. ? ? This program is free software; you can redistribute it and/or modify it ? ? under the terms of the GNU General Public License as published by the Free ? ? Software Foundation; either version 2, or (at your option) any later ? ? version. ... Merge with gcc-core-3.4.5. ?This is now the default backend. in either case, there is a notion of "GPL compatible". For example the BSD licenses are GPL compatible. They can be linked with GPL, or such. And then..there is the question of derivation and ownership. There is only a few thousand lines here. The slightly interesting part is the reading of the IR. The building up of gcc tree's is gcc specific. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 20:42:51 +0000 > > Tony changed the license from DEC to FSF here (possibly someone else did in another branch) > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 > > > There is this growth: > > jair:~ jay$ wc -l parse.c.current > 6516 parse.c.current > > jair:~ jay$ wc -l parse.c.old > 4190 parse.c.old > > though derivation is still hard to argue with. > > - Jay > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Date: Mon, 27 Jun 2016 20:30:57 +0000 >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> Some number should be reasonable yes, but I think you exaggerate how much I've done. :) >> I'll look later. >> >> - Jay >> >> >> >> ---------------------------------------- >>> Date: Mon, 27 Jun 2016 15:26:39 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] parse.c licensing question, dual? >>> >>> >>> >>> On 06/27/2016 03:19 PM, Jay K wrote: >>>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>>> >>>> But parse.c might be ownerless and stuck? >>> >>> Isn't there some law that if more than 25% of a work is changed, it ceases >>> to be a derived work but becomes a new work by the changer? If so, I would >>> guess the current parse.c would meet this criterion, and I think Jay would >>> own it, since I think he has done the majority of the changes. >>> >>> IANAL. >>> >>>> >>>> - Jay >>>> >>>> >>>> ---------------------------------------- >>>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>>> From: lemming at henning-thielemann.de >>>>> To: jay.krell at cornell.edu >>>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>> >>>>> >>>>> On Mon, 27 Jun 2016, Jay K wrote: >>>>> >>>>>> Can the owner relicense parse.c, >>>>> >>>>> Sure, the owner can always relicense. >>>>> >>>>>> or it is stuck with GPL because it links to gcc? >>>>> >>>>> He can choose any license that is compatible with GPL. >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From jay.krell at cornell.edu Tue Jun 28 07:37:17 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 05:37:17 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , <57718BFF.1070501@lcwb.coop>, , , , Message-ID: I can understand RMS's reaction, since this subverts the spirit of the license, by introducing a process boundary to break the linkage, where normally and conceptually there is a linkage, but I"m wondering about the letter not the spirit. :) > As I understand it, because parse.c is an integrated front-end to gcc it must carry the GPL license This isn't clear to me. There is some notion of "GPL compatible", which includes BSD licenses. ?> Since it is based on other gcc code it becomes GPL Also slightly unclear -- "based on" could be 99% resembling or 1% resembling. In either case...on my big list, maybe, is write a new front end in C or C++ and integrate with gcc or LLVM. Or write out C (which is, still, an integrate path for gcc/LLVM). I can translate *most* of Modula-3 to C in my head -- it almost trivially isomorphic. There are a few parts of the ABI that aren't obvious to me -- open arrays and exceptions. I understand part of them -- but e.g. the "dynamic construction" of exceptions, seems to almost resemble C++, but that doesn't really compute for me in the context of Modula-3. I also suspect it matters less now. There are larger enemies now. ?- Jay ---------------------------------------- > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Tue, 28 Jun 2016 05:14:53 +0000 > > As I understand it, because parse.c is an integrated front-end to gcc it must carry the GPL license. Since it is based on other gcc code it becomes GPL-contaminated. I recall that Stallman hit the roof when he found out that M3 was using a shim front-end to gcc the allowed M3 to avoid GPL. > >> On 28 Jun 2016, at 3:00 PM, Jay K wrote: >> >> Links paste funny. >> >> I'll try again: >> >> 1.17 >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.17;content-type=text%2Fplain >> vs. >> >> v1.18 >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.18;content-type=text%2Fplain >> >> >> Tony, you changed from..er...hold on.. >> that was not really a change. >> >> >> here vs. 1.12: >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.12;content-type=text%2Fplain >> >> /* Copyright (C) 1993, Digital Equipment Corporation */ >> /* All rights reserved. */ >> /* See the file COPYRIGHT for a full description. */ >> >> >> v 1.13: >> >> >> Copyright (C) 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. >> >> This program is free software; you can redistribute it and/or modify it >> under the terms of the GNU General Public License as published by the Free >> Software Foundation; either version 2, or (at your option) any later >> version. >> ... >> >> Merge with gcc-core-3.4.5. This is now the default backend. >> >> >> in either case, there is a notion of "GPL compatible". >> >> For example the BSD licenses are GPL compatible. >> They can be linked with GPL, or such. >> >> And then..there is the question of derivation and ownership. >> >> There is only a few thousand lines here. >> >> The slightly interesting part is the reading of the IR. >> The building up of gcc tree's is gcc specific. >> >> - Jay >> >> >> >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: [M3devel] parse.c licensing question, dual? >>> Date: Mon, 27 Jun 2016 20:42:51 +0000 >>> >>> Tony changed the license from DEC to FSF here (possibly someone else did in another branch) >>> >>> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 >>> >>> >>> There is this growth: >>> >>> jair:~ jay$ wc -l parse.c.current >>> 6516 parse.c.current >>> >>> jair:~ jay$ wc -l parse.c.old >>> 4190 parse.c.old >>> >>> though derivation is still hard to argue with. >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>>> Date: Mon, 27 Jun 2016 20:30:57 +0000 >>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>> >>>> Some number should be reasonable yes, but I think you exaggerate how much I've done. :) >>>> I'll look later. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> Date: Mon, 27 Jun 2016 15:26:39 -0500 >>>>> From: rodney_bates at lcwb.coop >>>>> To: m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>> >>>>> >>>>> >>>>> On 06/27/2016 03:19 PM, Jay K wrote: >>>>>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>>>>> >>>>>> But parse.c might be ownerless and stuck? >>>>> >>>>> Isn't there some law that if more than 25% of a work is changed, it ceases >>>>> to be a derived work but becomes a new work by the changer? If so, I would >>>>> guess the current parse.c would meet this criterion, and I think Jay would >>>>> own it, since I think he has done the majority of the changes. >>>>> >>>>> IANAL. >>>>> >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> ---------------------------------------- >>>>>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>>>>> From: lemming at henning-thielemann.de >>>>>>> To: jay.krell at cornell.edu >>>>>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>>>> >>>>>>> >>>>>>> On Mon, 27 Jun 2016, Jay K wrote: >>>>>>> >>>>>>>> Can the owner relicense parse.c, >>>>>>> >>>>>>> Sure, the owner can always relicense. >>>>>>> >>>>>>>> or it is stuck with GPL because it links to gcc? >>>>>>> >>>>>>> He can choose any license that is compatible with GPL. >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>>>> >>>>> >>>>> -- >>>>> Rodney Bates >>>>> rodney.m.bates at acm.org >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From rodney_bates at lcwb.coop Tue Jun 28 18:28:01 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 28 Jun 2016 11:28:01 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: <5772A591.3080305@lcwb.coop> On 06/27/2016 08:03 PM, Jay K wrote: > So..rambling a bit..but I have discussed some with people > and considered it. > > "the interface to the compiler backend" > > > and my half serious answer: > > > All of the compilers have a well documented very stable > interface to their backends, and it is in fact the same, > roughly, interface to all the backends: source code. > > > It is true it isn't the most efficient representation. > Maybe the least efficient. > > > It might not expose all the internals at least portable (e.g. tail call). > But it works, is heavily used, is well known, documented, > has high compatibility requirements, somewhat readable with > standard tools, etc. > > > I would advocate that C and C++ be evolved a little bit > for these purposes. In particular, C needs exception handling. > C and C++ need a tail call notion. > _alloca should maybe be standardized. > I should be able to generate image-relative pointers/offsets. > (trivial in Microsoft assembly with "imagerel"), to help > me layout position indepenent data structures. > > > C-- is kind of this, and there was some C-resembling assembly,' > but I think really C should be the starting point, as it is pretty close. > > > Probably need some extensions like non-int bitfields, and > rotate, and shift right with both sign copying and zero fill. > > > > A teacher of mine called this behavior "version junkie". > > > There are at least two big reasons for this. > > - The language really is improving. Programs > written in the newer language are easier to read > and often easier to optimize and sometimes easier > to implement a compiler for. > > - Dogfooding -- using the language helps inform the > language implementer where they have done things right, > or wrong, and what to improve. > > > I believe in fact that under-dogfooding of C++ led > to some early omissions. The need for auto for example. > Granted, Stroustrup put it in in the 1980s and had to remove it. > But with more dogfooding by more implementers, it would > have been added earlier. Similarly "template typedefs". > Are obviously needed once you use templates for about a day. > > > Modula-3 has similar failings imho. > For example, the fact that with/var imply > a nesting that "needs" indentation and needs "end". > This is something that C++ and much later even C fixed. > > > Perhaps though, perhaps the Modula-3 designers were > balancing the specification and implementation against > user convenience, as the current design is obviously > simpler to specify and implement. But tedious to use. > > So the binary form of LLVM bit code is more stable than the text form? > That's what they are saying. A terminology point: llvm would consider, I think, that "bit code" refers only to the binary form, and the text form would be called something like "llvm assembly code". Files of binary form are conventionally named .bc and assembly code .ll. In the M3 llvm back end binary files are .ib and .mb, to distinguish interfaces and modules. I am also using .ill and .mll for assembly files, but I'm not sure these suffixes are coded in anywhere. > > - Jay > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > >> And no, the names and operator spellings are not close to adequate > > >> to clue you in. They have gone to every length possible to use > > >> every clever new C++ "feature" that comes out in the latest > > >> C++- standard, which always just increases the complexity > > >> of the search to a declaration. So I don't fancy doing any of > > >> this. (BTW, =17 in recent discussions.) > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > Yes, yes. > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > >> one place they do talk about compatibility. But m3llvm uses calls > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > >> is irrelevant to us. I once thought about producing llvm bitcode > > >> directly, but that seems like a pretty big job. It would, however, > > >> obviate creating most of those wretched bindings. > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > Yes. But, according to the list talk, they don't have the intention to > > make it as stable as the bitcode format. > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 28 18:40:06 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 28 Jun 2016 11:40:06 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: <5772A866.7020601@lcwb.coop> On 06/27/2016 08:03 PM, Jay K wrote: > So..rambling a bit..but I have discussed some with people > and considered it. > > "the interface to the compiler backend" > > > and my half serious answer: > > > All of the compilers have a well documented very stable > interface to their backends, and it is in fact the same, > roughly, interface to all the backends: source code. > > > It is true it isn't the most efficient representation. > Maybe the least efficient. > > > It might not expose all the internals at least portable (e.g. tail call). > But it works, is heavily used, is well known, documented, > has high compatibility requirements, somewhat readable with > standard tools, etc. > > > I would advocate that C and C++ be evolved a little bit > for these purposes. In particular, C needs exception handling. > C and C++ need a tail call notion. > _alloca should maybe be standardized. > I should be able to generate image-relative pointers/offsets. > (trivial in Microsoft assembly with "imagerel"), to help > me layout position indepenent data structures. > > > C-- is kind of this, and there was some C-resembling assembly,' > but I think really C should be the starting point, as it is pretty close. > > > Probably need some extensions like non-int bitfields, and > rotate, and shift right with both sign copying and zero fill. > > > > A teacher of mine called this behavior "version junkie". > > > There are at least two big reasons for this. > > - The language really is improving. Programs > written in the newer language are easier to read > and often easier to optimize and sometimes easier > to implement a compiler for. Sometimes so, sometimes not. Sadly, many language "features" reflect an implicit but very misguided belief that fewer keystrokes/characters means increased readability. Or at least that writability is more important than readability. So often, this means actual code is less explicit. But this makes maintenance far worse. E.g., Ada decided use parentheses for both actual parameter lists to function calls and subscript lists to arrays. Along with optional implicit operations like dereferencing, there are somewhere in the teens of possible meanings for the innocent looking "f(x)". I have forgotten the exact number, but once had to do the semantic analysis. That was Ada 83. Maybe more have been added since. For the poor schmuck who gets called at 3:00 AM to fix a bug in half a million lines of code she didn't write, this is a readability disaster. The savings of keystrokes in not making the operations explicit is penny-wise and million-pound foolish. > > - Dogfooding -- using the language helps inform the > language implementer where they have done things right, > or wrong, and what to improve. > > > I believe in fact that under-dogfooding of C++ led > to some early omissions. The need for auto for example. > Granted, Stroustrup put it in in the 1980s and had to remove it. > But with more dogfooding by more implementers, it would > have been added earlier. Similarly "template typedefs". > Are obviously needed once you use templates for about a day. > > > Modula-3 has similar failings imho. > For example, the fact that with/var imply > a nesting that "needs" indentation and needs "end". > This is something that C++ and much later even C fixed. > > > Perhaps though, perhaps the Modula-3 designers were > balancing the specification and implementation against > user convenience, as the current design is obviously > simpler to specify and implement. But tedious to use. > > So the binary form of LLVM bit code is more stable than the text form? > > > - Jay > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > >> And no, the names and operator spellings are not close to adequate > > >> to clue you in. They have gone to every length possible to use > > >> every clever new C++ "feature" that comes out in the latest > > >> C++- standard, which always just increases the complexity > > >> of the search to a declaration. So I don't fancy doing any of > > >> this. (BTW, =17 in recent discussions.) > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > Yes, yes. > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > >> one place they do talk about compatibility. But m3llvm uses calls > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > >> is irrelevant to us. I once thought about producing llvm bitcode > > >> directly, but that seems like a pretty big job. It would, however, > > >> obviate creating most of those wretched bindings. > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > Yes. But, according to the list talk, they don't have the intention to > > make it as stable as the bitcode format. > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 28 20:12:53 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 18:12:53 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5772A866.7020601@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, ,,<12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, ,,, , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop>, , , , <5771C579.2000607@lcwb.coop> ,<5772A866.7020601@lcwb.coop> Message-ID: > The savings of keystrokes in not making the operations explicit is penny-wise and million-pound foolish. I hear this a lot. There is a flip side. There is such a thing as too verbose, or explicitly saying everything all the time is inefficient. In fact, if we had to say everything, all the time, we'd instantly run out of space. Requiring too much verbosity can be tedious and hide the important details in the noise of obvious repeated stuff. Some balance must be found. Consider: C: a = b vs. asssembly hypothetical: mov ra, rb "=" has become "mov r r" -- a large multiplication of noise that the human reader has to sift through to see the meaning or worse mov r1, rsp[b] mov rsp[a], r1 Is the assembly clearer because it is more explicit? operator= can be overloaded It means assignment, however the author of the types of a and b deemed appropriate it might be on the order of 1-3 instructions, or it might be a function call. Ignoring performance, it doesn't matter. It is the right notation for "assignment". We must pick local vocabularies or "context" and communicate within it. It must be easy for a newcomer to step back and learn the vocabulary -- that is probably the unsolved part. And then I often read code, that reads like: // foo the bar foo(bar) I don't know what a "bar" is, I don't know what it means to "foo" it, or why/when you would want to. The pattern works in Modula-3, C, C++, etc. even assembly ; foo the bar mov rcx, rsp[bar] call foo all equally gibberish to the newcomer. Though to the initiated, assuming foo is at least a few lines of code, this is probably the right level to code at. - Jay > Date: Tue, 28 Jun 2016 11:40:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 08:03 PM, Jay K wrote: > > So..rambling a bit..but I have discussed some with people > > and considered it. > > > > "the interface to the compiler backend" > > > > > > and my half serious answer: > > > > > > All of the compilers have a well documented very stable > > interface to their backends, and it is in fact the same, > > roughly, interface to all the backends: source code. > > > > > > It is true it isn't the most efficient representation. > > Maybe the least efficient. > > > > > > It might not expose all the internals at least portable (e.g. tail call). > > But it works, is heavily used, is well known, documented, > > has high compatibility requirements, somewhat readable with > > standard tools, etc. > > > > > > I would advocate that C and C++ be evolved a little bit > > for these purposes. In particular, C needs exception handling. > > C and C++ need a tail call notion. > > _alloca should maybe be standardized. > > I should be able to generate image-relative pointers/offsets. > > (trivial in Microsoft assembly with "imagerel"), to help > > me layout position indepenent data structures. > > > > > > C-- is kind of this, and there was some C-resembling assembly,' > > but I think really C should be the starting point, as it is pretty close. > > > > > > Probably need some extensions like non-int bitfields, and > > rotate, and shift right with both sign copying and zero fill. > > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > There are at least two big reasons for this. > > > > - The language really is improving. Programs > > written in the newer language are easier to read > > and often easier to optimize and sometimes easier > > to implement a compiler for. > > Sometimes so, sometimes not. Sadly, many language "features" reflect > an implicit but very misguided belief that fewer keystrokes/characters > means increased readability. Or at least that writability is more > important than readability. So often, this means actual code is less > explicit. But this makes maintenance far worse. > > E.g., Ada decided use parentheses for both actual parameter lists to > function calls and subscript lists to arrays. Along with optional > implicit operations like dereferencing, there are somewhere in the > teens of possible meanings for the innocent looking "f(x)". I have forgotten > the exact number, but once had to do the semantic analysis. That was > Ada 83. Maybe more have been added since. For the poor schmuck who > gets called at 3:00 AM to fix a bug in half a million lines of code > she didn't write, this is a readability disaster. The savings of > keystrokes in not making the operations explicit is penny-wise and > million-pound foolish. > > > > > - Dogfooding -- using the language helps inform the > > language implementer where they have done things right, > > or wrong, and what to improve. > > > > > > I believe in fact that under-dogfooding of C++ led > > to some early omissions. The need for auto for example. > > Granted, Stroustrup put it in in the 1980s and had to remove it. > > But with more dogfooding by more implementers, it would > > have been added earlier. Similarly "template typedefs". > > Are obviously needed once you use templates for about a day. > > > > > > Modula-3 has similar failings imho. > > For example, the fact that with/var imply > > a nesting that "needs" indentation and needs "end". > > This is something that C++ and much later even C fixed. > > > > > > Perhaps though, perhaps the Modula-3 designers were > > balancing the specification and implementation against > > user convenience, as the current design is obviously > > simpler to specify and implement. But tedious to use. > > > > So the binary form of LLVM bit code is more stable than the text form? > > > > > > - Jay > > > > > > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > > > >> And no, the names and operator spellings are not close to adequate > > > >> to clue you in. They have gone to every length possible to use > > > >> every clever new C++ "feature" that comes out in the latest > > > >> C++- standard, which always just increases the complexity > > > >> of the search to a declaration. So I don't fancy doing any of > > > >> this. (BTW, =17 in recent discussions.) > > > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > > Yes, yes. > > > > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > > >> one place they do talk about compatibility. But m3llvm uses calls > > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > > >> is irrelevant to us. I once thought about producing llvm bitcode > > > >> directly, but that seems like a pretty big job. It would, however, > > > >> obviate creating most of those wretched bindings. > > > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > > > Yes. But, according to the list talk, they don't have the intention to > > > make it as stable as the bitcode format. > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Jun 28 22:15:25 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 28 Jun 2016 22:15:25 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5772A866.7020601@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> Message-ID: <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> On Tue, 28 Jun 2016 11:40:06 -0500 "Rodney M. Bates" wrote: > On 06/27/2016 08:03 PM, Jay K wrote: > > > A teacher of mine called this behavior "version junkie". > > > > > > There are at least two big reasons for this. > > > > - The language really is improving. Programs > > written in the newer language are easier to read > > and often easier to optimize and sometimes easier > > to implement a compiler for. > > Sometimes so, sometimes not. Sadly, many language "features" reflect > an implicit but very misguided belief that fewer keystrokes/characters > means increased readability. Or at least that writability is more > important than readability. So often, this means actual code is less > explicit. But this makes maintenance far worse. > > E.g., Ada decided use parentheses for both actual parameter lists to > function calls and subscript lists to arrays. Along with optional > implicit operations like dereferencing, there are somewhere in the > teens of possible meanings for the innocent looking "f(x)". I have forgotten > the exact number, but once had to do the semantic analysis. That was > Ada 83. Maybe more have been added since. For the poor schmuck who > gets called at 3:00 AM to fix a bug in half a million lines of code > she didn't write, this is a readability disaster. The savings of > keystrokes in not making the operations explicit is penny-wise and > million-pound foolish. I couldn't agree more with this. My focus seems to have moved away from programming to reading and analyzing code, too. Often it is almost next to impossible for me to find the exact meaning or definition of an expression or call without knowing all the other code which is not obviously related to the location in question. I would favour longer explicit syntax and clear meaning above shorter expressions any time. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Jun 28 23:34:35 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 21:34:35 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, <5771C579.2000607@lcwb.coop>, <5772A866.7020601@lcwb.coop>, <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: I find myself on the other side of this. There are many people on both sides. There are countless examples. Here are two. People don't like operator loading. They don't want me to write string operator+(string, string); but nobody seems to mind: float f = 1.0 + 2.0; int i = 1 + 2; why is + ok on floats and ints, overloaded, but not user defined types such as string? People don' t like type inferencing. auto a = 1.0 + 2.0; auto b = 1 + 2; Modula-3 has this more than C and C++98 already, so maybe people here don't mind. VAR a := 1.0 + 2.0; VAR b := 1 + 2; In either case, nobody seems to mind temporaries without explicit types in function calls or more generally subexpressions. F(1 + 2); F2(1.0 + 2.0); - Jay > Date: Tue, 28 Jun 2016 22:15:25 +0200 > From: wagner at elegosoft.com > To: rodney.m.bates at acm.org > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Tue, 28 Jun 2016 11:40:06 -0500 > "Rodney M. Bates" wrote: > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > There are at least two big reasons for this. > > > > > > - The language really is improving. Programs > > > written in the newer language are easier to read > > > and often easier to optimize and sometimes easier > > > to implement a compiler for. > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > an implicit but very misguided belief that fewer keystrokes/characters > > means increased readability. Or at least that writability is more > > important than readability. So often, this means actual code is less > > explicit. But this makes maintenance far worse. > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > function calls and subscript lists to arrays. Along with optional > > implicit operations like dereferencing, there are somewhere in the > > teens of possible meanings for the innocent looking "f(x)". I have forgotten > > the exact number, but once had to do the semantic analysis. That was > > Ada 83. Maybe more have been added since. For the poor schmuck who > > gets called at 3:00 AM to fix a bug in half a million lines of code > > she didn't write, this is a readability disaster. The savings of > > keystrokes in not making the operations explicit is penny-wise and > > million-pound foolish. > > I couldn't agree more with this. My focus seems to have moved away from > programming to reading and analyzing code, too. Often it is almost next > to impossible for me to find the exact meaning or definition of an > expression or call without knowing all the other code which is not > obviously related to the location in question. > > I would favour longer explicit syntax and clear meaning above > shorter expressions any time. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Jun 29 00:39:17 2016 From: lists at darko.org (Darko Volaric) Date: Wed, 29 Jun 2016 00:39:17 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: On Tue, Jun 28, 2016 at 11:34 PM, Jay K wrote: > I find myself on the other side of this. > There are many people on both sides. > > There are countless examples. > > > Here are two. > > > People don't like operator loading. > > They don't want me to write > > string operator+(string, string); > > > but nobody seems to mind: > > > float f = 1.0 + 2.0; > int i = 1 + 2; > > why is + ok on floats and ints, overloaded, but not user defined types > such as string? > > > The notion that limited fixed operator overloading justifies unlimited user operator overloading is an asinine argument. The former you can easily keep in your head, since the rules are simple and clear. You know given a + b + c + 1.0 that a, b and c are floating point values, or if you didn't have the literal you'd know by knowing the type of one value. But given user overloading, what is the type of a + b + c + 1.0 or some other expression? You have to know what the type of each value is, you need to know the definitions of the overloads for each type pair, you have to work out what the precedence and associativity rules are so you're evaluating the expression correctly and you have to know the type that each overloaded operator returns and work out what the new type pairs are at each stage of evaluation of the expression. > People don' t like type inferencing. > > auto a = 1.0 + 2.0; > auto b = 1 + 2; > > Modula-3 has this more than C and C++98 already, so maybe people here > don't mind. > > VAR a := 1.0 + 2.0; > VAR b := 1 + 2; > > In either case, nobody seems to mind temporaries without explicit types > in function calls or more generally subexpressions. > > F(1 + 2); > F2(1.0 + 2.0); > > Again, this is "type inference" in the most simplistic case and hardly justifies unbounded type inferencing. The static type pf every expression (and subexpression) is known in M3, merely using it introduces no complexity. The static type is easily determined because the rules for determining the type of any expression are small, fixed, simple and strict. If you really want to understand type inferencing and overloading, have a look at the Swift programming language. You'll find yourself writing an innocuous expression and the compiler will tell you that the expression is "ambiguous" or "too complex" even though there doesn't seem any other way to interpret the expression or it involves only one operator/type combo used more than once. What the compiler is really telling you is that it can't infer the type of some subexpression or choose the right method and you must now start applying casts to make it obvious (this doesn't always work and sometimes you have to rewrite it wholesale). Why does this happen? Because Swift allows you to overload functions and operators based on parameter type and return type. You say "people are on both sides of this" but good design isn't a democracy or a popularity contest. If you walk around downtown San Francisco you can find many people who will agree with you. You will also find as many people who claim that the government is beaming thoughts into their heads via satellite. How people feel about language features and the fact that they are untroubled by C++ and its misfeatures is not a logical, convincing argument. The stated design goals for M3 is simplicity and safety, not feature richness or novelty. That should be what guides any discussion of M3 language design. > > - Jay > > > > Date: Tue, 28 Jun 2016 22:15:25 +0200 > > From: wagner at elegosoft.com > > To: rodney.m.bates at acm.org > > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > On Tue, 28 Jun 2016 11:40:06 -0500 > > "Rodney M. Bates" wrote: > > > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > > > > There are at least two big reasons for this. > > > > > > > > - The language really is improving. Programs > > > > written in the newer language are easier to read > > > > and often easier to optimize and sometimes easier > > > > to implement a compiler for. > > > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > > an implicit but very misguided belief that fewer keystrokes/characters > > > means increased readability. Or at least that writability is more > > > important than readability. So often, this means actual code is less > > > explicit. But this makes maintenance far worse. > > > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > > function calls and subscript lists to arrays. Along with optional > > > implicit operations like dereferencing, there are somewhere in the > > > teens of possible meanings for the innocent looking "f(x)". I have > forgotten > > > the exact number, but once had to do the semantic analysis. That was > > > Ada 83. Maybe more have been added since. For the poor schmuck who > > > gets called at 3:00 AM to fix a bug in half a million lines of code > > > she didn't write, this is a readability disaster. The savings of > > > keystrokes in not making the operations explicit is penny-wise and > > > million-pound foolish. > > > > I couldn't agree more with this. My focus seems to have moved away from > > programming to reading and analyzing code, too. Often it is almost next > > to impossible for me to find the exact meaning or definition of an > > expression or call without knowing all the other code which is not > > obviously related to the location in question. > > > > I would favour longer explicit syntax and clear meaning above > > shorter expressions any time. > > > > Olaf > > -- > > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Jun 29 01:54:06 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 28 Jun 2016 23:54:06 +0000 (UTC) Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: <53036089.5289165.1467158046154.JavaMail.yahoo@mail.yahoo.com> Hi all:it depends, Cardelli has stated in an interview the type system was the top part thing of language designers at its time. I recall SPwM3 states also that they video recorded all design sessions, though provably these would not be very attracting to the new modula-3 programmer or computer scientists (instead of it, discouraging). Probably the safety and simplicity were user/programmer priorities Thanks in advance El Martes 28 de junio de 2016 17:39, Darko Volaric escribi?: On Tue, Jun 28, 2016 at 11:34 PM, Jay K wrote: ?I find myself on the other side of this. ?There are many people on both sides. ?There are countless examples. ? Here are two. ?People don't like operator loading.? ?They don't want me to write? ?string operator+(string, string); ?but nobody seems to mind: ???float f = 1.0 + 2.0;? ? int?i = 1 + 2; ? why is + ok on floats and ints, overloaded, but not user defined types such as string??? The notion that limited fixed operator overloading justifies unlimited user operator overloading is an asinine argument. The former you can easily keep in your head, since the rules are simple and clear. You know given a + b + c + 1.0 that a, b and c are floating point values, or if you didn't have the literal you'd know by knowing the type of one value. But given user overloading, what is the type of a + b + c + 1.0 or some other expression? You have to know what the type of each value is, you need to know the definitions of the overloads for each type pair, you have to work out what the precedence and associativity rules are so you're evaluating the expression correctly and you have to know the type that each overloaded operator returns and work out what the new type pairs are at each stage of evaluation of the expression. ? ?People don' t like type inferencing.? ??? auto a = 1.0 + 2.0;? ???auto b = 1 + 2; ??Modula-3 has this more than C and C++98 already, so maybe people here don't mind. ???VAR a := 1.0 + 2.0; ??VAR b := 1 + 2; ?In either case, nobody seems to mind temporaries without explicit types in function calls or more generally subexpressions. ?F(1 + 2); ?F2(1.0 + 2.0); Again, this is "type inference" in the most simplistic case and hardly justifies unbounded type inferencing. The static type pf every expression (and subexpression) is known in M3, merely using it introduces no complexity. The static type is easily determined because the rules for determining the type of any expression are small, fixed, simple and strict. If you really want to understand type inferencing and overloading, have a look at the Swift programming language. You'll find yourself writing an innocuous expression and the compiler will tell you that the expression is "ambiguous" or "too complex" even though there doesn't seem any other way to interpret the expression or it involves only one operator/type combo used more than once. What the compiler is really telling you is that it can't infer the type of some subexpression or choose the right method and you must now start applying casts to make it obvious (this doesn't always work and sometimes you have to rewrite it wholesale). Why does this happen? Because Swift allows you to overload functions and operators based on parameter type and return type. You say "people are on both sides of this" but good design isn't a democracy or a popularity contest. If you walk around downtown San Francisco you can find many people who will agree with you. You will also find as many people who claim that the government is beaming thoughts into their heads via satellite. How people feel about language features and the fact that they are untroubled by C++ and its misfeatures is not a logical, convincing argument.? The stated design goals for M3 is simplicity and safety, not feature richness or novelty. That should be what guides any discussion of M3 language design. ? ?- Jay > Date: Tue, 28 Jun 2016 22:15:25 +0200 > From: wagner at elegosoft.com > To: rodney.m.bates at acm.org > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Tue, 28 Jun 2016 11:40:06 -0500 > "Rodney M. Bates" wrote: > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > There are at least two big reasons for this. > > > > > > - The language really is improving. Programs > > > written in the newer language are easier to read > > > and often easier to optimize and sometimes easier > > > to implement a compiler for. > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > an implicit but very misguided belief that fewer keystrokes/characters > > means increased readability. Or at least that writability is more > > important than readability. So often, this means actual code is less > > explicit. But this makes maintenance far worse. > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > function calls and subscript lists to arrays. Along with optional > > implicit operations like dereferencing, there are somewhere in the > > teens of possible meanings for the innocent looking "f(x)". I have forgotten > > the exact number, but once had to do the semantic analysis. That was > > Ada 83. Maybe more have been added since. For the poor schmuck who > > gets called at 3:00 AM to fix a bug in half a million lines of code > > she didn't write, this is a readability disaster. The savings of > > keystrokes in not making the operations explicit is penny-wise and > > million-pound foolish. > > I couldn't agree more with this. My focus seems to have moved away from > programming to reading and analyzing code, too. Often it is almost next > to impossible for me to find the exact meaning or definition of an > expression or call without knowing all the other code which is not > obviously related to the location in question. > > I would favour longer explicit syntax and clear meaning above > shorter expressions any time. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 29 04:02:44 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 02:02:44 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> ,<5771C579.2000607@lcwb.coop> , <5772A866.7020601@lcwb.coop>, <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com>, , Message-ID: Integers and floats are not the universal types that all programmers should know about,?and no others. Each code base has types it deals with and functions over those types. The type of ?a + b? is just as clear or unclear as the type of: ?Add(a, b)? ?or, close:? ?Foo.Add(a, b)? ?The last is arguably most clear, the type is *probably* Foo.T.? ?but none of the cases are particularly clear, and far from all code? ?is written in this Foo_Add or Foo.Add fashion.? ?Plus the problem of mixing types.? ?VAR a,b,c:Foo.T;? ?b := Foo.Add(a, b);? ?c := Foo.AddI(a, 1);? ?(See there, I expanded to name overloading, this should be:? ?c := Foo.Add(a, 1);? ?or simply? ?c := a + 1; ?) ? Keep in mind, that not only are integers and floating point numbers not the sole and central types, but I can implement other "numbers". What if I have a multiprecision arithmetic library? Using "+" is very natural there. Giving it a longer name like Add hurts, doesn't help. Context is critical, and there is no one small context that suffices. What if I could say: a.+(b) ? Is that also bad? Now, I'm actually very open minded and somewhat torn on the matter. I am faced with a very large code base, and plain text search with very little language awareness. In that situation, C is actually advantageous. Indeed I don't search for "+" or "add", but specifically Foo_Add. Some argue that an IDE lets me just hover over some identifier and it shows me the type. But this does not scale. IDEs do not scale. ?(Not to mention that I don't use one...) Plain text search does scale. I believe language aware search can scale, but not everybody is using that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. It should have been pitched, imho, for its browsing/reading, not for its editing/development, at least as far as it got -- apparently browser-based development is viable these days. And also preprocessor token pasting is a bit of an enemy of plain text search. Also, I would have found Modula-3 and/or Quake much clear if they had used "+" for string concatenation, instead of ampersand. Context is kind of a sliding scale. Sometimes I have a lot, sometimes very little. Code presentation should perhaps adapt based on its viewer, but that is a fantasy, as code presentation is stuck in plain text, as directly typed by a human. Nobody is yet editing or viewing an abstract representation. Anyway, I'm well aware that there are a fair number of people on both sides of these positions.? ?- Jay ________________________________ > From: lists at darko.org > Date: Wed, 29 Jun 2016 00:39:17 +0200 > To: jay.krell at cornell.edu > Subject: Re: [M3devel] cm3 llvm backend? > CC: m3devel at elegosoft.com; rodney.m.bates at acm.org > > > > On Tue, Jun 28, 2016 at 11:34 PM, Jay K > > wrote: > I find myself on the other side of this. > There are many people on both sides. > > There are countless examples. > > > Here are two. > > > People don't like operator loading. > > They don't want me to write > > string operator+(string, string); > > > but nobody seems to mind: > > > float f = 1.0 + 2.0; > int i = 1 + 2; > > why is + ok on floats and ints, overloaded, but not user defined > types such as string? > > > > > The notion that limited fixed operator overloading justifies unlimited > user operator overloading is an asinine argument. The former you can > easily keep in your head, since the rules are simple and clear. You > know given a + b + c + 1.0 that a, b and c are floating point values, > or if you didn't have the literal you'd know by knowing the type of one > value. > > But given user overloading, what is the type of a + b + c + 1.0 or some > other expression? You have to know what the type of each value is, you > need to know the definitions of the overloads for each type pair, you > have to work out what the precedence and associativity rules are so > you're evaluating the expression correctly and you have to know the > type that each overloaded operator returns and work out what the new > type pairs are at each stage of evaluation of the expression. > > > > > People don' t like type inferencing. > > auto a = 1.0 + 2.0; > auto b = 1 + 2; > > Modula-3 has this more than C and C++98 already, so maybe people here > don't mind. > > VAR a := 1.0 + 2.0; > VAR b := 1 + 2; > > In either case, nobody seems to mind temporaries without explicit > types in function calls or more generally subexpressions. > > F(1 + 2); > F2(1.0 + 2.0); > > > > Again, this is "type inference" in the most simplistic case and hardly > justifies unbounded type inferencing. The static type pf every > expression (and subexpression) is known in M3, merely using it > introduces no complexity. The static type is easily determined because > the rules for determining the type of any expression are small, fixed, > simple and strict. > > > If you really want to understand type inferencing and overloading, have > a look at the Swift programming language. You'll find yourself writing > an innocuous expression and the compiler will tell you that the > expression is "ambiguous" or "too complex" even though there doesn't > seem any other way to interpret the expression or it involves only one > operator/type combo used more than once. What the compiler is really > telling you is that it can't infer the type of some subexpression or > choose the right method and you must now start applying casts to make > it obvious (this doesn't always work and sometimes you have to rewrite > it wholesale). Why does this happen? Because Swift allows you to > overload functions and operators based on parameter type and return > type. > > > > You say "people are on both sides of this" but good design isn't a > democracy or a popularity contest. If you walk around downtown San > Francisco you can find many people who will agree with you. You will > also find as many people who claim that the government is beaming > thoughts into their heads via satellite. How people feel about language > features and the fact that they are untroubled by C++ and its > misfeatures is not a logical, convincing argument. > > The stated design goals for M3 is simplicity and safety, not feature > richness or novelty. That should be what guides any discussion of M3 > language design. > > > > > - Jay > > >> Date: Tue, 28 Jun 2016 22:15:25 +0200 >> From: wagner at elegosoft.com >> To: rodney.m.bates at acm.org >> CC: rodney_bates at lcwb.coop; > jay.krell at cornell.edu; > m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 llvm backend? >> >> On Tue, 28 Jun 2016 11:40:06 -0500 >> "Rodney M. Bates" > > wrote: >> >>> On 06/27/2016 08:03 PM, Jay K wrote: >>>>> A teacher of mine called this behavior "version junkie". >>>> >>>> >>>> There are at least two big reasons for this. >>>> >>>> - The language really is improving. Programs >>>> written in the newer language are easier to read >>>> and often easier to optimize and sometimes easier >>>> to implement a compiler for. >>> >>> Sometimes so, sometimes not. Sadly, many language "features" reflect >>> an implicit but very misguided belief that fewer keystrokes/characters >>> means increased readability. Or at least that writability is more >>> important than readability. So often, this means actual code is less >>> explicit. But this makes maintenance far worse. >>> >>> E.g., Ada decided use parentheses for both actual parameter lists to >>> function calls and subscript lists to arrays. Along with optional >>> implicit operations like dereferencing, there are somewhere in the >>> teens of possible meanings for the innocent looking "f(x)". I have > forgotten >>> the exact number, but once had to do the semantic analysis. That was >>> Ada 83. Maybe more have been added since. For the poor schmuck who >>> gets called at 3:00 AM to fix a bug in half a million lines of code >>> she didn't write, this is a readability disaster. The savings of >>> keystrokes in not making the operations explicit is penny-wise and >>> million-pound foolish. >> >> I couldn't agree more with this. My focus seems to have moved away from >> programming to reading and analyzing code, too. Often it is almost next >> to impossible for me to find the exact meaning or definition of an >> expression or call without knowing all the other code which is not >> obviously related to the location in question. >> >> I would favour longer explicit syntax and clear meaning above >> shorter expressions any time. >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 >> Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Wed Jun 29 04:18:03 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 28 Jun 2016 22:18:03 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: <20160629021803.GC6086@topoi.pooq.com> On Wed, Jun 29, 2016 at 02:02:44AM +0000, Jay K wrote: > I believe language aware search can scale, but not everybody is using > that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. > It should have been pitched, imho, for its browsing/reading, not for its > editing/development, at least as far as it got -- apparently browser-based > development is viable these days. I remember reading Modula 3 code via a browser. Expecially following classes and their inheritance from module to module. I don't recall being able to edit it there. Are those viewing tools still around? Might tat have been a tool with PM3? -- hendrik From jay.krell at cornell.edu Wed Jun 29 05:54:15 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 03:54:15 +0000 Subject: [M3devel] m3front scanner div wierdness? Message-ID: Does anyone understand this stuff in m3front/Scanner.m3: ?Here vs. LocalHere? ?SameFile? ? ?I understand only this nuance: ? offset MOD MaxLines ?? ? MaxLines ?= 100000; ?is to crudely handle that when asserts fail, ?they pack the line number in with the assertion failure code, ?potentially loosing bits. ? ?I don't think this is a good design, they should just be separate INTEGERS, ?but this is besides the point. ? ?What doesn't makes sense to me is the machinations around file name. Here:? ? ? file := files [offset DIV MaxLines]; vs. LocalHere: ? ? file := local_files [fnum]; LocalHere makes sense. Here does not. PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = ? BEGIN ? ? RETURN (a DIV MaxLines) = (b DIV MaxLines); ? END SameFile; Shouldn't this just be a = b? Well, anyway, this SameFile function isn't called. Here and LocalHere are used. I'm looking here because I want to add a temporary measure such that the file names are leaf-only. In particular, because generic modules have target names in their paths and I want to temporarly remove target names from output, so I can prove that a few targets are identical. I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. This has to percolate quite a bit through the system -- the backends and the runtime. And then this Here vs. LocalHere difference should fall away. But still, what is it trying to do? Thank you, ?- Jay From jay.krell at cornell.edu Wed Jun 29 06:08:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 04:08:07 +0000 Subject: [M3devel] release 3.6, and history Message-ID: I'm wondering: ?- if anyone has the 3.6 release, can make it available? ?- I can probably find it. We can put it on github? And more so, history prior to 3.6? And between 3.6 and 4.0? Or if Bill Kalsow is still around and available to ask questions? The Scanner thing is wierd to me. I do see some changes..maybe I did this.. old version: PROCEDURE Here (VAR file: TEXT; VAR line: INTEGER) = BEGIN file := files [offset DIV MaxLines]; line := offset MOD MaxLines; END Here; PROCEDURE LocalHere (VAR file: TEXT; VAR line: INTEGER) = VAR fnum := offset DIV MaxLines; BEGIN IF (local_files[fnum] = NIL) THEN local_files[fnum] := Host.FileTail (files[fnum]); END; file := local_files [fnum]; line := offset MOD MaxLines; END LocalHere; There is this div/mod relationship, but I still find it wierd to use div there. You can see the FileTail which is maybe the change I want anyway. Here, yes, me: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3front/src/misc/Scanner.m3.diff?r1=1.5;r2=1.6 Revision?1.6:?download?- view:?text,?markup,?annotated?-?select?for?diffs Thu Feb 28 09:23:22 2008 MET?(8 years, 4 months ago) by?jkrell Branches:?MAIN Diff to: previous 1.5:?preferred,?unified Changes since revision 1.5: +1 -1 lines put full paths to source files in debug info This has the minor downsides of: 1) grows the debug info (it is already huge; who is counting?) 2) reveals file system layout in debug info (privacy?) 3) does it inhibit debugging files from other people's machines or does gdb dir still work? but definitely makes for a more pleasant debugging experience when debugging stuff you have built yourself. The linear searching to see if a name has been allocated a number yet will obviously slow way down due to a large increase in common prefixes, but that should be a hash table anyway. Linear search is lame. (or a trie, but working from the ends of the strings, minus the last one or few characters, due to common prefixes as well as common suffixes) Note that both m3front and m3cc changes are needed as m3front has paths relative to the current working directory or such. For most packages, you can get by without the m3front change and just prepend "../src/" to the path in m3cc, but that doesn't work for hierarchical packages such as libm3 and m3core which I am debugging. I still believe debug info should have full paths. It always does on Windows for C and C++ (and this isn't a language thing). It didn't always but they fixed it at some point. But generic modules maybe something else -- like line directives and full paths back to the .mg files. ?- Jay From jay.krell at cornell.edu Wed Jun 29 07:58:30 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 05:58:30 +0000 Subject: [M3devel] release 3.6, and history In-Reply-To: <08431E36-275B-474B-8D9C-A8517F287B8E@purdue.edu> References: , <08431E36-275B-474B-8D9C-A8517F287B8E@purdue.edu> Message-ID: The cvs history seems to be there, but isn't always clear. ?I found the elegosoft cvsweb sometimes easier to navigate. ? And other things -- it isn't obvious in github how to get file contents either before or after a change -- one is easy and for the other I look at the next/previous change and its before/after file contents. I guess I'll wait, eventually git might be as good as perforce.. I there is nothing before 5.0 or 4.0 though I think. I should have 3.6 somewhere around. ?- Jay ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] release 3.6, and history > Date: Wed, 29 Jun 2016 05:22:33 +0000 > > All the history should still be there on github. > > On 29 Jun 2016, at 2:08 PM, Jay K > > wrote: > > I'm wondering: > - if anyone has the 3.6 release, can make it available? > - I can probably find it. We can put it on github? > From wagner at elegosoft.com Wed Jun 29 08:48:05 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 29 Jun 2016 08:48:05 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160629021803.GC6086@topoi.pooq.com> References: <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> <20160629021803.GC6086@topoi.pooq.com> Message-ID: <20160629084805.31275c2f310cb4a614628bf2@elegosoft.com> On Tue, 28 Jun 2016 22:18:03 -0400 Hendrik Boom wrote: > On Wed, Jun 29, 2016 at 02:02:44AM +0000, Jay K wrote: > > > I believe language aware search can scale, but not everybody is using > > that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. > > It should have been pitched, imho, for its browsing/reading, not for its > > editing/development, at least as far as it got -- apparently browser-based > > development is viable these days. > > I remember reading Modula 3 code via a browser. Expecially following > classes and their inheritance from module to module. I don't recall > being able to edit it there. Are those viewing tools still around? > > Might tat have been a tool with PM3? No, it's still there. The code is in * https://github.com/modula3/cm3/tree/master/m3-tools/m3browser * https://github.com/modula3/cm3/tree/master/m3-tools/m3tohtml * https://github.com/modula3/cm3/tree/master/m3-sys/cm3ide ("Reactor") Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Jun 29 11:41:52 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 09:41:52 +0000 Subject: [M3devel] full source paths in debug info, environment variable to contain switches? Message-ID: I raised this matter years ago and people disagreed. Debug info, and probably compiler diagnostics, should have full source paths. Ease of use of compiler diagnostics varies, better and worse, with full paths. Debuggers similar. gcc and clang on Darwin use full paths. Observe: jair:~ jay$ cat 1.c int main() { } jair:~ jay$ rm a.out 1.o jair:~ jay$ /Users/jay/gcc-6.1.0-min/bin/gcc -g -c 1.c jair:~ jay$ dwarfdump 1.o | fgrep 1.c ?? ? ? ? ? ? AT_name( "1.c" ) ?? ? ? ? ? ? ? ? AT_decl_file( "/Users/jay/1.c" ) jair:~ jay$ rm a.out 1.o rm: a.out: No such file or directory jair:~ jay$ clang -c -g 1.c jair:~ jay$ dwarfdump 1.o | fgrep 1.c ?? ? ? ? ? ? AT_name( "1.c" ) ?? ? ? ? ? ? ? ? AT_decl_file( "/Users/jay/1.c" ) Now, for my immediate task at hand, I actually don't want this, at least for instantiated generic modules/interfaces. I propose cm3 should have some command line switches to control this behavior. The default should be full paths in debug info. I further propose that some environment variables be implemented, which shall contain cm3 switches. There is some precedent for this in the world. In the Microsoft C/C++ toolset: INCLUDES is include paths LIB is library paths CL, _CL_, LINK, and _LINK_ environment variables all contribute to compiler/linker command line (I think prepending and appending). I don't propose that many or generally that short of names, but maybe CM3_OPTIONS or CM3_ENV ? I don't want to use just CM3, because people might want that to contain the path to cm3 (without switches( Probably some command line option should quash this too, like cm3 -noenv Which maybe should also work, if it is in CM3_ENV, means to ignore the rest ? This is at once crude and powerfuli and elegant and hacky and efficient and inefficient. It is good because it is easy to use and will usually do what people want. It is bad because it is not scoped -- it travels through entire process trees. It is inefficient because it travels into processes that don't use it. For my current work, I went to shorten paths such that the target doesn't occur in them, so I can show that various targets generate identical C code, so I can fold them down to fewer targets. The paths to generic instantiations messes this up, at least. ?- Jay From jay.krell at cornell.edu Wed Jun 29 11:44:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 09:44:51 +0000 Subject: [M3devel] source paths to generics? Message-ID: I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, the source file I am told about is the instantiated i3/m3 file. This isn't particularly useful for the programmer or convenient for the compiler, right? The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? I guess it is slightly easier in the compiler, but easy to do better? I should look into make it so? My real agenda is I want to see: ? ../src/foo.mg instead of ?I386_DARWIN/foo.m3 I don't want the target variation, but other points seem true also, right? Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? I'll try to poke around more. ?- Jay From jay.krell at cornell.edu Wed Jun 29 17:22:14 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 15:22:14 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: References: Message-ID: mfront/src/misc/M3Header.m3 has this: ? ? (* build a synthetic file name & start reading *) ? ? filename := old_filename & " => " & filename; ? ? Scanner.Push (filename, s.generic, is_main := Scanner.in_main); and instantiated generics aren't what I thought. I thought the build system or compiler did a textual substitution of the generic parameters into the entire implementation, and saved that to the file system, and had the assert/debug paths point to it. So could step through what looks exactly like an .m3 file. But it doesn't. The instantiated file is a little stub, like: jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 (*generated by m3build*) MODULE ?TextAtomTbl = Table (Text, Atom) END TextAtomTbl. so I have one or two proposals. 1) old_filename above contains the target, it looks like: "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" or "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" These both occur. I'm not sure what the second is, seems like a mistake. I suggest the first string in both pairs be changed to be the leaf element, like: "TextAtomTbl.m3 => ../src/table/Table.mg" or "TextAtomTbl.m3 => ../src/table" and maybe the second string in both cases should be as it is in the second pair. 2) Possibly it should reall just be: ?../src/table/Table.mg? ? ?and developer can step through that, knowing that it is a somewhat abstracted ?form of what is running ? ?I'm willing to do this under like: ? ?CONST TemporaryForTargetConvergence = TRUE ? ?or VAR TemporaryForTargetConvergence: BOOLEAN; and a command line switch, but I suspect it isn't really useful to anyone asis, so might as well remove target unconditionally. ? ?? ? - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Wed, 29 Jun 2016 09:44:51 +0000 > Subject: [M3devel] source paths to generics? > > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > the source file I am told about is the instantiated i3/m3 file. > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > I guess it is slightly easier in the compiler, but easy to do better? > > I should look into make it so? > > My real agenda is I want to see: > ../src/foo.mg > > instead of > I386_DARWIN/foo.m3 > > I don't want the target variation, but other points seem true also, right? > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > I'll try to poke around more. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Wed Jun 29 18:09:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 11:09:34 -0500 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: Message-ID: <5773F2BE.3050902@lcwb.coop> On 06/28/2016 10:54 PM, Jay K wrote: > Does anyone understand this stuff in m3front/Scanner.m3: > > Here vs. LocalHere? > SameFile? > > > I understand only this nuance: > offset MOD MaxLines > > MaxLines = 100000; > > > is to crudely handle that when asserts fail, > they pack the line number in with the assertion failure code, > potentially loosing bits. > > > I don't think this is a good design, they should just be separate INTEGERS, > but this is besides the point. > This is just pure speculation, but I am very confident of it. These offsets have a very high occurrence count. There is code all over m3front that copies Scanner.offset into various data structures. So the small space saving of one INTEGER instead of two would be multiplied by a big number. I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. Today, I have 8 Gig in the one I bought, and could easily afford more, if I thought I needed it. Two integers would be cleaner, but this design is not too bad *if* you know the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one field. As pure documentation, there really should be a distinct type name (say FileNoAndLineNo?) for all values that use this representation, even though it just equates to INTEGER. There are a lot of variables lying around all over the front end that use this invariant, but are just declared as INTEGER. That's maintainer-hostile. > > What doesn't makes sense to me is the machinations around file name. > > > Here: > file := files [offset DIV MaxLines]; > > vs. LocalHere: > file := local_files [fnum]; > > > LocalHere makes sense. Here does not. > > > PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = > BEGIN > RETURN (a DIV MaxLines) = (b DIV MaxLines); > END SameFile; > > > > Shouldn't this just be a = b? > As coded, this will return TRUE if a and b are different line numbers within the same file. The name "SameFile" at least suggests that is what is intended. A good example of a place where it would have been clearer if a & b were declared as the type name I proposed above. > > Well, anyway, this SameFile function isn't called. > > Here and LocalHere are used. > > > I'm looking here because I want to add a temporary measure > such that the file names are leaf-only. > > > In particular, because generic modules have target names in their paths > and I want to temporarly remove target names from output, so I can prove > that a few targets are identical. > > > I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. > This has to percolate quite a bit through the system -- the backends and the runtime. > > > And then this Here vs. LocalHere difference should fall away. > But still, what is it trying to do? > > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Jun 29 18:18:18 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 11:18:18 -0500 Subject: [M3devel] release 3.6, and history In-Reply-To: References: Message-ID: <5773F4CA.2030309@lcwb.coop> On 06/28/2016 11:08 PM, Jay K wrote: > I'm wondering: > - if anyone has the 3.6 release, can make it available? > - I can probably find it. We can put it on github? > > And more so, history prior to 3.6? And between 3.6 and 4.0? > > Or if Bill Kalsow is still around and available to ask questions? > > The Scanner thing is wierd to me. > > I do see some changes..maybe I did this.. > > old version: > PROCEDURE Here (VAR file: TEXT; VAR line: INTEGER) = > BEGIN > file := files [offset DIV MaxLines]; > line := offset MOD MaxLines; > END Here; > > PROCEDURE LocalHere (VAR file: TEXT; VAR line: INTEGER) = > VAR fnum := offset DIV MaxLines; > BEGIN > IF (local_files[fnum] = NIL) THEN > local_files[fnum] := Host.FileTail (files[fnum]); ^------------------------ This explains it. As currently coded, Here and LocalHere always just return the same VAR parameter results, with LocalHere having the side effect of copying the file name from array files to local_files. But local_files isn't used any other way, so the only effect is that a subsequent call to LocalHere, for the same file, computes the same "file" result slightly differently. But as in this old version, Here gives the full path and LocalHere gives a shortened one, as Host.FileTail computes. local_files just caches this, so it won't be recomputed. on a subsequent call. I think this should give you what you want. There are calls to LocalHere in CG, WebInfo, and InfoThisFile, plus several elsewhere to Here. So if you put the Host.FileTail call back, Here & LocalHere make sense and give you a choice at each place. > END; > file := local_files [fnum]; > line := offset MOD MaxLines; > END LocalHere; > > There is this div/mod relationship, but I still find it wierd to use div there. > You can see the FileTail which is maybe the change I want anyway. > > > Here, yes, me: > > > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3front/src/misc/Scanner.m3.diff?r1=1.5;r2=1.6 > > Revision 1.6: download - view: text, markup, annotated - select for diffs > Thu Feb 28 09:23:22 2008 MET (8 years, 4 months ago) by jkrell > Branches: MAIN > Diff to: previous 1.5: preferred, unified > Changes since revision 1.5: +1 -1 lines > put full paths to source files in debug info > > This has the minor downsides of: > 1) grows the debug info (it is already huge; who is counting?) > 2) reveals file system layout in debug info (privacy?) > 3) does it inhibit debugging files from other people's machines or does gdb dir still work? > > but definitely makes for a more pleasant debugging experience when > debugging stuff you have built yourself. > > The linear searching to see if a name has been allocated a number yet > will obviously slow way down due to a large increase in common prefixes, > but that should be a hash table anyway. Linear search is lame. > (or a trie, but working from the ends of the strings, minus the last one or few > characters, due to common prefixes as well as common suffixes) > > Note that both m3front and m3cc changes are needed as m3front has paths > relative to the current working directory or such. > > For most packages, you can get by without the m3front change and just prepend > "../src/" to the path in m3cc, but that doesn't work for hierarchical packages > such as libm3 and m3core which I am debugging. > > I still believe debug info should have full paths. > It always does on Windows for C and C++ (and this isn't a language thing). > It didn't always but they fixed it at some point. > > > But generic modules maybe something else -- like line directives and full paths back to the .mg files. > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 29 18:31:19 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 16:31:19 +0000 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: <5773F2BE.3050902@lcwb.coop> References: , <5773F2BE.3050902@lcwb.coop> Message-ID: My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. ?(in either case, the address space was 16 bit and ROM and periphal memory was in there, so various bank switching employed to gain access; later similar with the 128k RAM machines...) I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine as you suggest). I know that 32bits is overkill for a line number. I also know 16 bits isn't overkill -- but more than 16 are used here so ok. There is/was a warning in the Visual C++ compiler about truncating line numbers or terminating debug information after 64k lines. Midl output would trigger it. I still don't follow completely. It seems there is an aliasing situation, where lines very far apart can be deemed the same. I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... In general you have to balance: ?1 System still compilable on 32bit.? ?2 vs. 64bit system can do more? For the first case, you want to limit integers to 32bits and for the second you do not. Also convenience and perf suggest *not* having to sprinkle div/mod all around, though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... ?> FileNoAndLineNo Lately this is called "location" and things even have starts and ends, so the error messages can output a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill but clang is there and I think gcc went there. Yes the data is larger. Maybe shorter FileAndLine? I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) ?- Jay ---------------------------------------- > Date: Wed, 29 Jun 2016 11:09:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3front scanner div wierdness? > > > > On 06/28/2016 10:54 PM, Jay K wrote: >> Does anyone understand this stuff in m3front/Scanner.m3: >> >> Here vs. LocalHere? >> SameFile? >> >> >> I understand only this nuance: >> offset MOD MaxLines >> >> MaxLines = 100000; >> >> >> is to crudely handle that when asserts fail, >> they pack the line number in with the assertion failure code, >> potentially loosing bits. >> >> >> I don't think this is a good design, they should just be separate INTEGERS, >> but this is besides the point. >> > > This is just pure speculation, but I am very confident of it. These > offsets have a very high occurrence count. There is code all over m3front > that copies Scanner.offset into various data structures. So the small space > saving of one INTEGER instead of two would be multiplied by a big number. > I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. > Today, I have 8 Gig in the one I bought, and could easily afford more, if I > thought I needed it. > > Two integers would be cleaner, but this design is not too bad *if* you know > the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one > field. As pure documentation, there really should be a distinct type name > (say FileNoAndLineNo?) for all values that use this representation, even > though it just equates to INTEGER. There are a lot of variables lying around > all over the front end that use this invariant, but are just declared as > INTEGER. That's maintainer-hostile. > > >> >> What doesn't makes sense to me is the machinations around file name. >> >> >> Here: >> file := files [offset DIV MaxLines]; >> >> vs. LocalHere: >> file := local_files [fnum]; >> >> >> LocalHere makes sense. Here does not. >> >> >> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >> BEGIN >> RETURN (a DIV MaxLines) = (b DIV MaxLines); >> END SameFile; >> >> >> >> Shouldn't this just be a = b? >> > > As coded, this will return TRUE if a and b are different line numbers within > the same file. The name "SameFile" at least suggests that is what is intended. > A good example of a place where it would have been clearer if a & b were > declared as the type name I proposed above. > >> >> Well, anyway, this SameFile function isn't called. >> >> Here and LocalHere are used. >> >> >> I'm looking here because I want to add a temporary measure >> such that the file names are leaf-only. >> >> >> In particular, because generic modules have target names in their paths >> and I want to temporarly remove target names from output, so I can prove >> that a few targets are identical. >> >> >> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >> This has to percolate quite a bit through the system -- the backends and the runtime. >> >> >> And then this Here vs. LocalHere difference should fall away. >> But still, what is it trying to do? >> >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Wed Jun 29 18:34:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 16:34:06 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: References: , Message-ID: There is similar code in Module.m3 and that is the code producing the data "I don't like". M3Header isn't dead but I haven't seen it run yet. I changed them from "=>" to "1=>" and "2=>" and looked where those occur in the output .s files under -keep. This is kinda something I wish were easier, like more strings need to be longer for easier finding w/o ambiguity (the flip side of my context arguments!) As things stand, I can't check that in. I suppose maybe with a CG.comment but nevermind. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Subject: RE: [M3devel] source paths to generics? > Date: Wed, 29 Jun 2016 15:22:14 +0000 > > mfront/src/misc/M3Header.m3 has this: > > > (* build a synthetic file name & start reading *) > filename := old_filename & " => " & filename; > Scanner.Push (filename, s.generic, is_main := Scanner.in_main); > > > and instantiated generics aren't what I thought. > I thought the build system or compiler did a textual > substitution of the generic parameters into the entire implementation, > and saved that to the file system, > and had the assert/debug paths point to it. > > So could step through what looks exactly like an .m3 file. > > But it doesn't. The instantiated file is a little stub, like: > > jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 > (*generated by m3build*) > > MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. > > > so I have one or two proposals. > > 1) old_filename above contains the target, it looks like: > > "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" > or > "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" > > These both occur. > I'm not sure what the second is, seems like a mistake. > > I suggest the first string in both pairs be changed to be the leaf element, like: > > "TextAtomTbl.m3 => ../src/table/Table.mg" > or > "TextAtomTbl.m3 => ../src/table" > > and maybe the second string in both cases should be as it is in the second pair. > > 2) Possibly it should reall just be: > ../src/table/Table.mg > > and developer can step through that, knowing that it is a somewhat abstracted > form of what is running > > I'm willing to do this under like: > > CONST TemporaryForTargetConvergence = TRUE > > or > VAR TemporaryForTargetConvergence: BOOLEAN; > > and a command line switch, but I suspect it isn't really useful to anyone asis, > so might as well remove target unconditionally. > > > ? > > - Jay > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Date: Wed, 29 Jun 2016 09:44:51 +0000 >> Subject: [M3devel] source paths to generics? >> >> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >> the source file I am told about is the instantiated i3/m3 file. >> >> This isn't particularly useful for the programmer or convenient for the compiler, right? >> >> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >> I guess it is slightly easier in the compiler, but easy to do better? >> >> I should look into make it so? >> >> My real agenda is I want to see: >> ../src/foo.mg >> >> instead of >> I386_DARWIN/foo.m3 >> >> I don't want the target variation, but other points seem true also, right? >> >> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >> >> I'll try to poke around more. >> >> - Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From rodney_bates at lcwb.coop Wed Jun 29 19:01:08 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 12:01:08 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: Message-ID: <5773FED4.7020509@lcwb.coop> On 06/29/2016 04:44 AM, Jay K wrote: > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > the source file I am told about is the instantiated i3/m3 file. > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > I guess it is slightly easier in the compiler, but easy to do better? Yes. m3gdb does this, in many cases: ------------------------------------------------------------------------------------------------------------ (m3gdb) b VarArray.mg:136 Breakpoint 2 at 0x4073c3: file ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg, line 136. (m3gdb) c Continuing. Breakpoint 2, New (InitElemValue=-9223372036854775808, InitialAlloc= RECORD Lo = 9223372036854775807; Hi = -9223372036854775808; END, ExpansionFactor=1.10000002) at ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg:136 136 LExpansionFactorR := MAX ( ExpansionFactor , MinExpansionFactorR ) (m3gdb) ------------------------------------------------------------------------------------------------------------ It has a bug though. Setting a breakpoint to a .mg file before any execution has started appears to work, but the breakpoint won't trigger. I have not looked at cases like runtime errors without m3gdb. This does raise the question, however, that you might very well want to distinguish different instantiations of the same generic when setting a breakpoint. This is a good example of where a debugger needs to have awareness of your language. Modula-3 instantiations, being separate compilation units and having separate generic and instantiation files is a model that, AFAIK, does not occur in other languages. > > I should look into make it so? > > My real agenda is I want to see: > ../src/foo.mg > > instead of > I386_DARWIN/foo.m3 > > I don't want the target variation, but other points seem true also, right? > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > I'll try to poke around more. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 29 20:10:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 18:10:50 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <5773FED4.7020509@lcwb.coop> References: , <5773FED4.7020509@lcwb.coop> Message-ID: Good point, and C++ systems do handle this.When you step into std::vector::foo, the debugger knows that the source line occurs many places and can prompt you for which you want,or it can assume you mean the current one. If you haven't yet stepped into it, it can prompt for which. Now, 1) this stuff is often inlined and 2) std::vector and std::vector, usually end up being identical code, so you can't break on them separately. The linker does these optimization, and some compilers.Even Modula-3 is subject to this optimization -- since the linker does it. It is *mostly* good, mostly just makes things smaller, but it also confuses a lot of people. The linker I believe has to be a little pessimistic, like if you take the address of a function, that can inhibit optimization, lest someone is comparing function pointers for equality, which is rare and frowned upon, but is sort of in the languages. it also has to get function comparison correct, which is a little tricky, what with functions referencing data and other functions. m3gdb is interpreting these strings I guess. what is with the strings that have only a directory and not a file name on the right side -- in M3Header.m3. If I make the left hand side just a leaf name, foo.m3, w/o ../AMD64_LINUX, that'll be nearly equivalent?i.e. it is assuming current directory is already AMD64_LINUX, or it is assuming src? See, funny thing is, the current paths resolve the same either way, but if I remove "../AMD64_LINUX", they don't. - Jay > Date: Wed, 29 Jun 2016 12:01:08 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > > > On 06/29/2016 04:44 AM, Jay K wrote: > > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > > the source file I am told about is the instantiated i3/m3 file. > > > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > > I guess it is slightly easier in the compiler, but easy to do better? > > Yes. m3gdb does this, in many cases: > ------------------------------------------------------------------------------------------------------------ > (m3gdb) b VarArray.mg:136 > Breakpoint 2 at 0x4073c3: file ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg, line 136. > (m3gdb) c > Continuing. > > Breakpoint 2, New (InitElemValue=-9223372036854775808, InitialAlloc= > RECORD Lo = 9223372036854775807; Hi = -9223372036854775808; END, ExpansionFactor=1.10000002) > at ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg:136 > 136 LExpansionFactorR := MAX ( ExpansionFactor , MinExpansionFactorR ) > (m3gdb) > > ------------------------------------------------------------------------------------------------------------ > > It has a bug though. Setting a breakpoint to a .mg file before any execution has > started appears to work, but the breakpoint won't trigger. > > I have not looked at cases like runtime errors without m3gdb. > > This does raise the question, however, that you might very well want to distinguish > different instantiations of the same generic when setting a breakpoint. > > This is a good example of where a debugger needs to have awareness of your language. > Modula-3 instantiations, being separate compilation units and having separate > generic and instantiation files is a model that, AFAIK, does not occur in other > languages. > > > > > I should look into make it so? > > > > My real agenda is I want to see: > > ../src/foo.mg > > > > instead of > > I386_DARWIN/foo.m3 > > > > I don't want the target variation, but other points seem true also, right? > > > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > > > I'll try to poke around more. > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 30 02:34:13 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 19:34:13 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: , Message-ID: <57746905.2080007@lcwb.coop> Hmm. I poked around a bit, and it looks like there is some cruft here, left over from something. The only call I can find on M3Header.Parse is M3Front.m3:37. This is in M3Front.ParseImports. I can find no calls on that at all. (There are other procedures by this name.) Also, it appears M3Header.ParseImports actually collects the exports, the imports, the generic actuals. As for the concocted (by M3Header) name of the form => , this looks to be used only to initialize the scanner's file number while scanning up through the formals of the generic, but that is not used. On 06/29/2016 11:34 AM, Jay K wrote: > There is similar code in Module.m3 and that is the code producing > the data "I don't like". > > M3Header isn't dead but I haven't seen it run yet. > > I changed them from "=>" to "1=>" and "2=>" and looked > where those occur in the output .s files under -keep. > > This is kinda something I wish were easier, like more strings > need to be longer for easier finding w/o ambiguity (the flip > side of my context arguments!) > > As things stand, I can't check that in. > I suppose maybe with a CG.comment but nevermind. > > - Jay > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Subject: RE: [M3devel] source paths to generics? >> Date: Wed, 29 Jun 2016 15:22:14 +0000 >> >> mfront/src/misc/M3Header.m3 has this: >> >> >> (* build a synthetic file name & start reading *) >> filename := old_filename & " => " & filename; >> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >> >> >> and instantiated generics aren't what I thought. >> I thought the build system or compiler did a textual >> substitution of the generic parameters into the entire implementation, >> and saved that to the file system, >> and had the assert/debug paths point to it. >> >> So could step through what looks exactly like an .m3 file. >> >> But it doesn't. The instantiated file is a little stub, like: >> >> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >> (*generated by m3build*) >> >> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >> >> >> so I have one or two proposals. >> >> 1) old_filename above contains the target, it looks like: >> >> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >> or >> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >> >> These both occur. >> I'm not sure what the second is, seems like a mistake. >> >> I suggest the first string in both pairs be changed to be the leaf element, like: >> >> "TextAtomTbl.m3 => ../src/table/Table.mg" >> or >> "TextAtomTbl.m3 => ../src/table" >> >> and maybe the second string in both cases should be as it is in the second pair. >> >> 2) Possibly it should reall just be: >> ../src/table/Table.mg >> >> and developer can step through that, knowing that it is a somewhat abstracted >> form of what is running >> >> I'm willing to do this under like: >> >> CONST TemporaryForTargetConvergence = TRUE >> >> or >> VAR TemporaryForTargetConvergence: BOOLEAN; >> >> and a command line switch, but I suspect it isn't really useful to anyone asis, >> so might as well remove target unconditionally. >> >> >> ? >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>> Subject: [M3devel] source paths to generics? >>> >>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>> the source file I am told about is the instantiated i3/m3 file. >>> >>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>> >>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>> I guess it is slightly easier in the compiler, but easy to do better? >>> >>> I should look into make it so? >>> >>> My real agenda is I want to see: >>> ../src/foo.mg >>> >>> instead of >>> I386_DARWIN/foo.m3 >>> >>> I don't want the target variation, but other points seem true also, right? >>> >>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>> >>> I'll try to poke around more. >>> >>> - Jay >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 06:49:00 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:49:00 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <57746905.2080007@lcwb.coop> References: , , , , <57746905.2080007@lcwb.coop> Message-ID: I guess neither of us looked outside of m3front. The code runs. Not clear if the strings can get output. Maybe from asserts or Compiler.ThisFile in a generic? I"ll try some more. jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 ? ? ids := M3Front.ParseImports (source, s.m3env); ?- Jay ---------------------------------------- > Date: Wed, 29 Jun 2016 19:34:13 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > Hmm. I poked around a bit, and it looks like there is some cruft here, > left over from something. > > The only call I can find on M3Header.Parse is M3Front.m3:37. This is > in M3Front.ParseImports. I can find no calls on that at all. (There > are other procedures by this name.) > > Also, it appears M3Header.ParseImports actually collects the exports, the imports, > the generic actuals. > > As for the concocted (by M3Header) name of the form => , > this looks to be used only to initialize the scanner's file number while scanning > up through the formals of the generic, but that is not used. > > On 06/29/2016 11:34 AM, Jay K wrote: >> There is similar code in Module.m3 and that is the code producing >> the data "I don't like". >> >> M3Header isn't dead but I haven't seen it run yet. >> >> I changed them from "=>" to "1=>" and "2=>" and looked >> where those occur in the output .s files under -keep. >> >> This is kinda something I wish were easier, like more strings >> need to be longer for easier finding w/o ambiguity (the flip >> side of my context arguments!) >> >> As things stand, I can't check that in. >> I suppose maybe with a CG.comment but nevermind. >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Subject: RE: [M3devel] source paths to generics? >>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>> >>> mfront/src/misc/M3Header.m3 has this: >>> >>> >>> (* build a synthetic file name & start reading *) >>> filename := old_filename & " => " & filename; >>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>> >>> >>> and instantiated generics aren't what I thought. >>> I thought the build system or compiler did a textual >>> substitution of the generic parameters into the entire implementation, >>> and saved that to the file system, >>> and had the assert/debug paths point to it. >>> >>> So could step through what looks exactly like an .m3 file. >>> >>> But it doesn't. The instantiated file is a little stub, like: >>> >>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>> (*generated by m3build*) >>> >>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>> >>> >>> so I have one or two proposals. >>> >>> 1) old_filename above contains the target, it looks like: >>> >>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>> or >>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>> >>> These both occur. >>> I'm not sure what the second is, seems like a mistake. >>> >>> I suggest the first string in both pairs be changed to be the leaf element, like: >>> >>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>> or >>> "TextAtomTbl.m3 => ../src/table" >>> >>> and maybe the second string in both cases should be as it is in the second pair. >>> >>> 2) Possibly it should reall just be: >>> ../src/table/Table.mg >>> >>> and developer can step through that, knowing that it is a somewhat abstracted >>> form of what is running >>> >>> I'm willing to do this under like: >>> >>> CONST TemporaryForTargetConvergence = TRUE >>> >>> or >>> VAR TemporaryForTargetConvergence: BOOLEAN; >>> >>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>> so might as well remove target unconditionally. >>> >>> >>> ? >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>> Subject: [M3devel] source paths to generics? >>>> >>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>> the source file I am told about is the instantiated i3/m3 file. >>>> >>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>> >>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>> I guess it is slightly easier in the compiler, but easy to do better? >>>> >>>> I should look into make it so? >>>> >>>> My real agenda is I want to see: >>>> ../src/foo.mg >>>> >>>> instead of >>>> I386_DARWIN/foo.m3 >>>> >>>> I don't want the target variation, but other points seem true also, right? >>>> >>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>> >>>> I'll try to poke around more. >>>> >>>> - Jay >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 06:52:30 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:52:30 +0000 Subject: [M3devel] atomic.mg Message-ID: diff --git a/m3-libs/m3core/src/m3makefile b/m3-libs/m3core/src/m3makefile index 7efc97d..e371499 100644 --- a/m3-libs/m3core/src/m3makefile +++ b/m3-libs/m3core/src/m3makefile @@ -52,7 +52,7 @@ include_dir ("main") ?include_dir ("weakref") ?include_dir ("word") ?include_dir ("types") -% include_dir ("atomic")? DISABLE UNTIL I CHECK IN THE COMPILER FRONT-END FIXES -- Tony +include_dir ("atomic") ? ?% m3_option ("-times") ? "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** bad stack:? expected [ Int32 Int32 Addr ? ] got [ Int32 Addr? Addr ? ] "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** non-empty stack: [ Int32]**NIL** might be good to finish this up. hey, that is funny, now I see more of what those strings are for. The "2" in "2=>" is a local edit. ?- Jay From jay.krell at cornell.edu Thu Jun 30 06:58:58 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:58:58 +0000 Subject: [M3devel] M3Mid/M3Middle? Message-ID: I need a place to put stuff shared by backend and frontend. I believe the package for that is m3middle. But there is no miscelleanous module. M3Misc? M3Mid? M3Middle? The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. M3Middle ok? None of the existing modules in the package seem right. Thank you, ?- Jay From jay.krell at cornell.edu Thu Jun 30 07:04:05 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:04:05 +0000 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: References: Message-ID: Alternative that I don't really like is: CONST BOOLEAN TempFoo; in multiple places that need to be in sync but alternative for temporary purposes I don't mind is just to put it in Target. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Thu, 30 Jun 2016 04:58:58 +0000 > Subject: [M3devel] M3Mid/M3Middle? > > I need a place to put stuff shared by backend and frontend. > I believe the package for that is m3middle. > > But there is no miscelleanous module. > > M3Misc? > M3Mid? > M3Middle? > > The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. > > M3Middle ok? > > None of the existing modules in the package seem right. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 07:24:03 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:24:03 +0000 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: , , <5773F2BE.3050902@lcwb.coop>, Message-ID: I think I get it now. I was/am missing some of the lines but I can imagine how it works. There are about 15 bits given to the file number and about 17 bits given to the line number. On a 32bit system. A file with more than 100,000 lines might have trouble and a package with more than 16,000 files might have trouble. I know 64,000 lines in generated .c files is not unheard of. I don't know what file counts are on the high end. It is almost a good place for a LONGINT, but TYPE SourceLocation = RECORD ? file, line: INTEGER or INT32 := 0; END? Ok to do this at some point? ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Date: Wed, 29 Jun 2016 16:31:19 +0000 > Subject: Re: [M3devel] m3front scanner div wierdness? > > My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. > (in either case, the address space was 16 bit and ROM and periphal memory was in there, > so various bank switching employed to gain access; later similar with the 128k RAM machines...) > > > I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine > as you suggest). > > > I know that 32bits is overkill for a line number. > I also know 16 bits isn't overkill -- but more than 16 are used here so ok. > There is/was a warning in the Visual C++ compiler about truncating line numbers > or terminating debug information after 64k lines. Midl output would trigger it. > > I still don't follow completely. > It seems there is an aliasing situation, where lines very far apart can be deemed the same. > > I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. > Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... > In general you have to balance: > 1 System still compilable on 32bit. > 2 vs. 64bit system can do more > > For the first case, you want to limit integers to 32bits and for the second you do not. > > Also convenience and perf suggest *not* having to sprinkle div/mod all around, > though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... > >> FileNoAndLineNo > > Lately this is called "location" and things even have starts and ends, so the error messages can output > a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill > but clang is there and I think gcc went there. Yes the data is larger. > > Maybe shorter FileAndLine? > I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) > > - Jay > > > ---------------------------------------- >> Date: Wed, 29 Jun 2016 11:09:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] m3front scanner div wierdness? >> >> >> >> On 06/28/2016 10:54 PM, Jay K wrote: >>> Does anyone understand this stuff in m3front/Scanner.m3: >>> >>> Here vs. LocalHere? >>> SameFile? >>> >>> >>> I understand only this nuance: >>> offset MOD MaxLines >>> >>> MaxLines = 100000; >>> >>> >>> is to crudely handle that when asserts fail, >>> they pack the line number in with the assertion failure code, >>> potentially loosing bits. >>> >>> >>> I don't think this is a good design, they should just be separate INTEGERS, >>> but this is besides the point. >>> >> >> This is just pure speculation, but I am very confident of it. These >> offsets have a very high occurrence count. There is code all over m3front >> that copies Scanner.offset into various data structures. So the small space >> saving of one INTEGER instead of two would be multiplied by a big number. >> I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. >> Today, I have 8 Gig in the one I bought, and could easily afford more, if I >> thought I needed it. >> >> Two integers would be cleaner, but this design is not too bad *if* you know >> the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one >> field. As pure documentation, there really should be a distinct type name >> (say FileNoAndLineNo?) for all values that use this representation, even >> though it just equates to INTEGER. There are a lot of variables lying around >> all over the front end that use this invariant, but are just declared as >> INTEGER. That's maintainer-hostile. >> >> >>> >>> What doesn't makes sense to me is the machinations around file name. >>> >>> >>> Here: >>> file := files [offset DIV MaxLines]; >>> >>> vs. LocalHere: >>> file := local_files [fnum]; >>> >>> >>> LocalHere makes sense. Here does not. >>> >>> >>> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >>> BEGIN >>> RETURN (a DIV MaxLines) = (b DIV MaxLines); >>> END SameFile; >>> >>> >>> >>> Shouldn't this just be a = b? >>> >> >> As coded, this will return TRUE if a and b are different line numbers within >> the same file. The name "SameFile" at least suggests that is what is intended. >> A good example of a place where it would have been clearer if a & b were >> declared as the type name I proposed above. >> >>> >>> Well, anyway, this SameFile function isn't called. >>> >>> Here and LocalHere are used. >>> >>> >>> I'm looking here because I want to add a temporary measure >>> such that the file names are leaf-only. >>> >>> >>> In particular, because generic modules have target names in their paths >>> and I want to temporarly remove target names from output, so I can prove >>> that a few targets are identical. >>> >>> >>> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >>> This has to percolate quite a bit through the system -- the backends and the runtime. >>> >>> >>> And then this Here vs. LocalHere difference should fall away. >>> But still, what is it trying to do? >>> >>> >>> Thank you, >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 07:28:00 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:28:00 +0000 Subject: [M3devel] atomic.mg In-Reply-To: <958EAC02-79C4-4051-9144-99AB17CAF0E3@anu.edu.au> References: , <958EAC02-79C4-4051-9144-99AB17CAF0E3@anu.edu.au> Message-ID: If you do get time, one of my top requests is cooperative suspend. This will have a few benefits: ? - remove a bunch of system-specific code? ? - improve portability ? ? - make NT386-on-amd64 much more trustworthy ?(SuspendThread + GetThreadContext are wonky there)? ? - make PPC_DARWIN on x86 DARWIN probably work where today it always fails (granted, obsolete system and never remotely worked) ?? ? - not break into debugger due to the signals ?? On the other hand, I realize the primitives we are after are "SuspendThread" and "GetThreadContext" and they might exist widely. I didn't realize what was being attempted before. I might also get time in the next few months for a bunch of stuff. :) ? - Jay ---------------------------------------- > Subject: Re: atomic.mg > From: antony.hosking at anu.edu.au > Date: Thu, 30 Jun 2016 15:11:03 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I may actually have some time for this in the next few months! > > Tony Hosking, Professor > Research School of Computer Science > The Australian National University > >> On 30 Jun 2016, at 2:52 PM, Jay K wrote: >> >> diff --git a/m3-libs/m3core/src/m3makefile b/m3-libs/m3core/src/m3makefile >> index 7efc97d..e371499 100644 >> --- a/m3-libs/m3core/src/m3makefile >> +++ b/m3-libs/m3core/src/m3makefile >> @@ -52,7 +52,7 @@ include_dir ("main") >> include_dir ("weakref") >> include_dir ("word") >> include_dir ("types") >> -% include_dir ("atomic") DISABLE UNTIL I CHECK IN THE COMPILER FRONT-END FIXES -- Tony >> +include_dir ("atomic") >> >> % m3_option ("-times") >> >> >> "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** bad stack: expected [ Int32 Int32 Addr ] got [ Int32 Addr Addr ] >> "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** non-empty stack: [ Int32]**NIL** >> >> >> might be good to finish this up. >> >> hey, that is funny, now I see more of what those strings are for. >> >> The "2" in "2=>" is a local edit. >> >> - Jay > From rodney_bates at lcwb.coop Thu Jun 30 17:53:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 10:53:34 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: , , , , <57746905.2080007@lcwb.coop> Message-ID: <5775407E.6000401@lcwb.coop> On 06/29/2016 11:49 PM, Jay K wrote: > I guess neither of us looked outside of m3front. > The code runs. Not clear if the strings can get output. > Maybe from asserts or Compiler.ThisFile in a generic? > I"ll try some more. > I started looking in all of m3-sys, since the different compiler packages there are linked together into cm3. Later, I looked in the entire cm3 repo and didn't see anything. In any case, since it's part of the compiler, it really would be strange to call it from outside m3-sys. But if it is executing, we must have missed something. How did you find it executing? In a debugger? Can you do a backtrace? > jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 > ids := M3Front.ParseImports (source, s.m3env); > > > - Jay > > > ---------------------------------------- >> Date: Wed, 29 Jun 2016 19:34:13 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] source paths to generics? >> >> Hmm. I poked around a bit, and it looks like there is some cruft here, >> left over from something. >> >> The only call I can find on M3Header.Parse is M3Front.m3:37. This is >> in M3Front.ParseImports. I can find no calls on that at all. (There >> are other procedures by this name.) >> >> Also, it appears M3Header.ParseImports actually collects the exports, the imports, >> the generic actuals. >> >> As for the concocted (by M3Header) name of the form => , >> this looks to be used only to initialize the scanner's file number while scanning >> up through the formals of the generic, but that is not used. >> >> On 06/29/2016 11:34 AM, Jay K wrote: >>> There is similar code in Module.m3 and that is the code producing >>> the data "I don't like". >>> >>> M3Header isn't dead but I haven't seen it run yet. >>> >>> I changed them from "=>" to "1=>" and "2=>" and looked >>> where those occur in the output .s files under -keep. >>> >>> This is kinda something I wish were easier, like more strings >>> need to be longer for easier finding w/o ambiguity (the flip >>> side of my context arguments!) >>> >>> As things stand, I can't check that in. >>> I suppose maybe with a CG.comment but nevermind. >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Subject: RE: [M3devel] source paths to generics? >>>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>>> >>>> mfront/src/misc/M3Header.m3 has this: >>>> >>>> >>>> (* build a synthetic file name & start reading *) >>>> filename := old_filename & " => " & filename; >>>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>>> >>>> >>>> and instantiated generics aren't what I thought. >>>> I thought the build system or compiler did a textual >>>> substitution of the generic parameters into the entire implementation, >>>> and saved that to the file system, >>>> and had the assert/debug paths point to it. >>>> >>>> So could step through what looks exactly like an .m3 file. >>>> >>>> But it doesn't. The instantiated file is a little stub, like: >>>> >>>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>>> (*generated by m3build*) >>>> >>>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>>> >>>> >>>> so I have one or two proposals. >>>> >>>> 1) old_filename above contains the target, it looks like: >>>> >>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>>> or >>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>>> >>>> These both occur. >>>> I'm not sure what the second is, seems like a mistake. >>>> >>>> I suggest the first string in both pairs be changed to be the leaf element, like: >>>> >>>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>>> or >>>> "TextAtomTbl.m3 => ../src/table" >>>> >>>> and maybe the second string in both cases should be as it is in the second pair. >>>> >>>> 2) Possibly it should reall just be: >>>> ../src/table/Table.mg >>>> >>>> and developer can step through that, knowing that it is a somewhat abstracted >>>> form of what is running >>>> >>>> I'm willing to do this under like: >>>> >>>> CONST TemporaryForTargetConvergence = TRUE >>>> >>>> or >>>> VAR TemporaryForTargetConvergence: BOOLEAN; >>>> >>>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>>> so might as well remove target unconditionally. >>>> >>>> >>>> ? >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: m3devel at elegosoft.com >>>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>>> Subject: [M3devel] source paths to generics? >>>>> >>>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>>> the source file I am told about is the instantiated i3/m3 file. >>>>> >>>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>>> >>>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>>> I guess it is slightly easier in the compiler, but easy to do better? >>>>> >>>>> I should look into make it so? >>>>> >>>>> My real agenda is I want to see: >>>>> ../src/foo.mg >>>>> >>>>> instead of >>>>> I386_DARWIN/foo.m3 >>>>> >>>>> I don't want the target variation, but other points seem true also, right? >>>>> >>>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>>> >>>>> I'll try to poke around more. >>>>> >>>>> - Jay >>>>> >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Jun 30 17:58:24 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 10:58:24 -0500 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: References: Message-ID: <577541A0.8050109@lcwb.coop> The compiler's being broken into several packages sometimes makes it hard to get info from where it is known to where it is needed. I did something kludgy about size of WIDECHAR. I don't remember details right now. M3Middle sounds OK. Is m3middle the lowest package in the compiler? On 06/29/2016 11:58 PM, Jay K wrote: > I need a place to put stuff shared by backend and frontend. > I believe the package for that is m3middle. > > But there is no miscelleanous module. > > M3Misc? > M3Mid? > M3Middle? > > The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. > > M3Middle ok? > > None of the existing modules in the package seem right. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 18:10:05 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:10:05 +0000 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: <577541A0.8050109@lcwb.coop> References: , <577541A0.8050109@lcwb.coop> Message-ID: My understand is not that it is the "lowest", but that it is shared between m3front and m3back. I share your frustration, but I also suspect the design is fairly sound. I'll send a larger proposal shortly. ?- Jay ---------------------------------------- > Date: Thu, 30 Jun 2016 10:58:24 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] M3Mid/M3Middle? > > The compiler's being broken into several packages sometimes makes > it hard to get info from where it is known to where it is needed. > I did something kludgy about size of WIDECHAR. I don't remember > details right now. M3Middle sounds OK. Is m3middle the lowest > package in the compiler? > > On 06/29/2016 11:58 PM, Jay K wrote: >> I need a place to put stuff shared by backend and frontend. >> I believe the package for that is m3middle. >> >> But there is no miscelleanous module. >> >> M3Misc? >> M3Mid? >> M3Middle? >> >> The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. >> >> M3Middle ok? >> >> None of the existing modules in the package seem right. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 18:11:39 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:11:39 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <5775407E.6000401@lcwb.coop> References: , , , , , , , <57746905.2080007@lcwb.coop> ,<5775407E.6000401@lcwb.coop> Message-ID: I put in a bunch of calls to RTIO.PutText("I'm here 1"); RTIO.Flush(), RTIO.PutText("I'm here 2"); RTIO.Flush(). When I saw those I knew I missed it somehow and looked again. I still don't see the strings in the output though. ?- Jay ---------------------------------------- > Date: Thu, 30 Jun 2016 10:53:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > > > On 06/29/2016 11:49 PM, Jay K wrote: >> I guess neither of us looked outside of m3front. >> The code runs. Not clear if the strings can get output. >> Maybe from asserts or Compiler.ThisFile in a generic? >> I"ll try some more. >> > > I started looking in all of m3-sys, since the different compiler > packages there are linked together into cm3. Later, I > looked in the entire cm3 repo and didn't see anything. > In any case, since it's part of the compiler, it really would be > strange to call it from outside m3-sys. > > But if it is executing, we must have missed something. How did you > find it executing? In a debugger? Can you do a backtrace? > >> jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 >> ids := M3Front.ParseImports (source, s.m3env); >> >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Wed, 29 Jun 2016 19:34:13 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] source paths to generics? >>> >>> Hmm. I poked around a bit, and it looks like there is some cruft here, >>> left over from something. >>> >>> The only call I can find on M3Header.Parse is M3Front.m3:37. This is >>> in M3Front.ParseImports. I can find no calls on that at all. (There >>> are other procedures by this name.) >>> >>> Also, it appears M3Header.ParseImports actually collects the exports, the imports, >>> the generic actuals. >>> >>> As for the concocted (by M3Header) name of the form => , >>> this looks to be used only to initialize the scanner's file number while scanning >>> up through the formals of the generic, but that is not used. >>> >>> On 06/29/2016 11:34 AM, Jay K wrote: >>>> There is similar code in Module.m3 and that is the code producing >>>> the data "I don't like". >>>> >>>> M3Header isn't dead but I haven't seen it run yet. >>>> >>>> I changed them from "=>" to "1=>" and "2=>" and looked >>>> where those occur in the output .s files under -keep. >>>> >>>> This is kinda something I wish were easier, like more strings >>>> need to be longer for easier finding w/o ambiguity (the flip >>>> side of my context arguments!) >>>> >>>> As things stand, I can't check that in. >>>> I suppose maybe with a CG.comment but nevermind. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: m3devel at elegosoft.com >>>>> Subject: RE: [M3devel] source paths to generics? >>>>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>>>> >>>>> mfront/src/misc/M3Header.m3 has this: >>>>> >>>>> >>>>> (* build a synthetic file name & start reading *) >>>>> filename := old_filename & " => " & filename; >>>>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>>>> >>>>> >>>>> and instantiated generics aren't what I thought. >>>>> I thought the build system or compiler did a textual >>>>> substitution of the generic parameters into the entire implementation, >>>>> and saved that to the file system, >>>>> and had the assert/debug paths point to it. >>>>> >>>>> So could step through what looks exactly like an .m3 file. >>>>> >>>>> But it doesn't. The instantiated file is a little stub, like: >>>>> >>>>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>>>> (*generated by m3build*) >>>>> >>>>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>>>> >>>>> >>>>> so I have one or two proposals. >>>>> >>>>> 1) old_filename above contains the target, it looks like: >>>>> >>>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>>>> or >>>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>>>> >>>>> These both occur. >>>>> I'm not sure what the second is, seems like a mistake. >>>>> >>>>> I suggest the first string in both pairs be changed to be the leaf element, like: >>>>> >>>>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>>>> or >>>>> "TextAtomTbl.m3 => ../src/table" >>>>> >>>>> and maybe the second string in both cases should be as it is in the second pair. >>>>> >>>>> 2) Possibly it should reall just be: >>>>> ../src/table/Table.mg >>>>> >>>>> and developer can step through that, knowing that it is a somewhat abstracted >>>>> form of what is running >>>>> >>>>> I'm willing to do this under like: >>>>> >>>>> CONST TemporaryForTargetConvergence = TRUE >>>>> >>>>> or >>>>> VAR TemporaryForTargetConvergence: BOOLEAN; >>>>> >>>>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>>>> so might as well remove target unconditionally. >>>>> >>>>> >>>>> ? >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> From: jay.krell at cornell.edu >>>>>> To: m3devel at elegosoft.com >>>>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>>>> Subject: [M3devel] source paths to generics? >>>>>> >>>>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>>>> the source file I am told about is the instantiated i3/m3 file. >>>>>> >>>>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>>>> >>>>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>>>> I guess it is slightly easier in the compiler, but easy to do better? >>>>>> >>>>>> I should look into make it so? >>>>>> >>>>>> My real agenda is I want to see: >>>>>> ../src/foo.mg >>>>>> >>>>>> instead of >>>>>> I386_DARWIN/foo.m3 >>>>>> >>>>>> I don't want the target variation, but other points seem true also, right? >>>>>> >>>>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>>>> >>>>>> I'll try to poke around more. >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Thu Jun 30 18:12:11 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 11:12:11 -0500 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: , , <5773F2BE.3050902@lcwb.coop>, Message-ID: <577544DB.7050900@lcwb.coop> On 06/30/2016 12:24 AM, Jay K wrote: > I think I get it now. I was/am missing some of the lines but I can imagine how it works. > > There are about 15 bits given to the file number and about 17 bits given to the line number. > On a 32bit system. > > A file with more than 100,000 lines might have trouble and a package with more than 16,000 files might have trouble. > I know 64,000 lines in generated .c files is not unheard of. > I don't know what file counts are on the high end. > I think 100,000 lines could be a bit marginal for mechanically generated code, but the file count space is probably over generous. I presume they made it a power of 10 so a human could do the DIV or MOD visually on the decimal value. Increasing to a million would still give 2147 files. > It is almost a good place for a LONGINT, but > TYPE SourceLocation = RECORD > file, line: INTEGER or INT32 := 0; > END? > > Ok to do this at some point? It will be pervasive and truly algorithmic. An alternative would be to keep it an INTEGER, but use a distinct type name on all variables/fields that use the MOD/DIV invariant. Pervasive too, but no risk of runtime breakage. If you put just the one INTEGER inside a record with an unlikely name, that would preserve the space savings but still get type checking and still get the compiler to find all the places that need to change. > > - Jay > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Date: Wed, 29 Jun 2016 16:31:19 +0000 >> Subject: Re: [M3devel] m3front scanner div wierdness? >> >> My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. >> (in either case, the address space was 16 bit and ROM and periphal memory was in there, >> so various bank switching employed to gain access; later similar with the 128k RAM machines...) >> >> >> I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine >> as you suggest). >> >> >> I know that 32bits is overkill for a line number. >> I also know 16 bits isn't overkill -- but more than 16 are used here so ok. >> There is/was a warning in the Visual C++ compiler about truncating line numbers >> or terminating debug information after 64k lines. Midl output would trigger it. >> >> I still don't follow completely. >> It seems there is an aliasing situation, where lines very far apart can be deemed the same. >> >> I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. >> Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... >> In general you have to balance: >> 1 System still compilable on 32bit. >> 2 vs. 64bit system can do more >> >> For the first case, you want to limit integers to 32bits and for the second you do not. >> >> Also convenience and perf suggest *not* having to sprinkle div/mod all around, >> though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... >> >>> FileNoAndLineNo >> >> Lately this is called "location" and things even have starts and ends, so the error messages can output >> a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill >> but clang is there and I think gcc went there. Yes the data is larger. >> >> Maybe shorter FileAndLine? >> I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Wed, 29 Jun 2016 11:09:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] m3front scanner div wierdness? >>> >>> >>> >>> On 06/28/2016 10:54 PM, Jay K wrote: >>>> Does anyone understand this stuff in m3front/Scanner.m3: >>>> >>>> Here vs. LocalHere? >>>> SameFile? >>>> >>>> >>>> I understand only this nuance: >>>> offset MOD MaxLines >>>> >>>> MaxLines = 100000; >>>> >>>> >>>> is to crudely handle that when asserts fail, >>>> they pack the line number in with the assertion failure code, >>>> potentially loosing bits. >>>> >>>> >>>> I don't think this is a good design, they should just be separate INTEGERS, >>>> but this is besides the point. >>>> >>> >>> This is just pure speculation, but I am very confident of it. These >>> offsets have a very high occurrence count. There is code all over m3front >>> that copies Scanner.offset into various data structures. So the small space >>> saving of one INTEGER instead of two would be multiplied by a big number. >>> I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. >>> Today, I have 8 Gig in the one I bought, and could easily afford more, if I >>> thought I needed it. >>> >>> Two integers would be cleaner, but this design is not too bad *if* you know >>> the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one >>> field. As pure documentation, there really should be a distinct type name >>> (say FileNoAndLineNo?) for all values that use this representation, even >>> though it just equates to INTEGER. There are a lot of variables lying around >>> all over the front end that use this invariant, but are just declared as >>> INTEGER. That's maintainer-hostile. >>> >>> >>>> >>>> What doesn't makes sense to me is the machinations around file name. >>>> >>>> >>>> Here: >>>> file := files [offset DIV MaxLines]; >>>> >>>> vs. LocalHere: >>>> file := local_files [fnum]; >>>> >>>> >>>> LocalHere makes sense. Here does not. >>>> >>>> >>>> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >>>> BEGIN >>>> RETURN (a DIV MaxLines) = (b DIV MaxLines); >>>> END SameFile; >>>> >>>> >>>> >>>> Shouldn't this just be a = b? >>>> >>> >>> As coded, this will return TRUE if a and b are different line numbers within >>> the same file. The name "SameFile" at least suggests that is what is intended. >>> A good example of a place where it would have been clearer if a & b were >>> declared as the type name I proposed above. >>> >>>> >>>> Well, anyway, this SameFile function isn't called. >>>> >>>> Here and LocalHere are used. >>>> >>>> >>>> I'm looking here because I want to add a temporary measure >>>> such that the file names are leaf-only. >>>> >>>> >>>> In particular, because generic modules have target names in their paths >>>> and I want to temporarly remove target names from output, so I can prove >>>> that a few targets are identical. >>>> >>>> >>>> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >>>> This has to percolate quite a bit through the system -- the backends and the runtime. >>>> >>>> >>>> And then this Here vs. LocalHere difference should fall away. >>>> But still, what is it trying to do? >>>> >>>> >>>> Thank you, >>>> - Jay >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 18:17:22 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:17:22 +0000 Subject: [M3devel] removing occurences of target name output? Message-ID: Ok here is are some choices/proposal. ?- Do nothing -- or at least commit nothing. ?I have to do something -- my goal is to show that various ?targets are the same, with the C backend and possibly cm3cg. ? ?- add switch cm3 -trim-paths ?confusing name? ? ?add swich cm3 -omit-target confusing name? ? ?Whatever is this switch, add it to cm3g also. The same name. ? ?Add switch to scripts/python/pylib.py -cm3:xxx ? The xxx part is passed to cm3. ? ?e.g. do-pk.py -cm3:-omit-target ? ? ?The behavior of the switch: ? in cm3, it is passed to the backend somehow.? ? ? ?Either as a flag to the quake function, or ? ? ?by setting a quake global; somehow shield ? ? ?the LLVM backend.? ? ? ? ? Bring back the function m3front/Host.Tail. ? This historically finds the last forward slash or ? backward slash or space in a string, and returns everything ? after it. ? Modify it to check this new flag, and if the new flag isn't set, ? return the path unchanged. ?? ?? ? In m3cc, dbxout.c and dwarf2out.c, there is one place ? each that outputs the current working directory. ? If this switch is set, don't do that. ?? ?? ? In m3front/M3Header.m3 and Scanner.m3 where it makes up those ? strings with "=>" in them, pass the left hand side through ? Host.Tail. ?? ? In m3front Scanner.Here, if the flag is set, return ? the value of LocalHere instead. I think. I think that is ? required to remove more occurences. I can double check. ?? ?? ? In m3back/M3C.c there is one place that outputs the ? target in a comment. Either just remove that unconditionally, or ? remove it subject to this switch. ?? ?? ? That last point, and the actual goal, is why "trim-path" ? isn't an accurate name. It does so happen that "paths" ? are usually where target occurs in the output semi-unnecessarily. ?? ?? ? It could very well be that we do want this behavior in cm3cg long term, ? such that whoever makes a distribution, at whatever path, gets the same result. ?? ?I do not want to go the route of omitting debug info entirely, which also fixes this. ?? ? It would be nice if something could be specified symbolically. ? Such as wherever the expanded form of $CM3_TARGET occurs in a path ? in the output, change it back to $CM3_TARGET. Like how the ? .m3ship files are optionally generated and like I've seen ? in other compilers -- there is a goal in general to have ? some sort of ull path, but not require everyone to build ? with the same paths, and still get identical outputs. ?This somewhat requires debugger cooperation, depending on their source search algorithm. ? ?? ? When cm3 parses the command line, store this boolean in... Target.OmitPathss? ? Target.TrimTargetString? M3Middle.LessTargetDependentOutput?? ? M3Middle.TrimTarget? ?? ?? ? Obviously this is a minor tactic -- sizeof(integer) will for now ? continue to be endemic in the IR and the backend output. ?? ? The actual goal here is not to make one C output, but a small number. ? I intend soon to introduce the following targets, something like: ? ? POSIX32LE ? ? ? POSIX32BE ? ? ? POSIX64LE ? ? ? POSIX64BE ? ? ? WIN32LE? ? ? WIN64LE? ?C or C++ backend is implied, by lack of processor.? ? ? ? ?Or maybe: ? ? C32EL_POSIX ? ? C64EL_POSIX ? ? C32BE_POSIX ? ? C64BE_POSIX ? ? C32EL_WIN32 ? ? C64EL_WIN32 These 6 targets shall be the basis of a "universal" download/bootstrap, easier for people to get started with. ? ?Eventually the output will use C++ exception handling so I'm leary ?of backing the name "C" in too much. ? ?But I'm getting ahead of myself. ? ?This "trim target" is an important validation step for that. ? ?Ok in principle? ?Ok in detail? ?Or just do nothing? ? ?- Jay From jayk123 at hotmail.com Wed Jun 1 11:28:08 2016 From: jayk123 at hotmail.com (Jay K) Date: Wed, 1 Jun 2016 09:28:08 +0000 Subject: [M3devel] [modula3/cm3] New release for Mac OS X, please? (#13) [Originally From]: notifications@github.com In-Reply-To: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> References: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> Message-ID: I kinda want to encourage ?a different approach, at least for Unix systems where binary compatibility is currently so low. So here is what I did/am doing: ?I have made assembly and C bootstraps for I386_LINUX and AMD64_LINUX.? ?I'm still testing them (waiting for boot2 to finish). ? ? ?Here is roughly how:? ? For assembly (gcc-bassed ) bootstrap:? ?cd ${CM3_ROOT}/m3-sys/m3cc? ? ?sh -c ./src/buildmany.sh? ? That actually does way more than necessary,? ? and might fail in the end. I didn't wait for it.? ? In particular, it builds a gcc-based backend for every target.? ?If you look -- buildmany.sh is trivial.? ?m3cc/src/m3makefile has always had the support. buildmany.sh is a tiny wrapper.? ? Actually the first step should be redundant with boot1.py anyway. ? ? cd ${CM3_ROOT}/scripts/python? ?# I don't think these are case sensitive even.? ?./do-cm3-all.py I386_LINUX realclean? ?./do-cm3-all.py I386_LINUX c realclean? ?./do-cm3-all.py AMD64_LINUX realclean? ?./do-cm3-all.py AMD64_LINUX c realclean? ?./boot1.py I386_LINUX c? ?./boot1.py I386_LINUX ? ?./boot1.py AMD64_LINUX c? ?./boot1.py AMD64_LINUX ? If you look, boot1.py doesn't seem obviously simple, but?it really is just a wrapper around roughly: ?cd ? ?cm3? ?mkdir? ?cp ? ?tar? ? ? ?"All else being equal", i.e. lacking broad adoption of the C or LLVM backends, ?we should make releases "this way" "and then some". ? ? ?"and then some": ? ?all supported/tested targets (Darwin, Solaris, FreeBSD, maybe NT). Enhancing it to include the entire system, installing a proper install, and supporting shared libraries. Shared libraries: either autotools or cmake or replicating the config files ? i.e. into pylib.py, or moving pylib.py logic into cm3, esp. cm3 -boot ?? Key advantages of this approach: ?1) It is cross building, we can do it all on one machine (or scale it out, but ? ? it doesn't take too long)? ?2) More importantly, the results are less machine-specific. There are ? ? no paths to dynamic linked libraries or versions thereof. This is roughly how 3.6 was released -- along with a matching source tree for the entire system and good directions to build the multiple pieces. ?The directions today are like:? ? get the right bootstrap archive ? get the source tree ? extract bootstrap ? build it (make) (again, if you look, simple stuff)? ? install it ? ? mkdir /somewhere/bin? ? ? mv cm3 /somewhere/bin ? ? ? export PATH=/somewhere/bin:$PATH ? ? ? ./boot2.py ? ? ? or ? ? ?./boot2.py c to use the C backend ? ? ? ? ? ? ? ?(mklib is only for NT targets)? ?? ?? I'd really soon like there to be fewer C variations though -- just unixc32le, unixc32be, unixc64le, posix64be, ntc32le, ntc64le. And this is almost overkill -- 64le is the vast majority. ? - Jay ---------------------------------------- Date: Tue, 31 May 2016 08:28:32 -0700 From: notifications at github.com To: cm3 at noreply.github.com CC: jay.krell at cornell.edu; comment at noreply.github.com Subject: Re: [modula3/cm3] New release for Mac OS X, please? (#13) Yes, I saw the min and all tar.gz files yesterday. I extracted the "all" tar file and the compiler worked fine. (The tar.gz files are gone from my /var directory now.) I'd be happy to help with a release. What should I do? From jay.krell at cornell.edu Wed Jun 1 11:30:16 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 09:30:16 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? Does/can it retain its incrementality? Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > Date: Tue, 31 May 2016 17:18:46 +0000 > Subject: Re: [M3devel] replace build system?? > > We should probably learn how to get the number of processors and optionally system load and make this more automatic. > Posix should standardize more of this. :( > > - Jay > > > > > > > ---------------------------------------- >> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >> To: dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 10:16:05 -0700 >> From: mika at async.caltech.edu >> Subject: Re: [M3devel] replace build system?? >> >> I mentioned this before a few times on this mailing list.... CM3 is already fairly >> parallel if you turn on the right options. Every back-end invocation can be done >> in parallel. >> >> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >> >> "Daniel Alejandro Benavides D." writes: >> ... >>>A make-based build solution would have other advantages as well like=20 >>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>four times faster. >> ... >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Wed Jun 1 13:17:43 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 11:17:43 +0000 Subject: [M3devel] IR init_int with out of range values? Message-ID: Is this reasonable? m3-libs/m3core/src/float/IEEE-be/LongRealRep.i3 INTERFACE LongRealRep; (* This interface describes the layout of IEEE double precision reals ? ?on big endian machines *) TYPE ? T = RECORD? ? ? sign ? ? ? ? : BITS ?1 FOR [0..1] ? ? ? ? ? ? ? ? ? ? ? ? ?:= 0; ? ? exponent ? ? : BITS 11 FOR [0..16_7FF] ? ? ? ? ? ? ? ? ? ? := 0; ? ? significand0 : BITS 20 FOR [0..16_FFFFF] ? ? ? ? ? ? ? ? ? := 0; ? ? significand1 : BITS 32 FOR [-16_7fffffff-1 .. 16_7fffffff] := 0; ? END; CONST ? NegInf = T { sign := 1, exponent := 16_7FF }; ? PosInf = T { sign := 0, exponent := 16_7FF }; ? Nan ? ?= T { sign := 0, exponent := 16_7FF, significand0 := 1 }; END LongRealRep. begin_init v.1 # constant NegInf # constant PosInf init_int 0 18442240474082181120 Int.64 # constant Nan init_int 8 9218868437227405312 Int.64 My C backend doesn't like 18442240474082181120 Int.64, since 18442240474082181120 does not fit in a signed 64bit integer. Should the frontend be obligated to make it Word.64? I guess I can make it work. jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ edit 1.c jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ cc 1.c 1.c:5:15: warning: integer literal is too large to be represented in a signed integer type, interpreting as unsigned ? ? ? [-Wimplicitly-unsigned-literal] long long a = 18442240474082181120ll; ? ? ? ? ? ? ? ^ 1 warning generated. jair:python jay$ ./a.out 18442240474082181120 fff0000000000000 -4503599627370496 fff0000000000000 Thank you, ?- Jay From rodney_bates at lcwb.coop Thu Jun 2 00:08:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 01 Jun 2016 17:08:34 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: <574F5CE2.8080506@lcwb.coop> On 06/01/2016 04:30 AM, Jay K wrote: > We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. It doesn't, not very well. A deep principle, if you can call it that, of C is that each compilation has no knowledge of any other compilation. Likewise, when linking them together, it's all just binary stuff, addresses, constants, etc. To get a semblance of type-safe compilation, all coders have to correctly follow some variant of a header file/#include idiom and create a matching makefile that correctly reflects the actual interdependencies. There is no checking that these are done correctly, and from direct experience, I can say they are routinely done wrong. All the modular languages have direct linguistic support for type-safe separate compilation. Implementing this correctly while using the C compilation model lies somewhere between very kludgy and impossible. > > Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? It couldn't possibly. Typical front end compilation actual work is similar to or less than executable program startup/shutdown overhead. This effect might be partially lost in the fact that m3cc is a separate executable and it has to start up for every .[im]3 file. But it would still about double the number of program executions and startup overheads. > Does/can it retain its incrementality? I don't think this is possible all. No matter what is in the makefile, Make only understands dependencies among whole source files and must rebuild if a depended-on file has been touched, even no changes to content. Our current build system works on declaration granularity. (I presume this is what you mean by incrementality." I am not aware of any language-independent build infrastructure that keeps track of dependencies on other that source file granularity. > > Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler frontend, and language-specific link-time functions in one executable. I see no benefit and almost certainly losses, in putting these in different executables. > - Jay > > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 17:18:46 +0000 >> Subject: Re: [M3devel] replace build system?? >> >> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >> Posix should standardize more of this. :( >> >> - Jay >> >> >> >> >> >> >> ---------------------------------------- >>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>> To: dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 10:16:05 -0700 >>> From: mika at async.caltech.edu >>> Subject: Re: [M3devel] replace build system?? >>> >>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>> parallel if you turn on the right options. Every back-end invocation can be done >>> in parallel. >>> >>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>> >>> "Daniel Alejandro Benavides D." writes: >>> ... >>>> A make-based build solution would have other advantages as well like=20 >>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>> four times faster. >>> ... >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From microcode at zoho.com Thu Jun 2 08:08:47 2016 From: microcode at zoho.com (microcode at zoho.com) Date: Thu, 2 Jun 2016 06:08:47 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> Message-ID: <20160602060847.GA3051@zoho.com> On Wed, Jun 01, 2016 at 05:08:34PM -0500, Rodney M. Bates wrote: > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. Agreed with everything you wrote not withstanding my total lack of M3 knowledge. Ada is another language (possibly the first?) that specifies a lot of how it needs to be built in the language spec. It deals with the issues you mentioned (and more I think) like checking parameters and types across the system, interface compliance, incremental compilation etc. There is an open source version available so it should be possible to at least get a look how they are accomplishing those things and see if it could be scrounged, adapted etc. for M3. I would think they're very Ada-centric as these kinds of things seem to always turn out to be, but who knows. The Ada compiler is part of gcc, it's called gnat or gcc-ada. The specific top-level piece that is responsible for figuring out all the dependencies is called gnatmake. From jay.krell at cornell.edu Fri Jun 3 09:22:45 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 07:22:45 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: , ,,, ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: I understand the C model very well, but I'm not certain of all the ramifications for Modula-3. I understand C++ has limited checking via name mangling. And that other systems carry names/signatures forward to runtime and check them there. Even dynamically linked C++. And Modula-3. However function names aren't everything -- i.e. they don't include structural hashes of types. It is name-based, not structure-based. The most common mistake in the loose C systems is not doing things wrong per se, but accidentally mixing old and new versions, and the result is undefined/crash. We have the same problem with our "extern" mechanism -- how many times I debugged stat corrupting stack until I put in the extra layer. C++ systems are getting better and check for "ODR violations" at link time. But checking at link time kinda contradicts my question. Goal being to fit into "generic" build systems but C++ only starts winning when the linker knows about it -- when the system is C++ specific. If C++ requires C++ linker/driver, we shouldn't feel too bad requiring a Modula-3 linker/driver. ?> Typical front end compilation actual work is similar to? ?> or less than executable program startup/shutdown overhead.? ?Ideally this is not the case. Ideally process startup is just referencing ?pages already in file system cache. Demand paging, unified file system cache ?and memory manager and all that (all systems NT, Linux, etc. are the same ?these days). But I realize in reality there is alway some amount of other CPU ?to burn at each process start. The ideal is not quite reached. ?> m3cc is a separate executable..has to start up for every .[im]3 file. B ? ?I wonder if we can fix this btw. ?If we can start it just once per library. ?Process create really is fast on typical Posix systems, so people maybe ?don't notice. ? ?There are a few methods: ? ?- alway keep the m3cg ir files ? ?- if any out of date, read them all into one large IR and write out one large assembly file ? ?- I want do that anyway, to achieve a rich/poor man's LTCG/LTO. ? ? ?poor: No need to worry about linker/driver support.? ? ? ?rich: We can afford the memory these days. Though at some point it wouldn't scale.? ? - I want do similar for C backend, as it seems noticable either the compiler startup ? ? or the few #includes I sometimes use (though C++ modules promise some relief..) ? ?? ? ? Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them? ? separately as usual, but within one m3cg invocation.? ?? ?> Does/can it retain its incrementality? ? ?I think the best but not great we could do is make would decide pessmisically ?to run the compiler, but the compiler could figure out that the old/existing ?output is still correct. The world to decide up-to-date is significant but ?perhaps somewhat less than the work to just do all the work. ? ?> ?see no benefit and almost certainly losses, in putting these in different executables. ?? ? Less code to maintain. Easier to redistribute. ? I like the model of distribution of C source. ? ? ?wget source && tar xf && configure && make && make install ? I want something like that. ? I am uncertain of "redistribution" and "active development" must/should be different. ? i.e. We do have something like this, and the two ways of working cooexist. ?? ? ? ?- Jay ---------------------------------------- > Date: Wed, 1 Jun 2016 17:08:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > > > On 06/01/2016 04:30 AM, Jay K wrote: >> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > > It doesn't, not very well. A deep principle, if you can call it that, of C is that > each compilation has no knowledge of any other compilation. Likewise, when linking > them together, it's all just binary stuff, addresses, constants, etc. To get a > semblance of type-safe compilation, all coders have to correctly follow some variant > of a header file/#include idiom and create a matching makefile that correctly reflects > the actual interdependencies. There is no checking that these are done correctly, and > from direct experience, I can say they are routinely done wrong. > > All the modular languages have direct linguistic support for type-safe separate > compilation. Implementing this correctly while using the C compilation model lies > somewhere between very kludgy and impossible. > >> >> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > > It couldn't possibly. Typical front end compilation actual work is similar to > or less than executable program startup/shutdown overhead. This effect > might be partially lost in the fact that m3cc is a separate executable and it > has to start up for every .[im]3 file. But it would still about double the > number of program executions and startup overheads. > >> Does/can it retain its incrementality? > > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. > >> >> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >> > > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > frontend, and language-specific link-time functions in one executable. I see no > benefit and almost certainly losses, in putting these in different executables. > >> - Jay >> >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 17:18:46 +0000 >>> Subject: Re: [M3devel] replace build system?? >>> >>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>> Posix should standardize more of this. :( >>> >>> - Jay >>> >>> >>> >>> >>> >>> >>> ---------------------------------------- >>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>> To: dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>> From: mika at async.caltech.edu >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>> parallel if you turn on the right options. Every back-end invocation can be done >>>> in parallel. >>>> >>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>> >>>> "Daniel Alejandro Benavides D." writes: >>>> ... >>>>> A make-based build solution would have other advantages as well like=20 >>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>> four times faster. >>>> ... >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Fri Jun 3 16:44:12 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:44:12 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575197BC.1030100@lcwb.coop> BTW, 'though the current build system is considerably sophisticated, it does have some bugs, which we should fix. I have the exact setup & symptom for one of them written down somewhere. I think it's when adding a procedure signature to an interface when a procedure body for it was already present in a compiled exporting module, things don't get recompiled right. Also, I occasionally get recompile failures that I don't think are valid, and when I clean the package and recompile that, all is well. I haven't ever tracked down how to reproduce this or noted its exact symptom. We also don't keep track of compile options that affect generated code, so as to recompile when only the options change. E.g., optimization level, debug info presence and format. I did do this for the size of WIDECHAR, since mismatches here would be disastrous. On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>> in parallel. >>>>> >>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>> >>>>> "Daniel Alejandro Benavides D." writes: >>>>> ... >>>>>> A make-based build solution would have other advantages as well like=20 >>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>> four times faster. >>>>> ... >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 3 16:53:16 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:53:16 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575199DC.4060305@lcwb.coop> On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > I like this idea, but we don't want to force serialization where we now do parallel runs of m3cg. OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands seem to scroll by slower than the unit compiles when running cm3, despite the fact that the latter include a main program invocation of m3cc. Surely it's not merely smaller units? And we are also running separately too. > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > We could just wrap what we have in makefile that, almost trivially invokes cm3 with a few odd options. Or is that cheating? > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>> in parallel. >>>>> >>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>> >>>>> "Daniel Alejandro Benavides D." writes: >>>>> ... >>>>>> A make-based build solution would have other advantages as well like=20 >>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>> four times faster. >>>>> ... >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Fri Jun 3 19:18:07 2016 From: estellnb at elstel.org (Elmar Stellnberger) Date: Fri, 3 Jun 2016 19:18:07 +0200 Subject: [M3devel] replace build system?? In-Reply-To: <575199DC.4060305@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> Message-ID: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs >> and process them >> separately as usual, but within one m3cg invocation. >> > > I like this idea, but we don't want to force serialization where we now > do parallel > runs of m3cg. > > OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile > commands > seem to scroll by slower than the unit compiles when running cm3, > despite the fact > that the latter include a main program invocation of m3cc. Surely it's > not merely > smaller units? And we are also running separately too. > Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' that the backend would generate code in parallel? - This would not be right what I want because actually obtaining deterministic build results is of high value for security reasons. It can be necessary to verify whether a given compiler toolkit / build environment must have been infected at the stage of build time with hindsight. I know deterministic builds are quite hard to realize even for some C/C++ applications; nonetheless I had never been thinking about Modula-3 or CM3 in specific. At least it was a well known problem with Python that builds were non-deterministic due to parallel code generation. However that should not happen if just different source files are translated at the same time. Moreover that could also parallelize the activity of the front (and middle)-end. From rodney_bates at lcwb.coop Fri Jun 3 20:31:40 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 13:31:40 -0500 Subject: [M3devel] replace build system?? In-Reply-To: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> Message-ID: <5751CD0C.1070903@lcwb.coop> On 06/03/2016 12:18 PM, Elmar Stellnberger wrote: > >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs >>> and process them >>> separately as usual, but within one m3cg invocation. >>> >> >> I like this idea, but we don't want to force serialization where we now >> do parallel >> runs of m3cg. >> >> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile >> commands >> seem to scroll by slower than the unit compiles when running cm3, >> despite the fact >> that the latter include a main program invocation of m3cc. Surely it's >> not merely >> smaller units? And we are also running separately too. >> > > Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' that the backend would generate code in parallel? > - This would not be right what I want because actually obtaining deterministic build results is of high value for security reasons. It can be necessary to verify whether a given compiler toolkit / build environment must have been infected at the stage of build time with hindsight. > I know deterministic builds are quite hard to realize even for some C/C++ applications; nonetheless I had never been thinking about Modula-3 or CM3 in specific. At least it was a well known problem with Python that builds were non-deterministic due to parallel code generation. > However that should not happen if just different source files are translated at the same time. Moreover that could also parallelize the activity of the front (and middle)-end. > ___ I don't believe the builder bugs I was talking about have anything to do with parallel compilation. In fact, they happen with no parallelism. Mika can weigh in, but I am sure the parallelism is carefully synchronized so the resulting compiled code is entirely deterministic, even when the cpu usage getting there is not. ____________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Fri Jun 3 21:48:25 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 19:48:25 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <575199DC.4060305@lcwb.coop> References: , , ,,, , ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , ,,<20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> ,<575199DC.4060305@lcwb.coop> Message-ID: > like this idea, but we don't want to force serialization where we now do parallel > runs of m3cg. Good point. We could use pthreads in there though, maybe. Except I'm not sure we own enough of the code..and..er...oh..nevermind. I believe gcc is replete with globals. As is the frontend. We could still partition the work. If we have say 40 files to compile, we could run 4 m3cg's in parallel that each compile 10 files... ?> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands ?> seem to scroll by slower than the unit compiles when running cm3, despite the fact I believe the C compiler is effectively slow. The C compiler is actually very fast, but it has to do the same or almost the same work very many times. Compiling a thousand or so lines is very quick, but #include windows.h or stdio.h or vector and you have really a ton of code, and it redoes the work for every source files. Precompiled headers can help. Though at least with the Microsoft compiler, they change then language very slightly. And you have to work through each compiler's command line options (maybe cmake/automake??) C++ modules will be the salvation, maybe soon. ?- Jay ---------------------------------------- > Date: Fri, 3 Jun 2016 09:53:16 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > > > On 06/03/2016 02:22 AM, Jay K wrote: >> I understand the C model very well, but I'm not certain of all >> the ramifications for Modula-3. >> >> >> I understand C++ has limited checking via name mangling. >> >> >> And that other systems carry names/signatures forward to runtime >> and check them there. Even dynamically linked C++. And Modula-3. >> >> >> However function names aren't everything -- i.e. they don't include >> structural hashes of types. It is name-based, not structure-based. >> >> >> The most common mistake in the loose C systems is not doing things >> wrong per se, but accidentally mixing old and new versions, >> and the result is undefined/crash. >> >> >> We have the same problem with our "extern" mechanism -- how many times >> I debugged stat corrupting stack until I put in the extra layer. >> >> >> C++ systems are getting better and check for "ODR violations" at link time. >> But checking at link time kinda contradicts my question. >> Goal being to fit into "generic" build systems but C++ only starts >> winning when the linker knows about it -- when the system is C++ specific. >> If C++ requires C++ linker/driver, we shouldn't feel too bad >> requiring a Modula-3 linker/driver. >> >> >>> Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. >> >> >> Ideally this is not the case. Ideally process startup is just referencing >> pages already in file system cache. Demand paging, unified file system cache >> and memory manager and all that (all systems NT, Linux, etc. are the same >> these days). But I realize in reality there is alway some amount of other CPU >> to burn at each process start. The ideal is not quite reached. >> >>> m3cc is a separate executable..has to start up for every .[im]3 file. B >> >> I wonder if we can fix this btw. >> If we can start it just once per library. >> Process create really is fast on typical Posix systems, so people maybe >> don't notice. >> >> There are a few methods: >> - alway keep the m3cg ir files >> - if any out of date, read them all into one large IR and write out one large assembly file >> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >> poor: No need to worry about linker/driver support. >> rich: We can afford the memory these days. Though at some point it wouldn't scale. >> - I want do similar for C backend, as it seems noticable either the compiler startup >> or the few #includes I sometimes use (though C++ modules promise some relief..) >> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >> separately as usual, but within one m3cg invocation. >> > > I like this idea, but we don't want to force serialization where we now do parallel > runs of m3cg. > > OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands > seem to scroll by slower than the unit compiles when running cm3, despite the fact > that the latter include a main program invocation of m3cc. Surely it's not merely > smaller units? And we are also running separately too. > >> >>> Does/can it retain its incrementality? >> >> I think the best but not great we could do is make would decide pessmisically >> to run the compiler, but the compiler could figure out that the old/existing >> output is still correct. The world to decide up-to-date is significant but >> perhaps somewhat less than the work to just do all the work. >> >>> see no benefit and almost certainly losses, in putting these in different executables. >> >> Less code to maintain. Easier to redistribute. >> I like the model of distribution of C source. >> wget source && tar xf && configure && make && make install >> > > We could just wrap what we have in makefile that, almost trivially invokes cm3 with > a few odd options. Or is that cheating? > >> I want something like that. >> I am uncertain of "redistribution" and "active development" must/should be different. >> i.e. We do have something like this, and the two ways of working cooexist. >> >> >> - Jay >> >> >> >> >> >> ---------------------------------------- >>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] replace build system?? >>> >>> >>> >>> On 06/01/2016 04:30 AM, Jay K wrote: >>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>> >>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>> each compilation has no knowledge of any other compilation. Likewise, when linking >>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>> semblance of type-safe compilation, all coders have to correctly follow some variant >>> of a header file/#include idiom and create a matching makefile that correctly reflects >>> the actual interdependencies. There is no checking that these are done correctly, and >>> from direct experience, I can say they are routinely done wrong. >>> >>> All the modular languages have direct linguistic support for type-safe separate >>> compilation. Implementing this correctly while using the C compilation model lies >>> somewhere between very kludgy and impossible. >>> >>>> >>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>> >>> It couldn't possibly. Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. This effect >>> might be partially lost in the fact that m3cc is a separate executable and it >>> has to start up for every .[im]3 file. But it would still about double the >>> number of program executions and startup overheads. >>> >>>> Does/can it retain its incrementality? >>> >>> I don't think this is possible all. No matter what is in the makefile, Make >>> only understands dependencies among whole source files and must rebuild if >>> a depended-on file has been touched, even no changes to content. Our current >>> build system works on declaration granularity. (I presume this is what you >>> mean by incrementality." I am not aware of any language-independent build >>> infrastructure that keeps track of dependencies on other that source file >>> granularity. >>> >>>> >>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>> >>> >>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>> frontend, and language-specific link-time functions in one executable. I see no >>> benefit and almost certainly losses, in putting these in different executables. >>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>> Posix should standardize more of this. :( >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>> To: dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>> in parallel. >>>>>> >>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>> >>>>>> "Daniel Alejandro Benavides D." writes: >>>>>> ... >>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>> four times faster. >>>>>> ... >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Fri Jun 3 21:51:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 19:51:07 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <575197BC.1030100@lcwb.coop> References: , , ,,, , ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , ,,<20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> ,<575197BC.1030100@lcwb.coop> Message-ID: Moving a module from one package to another is a problem I have hit. I moved some path manipulation module a few years ago and incremental builds failed. Not tracking optimization can be seen as a feature, or a bug. ? I might just want to not optimize a few files that I'm debugging. WIDECHAR certainly. Choice of C backend is another. :) Nested functions have a different interface. But appending "c" to BUILD_DIR is maybe a good workaround. Maybe. I wish I could be link compatible but I don't see how. gcc puts the static link often in a special register dedicated to the purpose -- ABIs actually designed and optimized with this in mind! I put it as the last parameter. ?- Jay ---------------------------------------- > Date: Fri, 3 Jun 2016 09:44:12 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > BTW, 'though the current build system is considerably sophisticated, > it does have some bugs, which we should fix. I have the exact setup & > symptom for one of them written down somewhere. I think it's when adding > a procedure signature to an interface when a procedure body for it was > already present in a compiled exporting module, things don't get recompiled > right. > > Also, I occasionally get recompile failures that I don't think are valid, > and when I clean the package and recompile that, all is well. I haven't > ever tracked down how to reproduce this or noted its exact symptom. > > We also don't keep track of compile options that affect generated code, > so as to recompile when only the options change. E.g., optimization level, > debug info presence and format. I did do this for the size of WIDECHAR, > since mismatches here would be disastrous. > > On 06/03/2016 02:22 AM, Jay K wrote: >> I understand the C model very well, but I'm not certain of all >> the ramifications for Modula-3. >> >> >> I understand C++ has limited checking via name mangling. >> >> >> And that other systems carry names/signatures forward to runtime >> and check them there. Even dynamically linked C++. And Modula-3. >> >> >> However function names aren't everything -- i.e. they don't include >> structural hashes of types. It is name-based, not structure-based. >> >> >> The most common mistake in the loose C systems is not doing things >> wrong per se, but accidentally mixing old and new versions, >> and the result is undefined/crash. >> >> >> We have the same problem with our "extern" mechanism -- how many times >> I debugged stat corrupting stack until I put in the extra layer. >> >> >> C++ systems are getting better and check for "ODR violations" at link time. >> But checking at link time kinda contradicts my question. >> Goal being to fit into "generic" build systems but C++ only starts >> winning when the linker knows about it -- when the system is C++ specific. >> If C++ requires C++ linker/driver, we shouldn't feel too bad >> requiring a Modula-3 linker/driver. >> >> >>> Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. >> >> >> Ideally this is not the case. Ideally process startup is just referencing >> pages already in file system cache. Demand paging, unified file system cache >> and memory manager and all that (all systems NT, Linux, etc. are the same >> these days). But I realize in reality there is alway some amount of other CPU >> to burn at each process start. The ideal is not quite reached. >> >>> m3cc is a separate executable..has to start up for every .[im]3 file. B >> >> I wonder if we can fix this btw. >> If we can start it just once per library. >> Process create really is fast on typical Posix systems, so people maybe >> don't notice. >> >> There are a few methods: >> - alway keep the m3cg ir files >> - if any out of date, read them all into one large IR and write out one large assembly file >> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >> poor: No need to worry about linker/driver support. >> rich: We can afford the memory these days. Though at some point it wouldn't scale. >> - I want do similar for C backend, as it seems noticable either the compiler startup >> or the few #includes I sometimes use (though C++ modules promise some relief..) >> >> >> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >> separately as usual, but within one m3cg invocation. >> >> >>> Does/can it retain its incrementality? >> >> I think the best but not great we could do is make would decide pessmisically >> to run the compiler, but the compiler could figure out that the old/existing >> output is still correct. The world to decide up-to-date is significant but >> perhaps somewhat less than the work to just do all the work. >> >>> see no benefit and almost certainly losses, in putting these in different executables. >> >> Less code to maintain. Easier to redistribute. >> I like the model of distribution of C source. >> wget source && tar xf && configure && make && make install >> >> I want something like that. >> I am uncertain of "redistribution" and "active development" must/should be different. >> i.e. We do have something like this, and the two ways of working cooexist. >> >> >> - Jay >> >> >> >> >> >> ---------------------------------------- >>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] replace build system?? >>> >>> >>> >>> On 06/01/2016 04:30 AM, Jay K wrote: >>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>> >>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>> each compilation has no knowledge of any other compilation. Likewise, when linking >>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>> semblance of type-safe compilation, all coders have to correctly follow some variant >>> of a header file/#include idiom and create a matching makefile that correctly reflects >>> the actual interdependencies. There is no checking that these are done correctly, and >>> from direct experience, I can say they are routinely done wrong. >>> >>> All the modular languages have direct linguistic support for type-safe separate >>> compilation. Implementing this correctly while using the C compilation model lies >>> somewhere between very kludgy and impossible. >>> >>>> >>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>> >>> It couldn't possibly. Typical front end compilation actual work is similar to >>> or less than executable program startup/shutdown overhead. This effect >>> might be partially lost in the fact that m3cc is a separate executable and it >>> has to start up for every .[im]3 file. But it would still about double the >>> number of program executions and startup overheads. >>> >>>> Does/can it retain its incrementality? >>> >>> I don't think this is possible all. No matter what is in the makefile, Make >>> only understands dependencies among whole source files and must rebuild if >>> a depended-on file has been touched, even no changes to content. Our current >>> build system works on declaration granularity. (I presume this is what you >>> mean by incrementality." I am not aware of any language-independent build >>> infrastructure that keeps track of dependencies on other that source file >>> granularity. >>> >>>> >>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>> >>> >>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>> frontend, and language-specific link-time functions in one executable. I see no >>> benefit and almost certainly losses, in putting these in different executables. >>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>> Posix should standardize more of this. :( >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>> To: dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>> From: mika at async.caltech.edu >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>> in parallel. >>>>>> >>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>> >>>>>> "Daniel Alejandro Benavides D." writes: >>>>>> ... >>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>> four times faster. >>>>>> ... >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Fri Jun 3 23:08:23 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 16:08:23 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , , , <574F5CE2.8080506@lcwb.coop> , <575197BC.1030100@lcwb.coop> Message-ID: <5751F1C7.5050806@lcwb.coop> On 06/03/2016 02:51 PM, Jay K wrote: > Moving a module from one package to another is a problem I have hit. > I moved some path manipulation module a few years ago and incremental builds failed. > Yes, intra-package dependencies are dealt with very poorly. There is no automatic rebuilding at all (do we really want it, given that a library may have multiple client packages?) Detection of problems doesn't happen until link time, and then with virtually useless error messages that contain only hash codes of language constructs. I'm not all confident of how complete the detection is either. I fixed just one such message to be slightly more helpful a couple of years ago, but several IR operators need additional operands to do anything decent. > Not tracking optimization can be seen as a feature, or a bug. > I might just want to not optimize a few files that I'm debugging. > > WIDECHAR certainly. > > Choice of C backend is another. :) > Nested functions have a different interface. > But appending "c" to BUILD_DIR is maybe a good workaround. Maybe. > I wish I could be link compatible but I don't see how. > gcc puts the static link often in a special register dedicated to the purpose -- ABIs actually designed and optimized with this in mind! > I put it as the last parameter. > > - Jay > > > ---------------------------------------- >> Date: Fri, 3 Jun 2016 09:44:12 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> BTW, 'though the current build system is considerably sophisticated, >> it does have some bugs, which we should fix. I have the exact setup & >> symptom for one of them written down somewhere. I think it's when adding >> a procedure signature to an interface when a procedure body for it was >> already present in a compiled exporting module, things don't get recompiled >> right. >> >> Also, I occasionally get recompile failures that I don't think are valid, >> and when I clean the package and recompile that, all is well. I haven't >> ever tracked down how to reproduce this or noted its exact symptom. >> >> We also don't keep track of compile options that affect generated code, >> so as to recompile when only the options change. E.g., optimization level, >> debug info presence and format. I did do this for the size of WIDECHAR, >> since mismatches here would be disastrous. >> >> On 06/03/2016 02:22 AM, Jay K wrote: >>> I understand the C model very well, but I'm not certain of all >>> the ramifications for Modula-3. >>> >>> >>> I understand C++ has limited checking via name mangling. >>> >>> >>> And that other systems carry names/signatures forward to runtime >>> and check them there. Even dynamically linked C++. And Modula-3. >>> >>> >>> However function names aren't everything -- i.e. they don't include >>> structural hashes of types. It is name-based, not structure-based. >>> >>> >>> The most common mistake in the loose C systems is not doing things >>> wrong per se, but accidentally mixing old and new versions, >>> and the result is undefined/crash. >>> >>> >>> We have the same problem with our "extern" mechanism -- how many times >>> I debugged stat corrupting stack until I put in the extra layer. >>> >>> >>> C++ systems are getting better and check for "ODR violations" at link time. >>> But checking at link time kinda contradicts my question. >>> Goal being to fit into "generic" build systems but C++ only starts >>> winning when the linker knows about it -- when the system is C++ specific. >>> If C++ requires C++ linker/driver, we shouldn't feel too bad >>> requiring a Modula-3 linker/driver. >>> >>> >>>> Typical front end compilation actual work is similar to >>>> or less than executable program startup/shutdown overhead. >>> >>> >>> Ideally this is not the case. Ideally process startup is just referencing >>> pages already in file system cache. Demand paging, unified file system cache >>> and memory manager and all that (all systems NT, Linux, etc. are the same >>> these days). But I realize in reality there is alway some amount of other CPU >>> to burn at each process start. The ideal is not quite reached. >>> >>>> m3cc is a separate executable..has to start up for every .[im]3 file. B >>> >>> I wonder if we can fix this btw. >>> If we can start it just once per library. >>> Process create really is fast on typical Posix systems, so people maybe >>> don't notice. >>> >>> There are a few methods: >>> - alway keep the m3cg ir files >>> - if any out of date, read them all into one large IR and write out one large assembly file >>> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. >>> poor: No need to worry about linker/driver support. >>> rich: We can afford the memory these days. Though at some point it wouldn't scale. >>> - I want do similar for C backend, as it seems noticable either the compiler startup >>> or the few #includes I sometimes use (though C++ modules promise some relief..) >>> >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them >>> separately as usual, but within one m3cg invocation. >>> >>> >>>> Does/can it retain its incrementality? >>> >>> I think the best but not great we could do is make would decide pessmisically >>> to run the compiler, but the compiler could figure out that the old/existing >>> output is still correct. The world to decide up-to-date is significant but >>> perhaps somewhat less than the work to just do all the work. >>> >>>> see no benefit and almost certainly losses, in putting these in different executables. >>> >>> Less code to maintain. Easier to redistribute. >>> I like the model of distribution of C source. >>> wget source && tar xf && configure && make && make install >>> >>> I want something like that. >>> I am uncertain of "redistribution" and "active development" must/should be different. >>> i.e. We do have something like this, and the two ways of working cooexist. >>> >>> >>> - Jay >>> >>> >>> >>> >>> >>> ---------------------------------------- >>>> Date: Wed, 1 Jun 2016 17:08:34 -0500 >>>> From: rodney_bates at lcwb.coop >>>> To: m3devel at elegosoft.com >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> >>>> >>>> On 06/01/2016 04:30 AM, Jay K wrote: >>>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >>>> >>>> It doesn't, not very well. A deep principle, if you can call it that, of C is that >>>> each compilation has no knowledge of any other compilation. Likewise, when linking >>>> them together, it's all just binary stuff, addresses, constants, etc. To get a >>>> semblance of type-safe compilation, all coders have to correctly follow some variant >>>> of a header file/#include idiom and create a matching makefile that correctly reflects >>>> the actual interdependencies. There is no checking that these are done correctly, and >>>> from direct experience, I can say they are routinely done wrong. >>>> >>>> All the modular languages have direct linguistic support for type-safe separate >>>> compilation. Implementing this correctly while using the C compilation model lies >>>> somewhere between very kludgy and impossible. >>>> >>>>> >>>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >>>> >>>> It couldn't possibly. Typical front end compilation actual work is similar to >>>> or less than executable program startup/shutdown overhead. This effect >>>> might be partially lost in the fact that m3cc is a separate executable and it >>>> has to start up for every .[im]3 file. But it would still about double the >>>> number of program executions and startup overheads. >>>> >>>>> Does/can it retain its incrementality? >>>> >>>> I don't think this is possible all. No matter what is in the makefile, Make >>>> only understands dependencies among whole source files and must rebuild if >>>> a depended-on file has been touched, even no changes to content. Our current >>>> build system works on declaration granularity. (I presume this is what you >>>> mean by incrementality." I am not aware of any language-independent build >>>> infrastructure that keeps track of dependencies on other that source file >>>> granularity. >>>> >>>>> >>>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>>>> >>>> >>>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >>>> frontend, and language-specific link-time functions in one executable. I see no >>>> benefit and almost certainly losses, in putting these in different executables. >>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> From: jay.krell at cornell.edu >>>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>>>> Subject: Re: [M3devel] replace build system?? >>>>>> >>>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>>>> Posix should standardize more of this. :( >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------- >>>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>>>> To: dabenavidesd at yahoo.es >>>>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>>>> From: mika at async.caltech.edu >>>>>>> Subject: Re: [M3devel] replace build system?? >>>>>>> >>>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>>>> in parallel. >>>>>>> >>>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>>>> >>>>>>> "Daniel Alejandro Benavides D." writes: >>>>>>> ... >>>>>>>> A make-based build solution would have other advantages as well like=20 >>>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>>>> four times faster. >>>>>>> ... >>>>>>> _______________________________________________ >>>>>>> M3devel mailing list >>>>>>> M3devel at elegosoft.com >>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>>> >>>> >>>> -- >>>> Rodney Bates >>>> rodney.m.bates at acm.org >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From mika at async.caltech.edu Sat Jun 4 00:16:34 2016 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 03 Jun 2016 15:16:34 -0700 Subject: [M3devel] replace build system?? In-Reply-To: <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575199DC.4060305@lcwb.coop> <51f3d79b-9320-bd75-e3b9-354d3929edd6@elstel.org> Message-ID: <20160603221635.019231A206A@async.async.caltech.edu> If you set M3_PARALLEL_BACK, the different individual invocations of m3cgc (or whatever the back end is called) are done in parallel, as separate Unix processes. Nothing tricky, not nondeterministic either. Nothing depends on these jobs except the linker. Mika Elmar Stellnberger writes: > >>> >>> Or, really, more simply, just accept a bunch of .mc/ms file pairs >>> and process them >>> separately as usual, but within one m3cg invocation. >>> >> >> I like this idea, but we don't want to force serialization where we now >> do parallel >> runs of m3cg. >> >> OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile >> commands >> seem to scroll by slower than the unit compiles when running cm3, >> despite the fact >> that the latter include a main program invocation of m3cc. Surely it's >> not merely >> smaller units? And we are also running separately too. >> > >Mika Nystroem, do you mean by 'setting M3_PARALLEL_BACK to 10 or 20' >that the backend would generate code in parallel? > - This would not be right what I want because actually obtaining >deterministic build results is of high value for security reasons. It >can be necessary to verify whether a given compiler toolkit / build >environment must have been infected at the stage of build time with >hindsight. > I know deterministic builds are quite hard to realize even for some >C/C++ applications; nonetheless I had never been thinking about Modula-3 >or CM3 in specific. At least it was a well known problem with Python >that builds were non-deterministic due to parallel code generation. > However that should not happen if just different source files are >translated at the same time. Moreover that could also parallelize the >activity of the front (and middle)-end. >_______________________________________________ >M3devel mailing list >M3devel at elegosoft.com >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Sat Jun 4 02:23:37 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 3 Jun 2016 20:23:37 -0400 Subject: [M3devel] replace build system?? In-Reply-To: <575197BC.1030100@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575197BC.1030100@lcwb.coop> Message-ID: <20160604002337.GA30109@topoi.pooq.com> On Fri, Jun 03, 2016 at 09:44:12AM -0500, Rodney M. Bates wrote: > BTW, 'though the current build system is considerably sophisticated, > it does have some bugs, which we should fix. I have the exact setup & > symptom for one of them written down somewhere. I think it's when adding > a procedure signature to an interface when a procedure body for it was > already present in a compiled exporting module, things don't get recompiled > right. > > Also, I occasionally get recompile failures that I don't think are valid, > and when I clean the package and recompile that, all is well. I haven't > ever tracked down how to reproduce this or noted its exact symptom. I get them too, and I too have not managed to reproduce this. I have the impression that sonetimes something that should have been recompiled isn't, but I don't really know. Deleting the LINIXLIBC directory usually makes everything work. -- hendrik > > We also don't keep track of compile options that affect generated code, > so as to recompile when only the options change. E.g., optimization level, > debug info presence and format. I did do this for the size of WIDECHAR, > since mismatches here would be disastrous. > > On 06/03/2016 02:22 AM, Jay K wrote: > >I understand the C model very well, but I'm not certain of all > >the ramifications for Modula-3. > > > > > >I understand C++ has limited checking via name mangling. > > > > > >And that other systems carry names/signatures forward to runtime > >and check them there. Even dynamically linked C++. And Modula-3. > > > > > >However function names aren't everything -- i.e. they don't include > >structural hashes of types. It is name-based, not structure-based. > > > > > >The most common mistake in the loose C systems is not doing things > >wrong per se, but accidentally mixing old and new versions, > >and the result is undefined/crash. > > > > > >We have the same problem with our "extern" mechanism -- how many times > >I debugged stat corrupting stack until I put in the extra layer. > > > > > >C++ systems are getting better and check for "ODR violations" at link time. > >But checking at link time kinda contradicts my question. > >Goal being to fit into "generic" build systems but C++ only starts > >winning when the linker knows about it -- when the system is C++ specific. > >If C++ requires C++ linker/driver, we shouldn't feel too bad > >requiring a Modula-3 linker/driver. > > > > > > > Typical front end compilation actual work is similar to > > > or less than executable program startup/shutdown overhead. > > > > > > Ideally this is not the case. Ideally process startup is just referencing > > pages already in file system cache. Demand paging, unified file system cache > > and memory manager and all that (all systems NT, Linux, etc. are the same > > these days). But I realize in reality there is alway some amount of other CPU > > to burn at each process start. The ideal is not quite reached. > > > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > > > I wonder if we can fix this btw. > > If we can start it just once per library. > > Process create really is fast on typical Posix systems, so people maybe > > don't notice. > > > > There are a few methods: > > - alway keep the m3cg ir files > > - if any out of date, read them all into one large IR and write out one large assembly file > > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > > poor: No need to worry about linker/driver support. > > rich: We can afford the memory these days. Though at some point it wouldn't scale. > > - I want do similar for C backend, as it seems noticable either the compiler startup > > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > > separately as usual, but within one m3cg invocation. > > > > > > > Does/can it retain its incrementality? > > > > I think the best but not great we could do is make would decide pessmisically > > to run the compiler, but the compiler could figure out that the old/existing > > output is still correct. The world to decide up-to-date is significant but > > perhaps somewhat less than the work to just do all the work. > > > > > see no benefit and almost certainly losses, in putting these in different executables. > > > > Less code to maintain. Easier to redistribute. > > I like the model of distribution of C source. > > wget source && tar xf && configure && make && make install > > > > I want something like that. > > I am uncertain of "redistribution" and "active development" must/should be different. > > i.e. We do have something like this, and the two ways of working cooexist. > > > > > > - Jay > > > > > > > > > > > >---------------------------------------- > >>Date: Wed, 1 Jun 2016 17:08:34 -0500 > >>From: rodney_bates at lcwb.coop > >>To: m3devel at elegosoft.com > >>Subject: Re: [M3devel] replace build system?? > >> > >> > >> > >>On 06/01/2016 04:30 AM, Jay K wrote: > >>>We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > >> > >>It doesn't, not very well. A deep principle, if you can call it that, of C is that > >>each compilation has no knowledge of any other compilation. Likewise, when linking > >>them together, it's all just binary stuff, addresses, constants, etc. To get a > >>semblance of type-safe compilation, all coders have to correctly follow some variant > >>of a header file/#include idiom and create a matching makefile that correctly reflects > >>the actual interdependencies. There is no checking that these are done correctly, and > >>from direct experience, I can say they are routinely done wrong. > >> > >>All the modular languages have direct linguistic support for type-safe separate > >>compilation. Implementing this correctly while using the C compilation model lies > >>somewhere between very kludgy and impossible. > >> > >>> > >>>Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > >> > >>It couldn't possibly. Typical front end compilation actual work is similar to > >>or less than executable program startup/shutdown overhead. This effect > >>might be partially lost in the fact that m3cc is a separate executable and it > >>has to start up for every .[im]3 file. But it would still about double the > >>number of program executions and startup overheads. > >> > >>>Does/can it retain its incrementality? > >> > >>I don't think this is possible all. No matter what is in the makefile, Make > >>only understands dependencies among whole source files and must rebuild if > >>a depended-on file has been touched, even no changes to content. Our current > >>build system works on declaration granularity. (I presume this is what you > >>mean by incrementality." I am not aware of any language-independent build > >>infrastructure that keeps track of dependencies on other that source file > >>granularity. > >> > >>> > >>>Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > >>> > >> > >>In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > >>frontend, and language-specific link-time functions in one executable. I see no > >>benefit and almost certainly losses, in putting these in different executables. > >> > >>>- Jay > >>> > >>> > >>> > >>> > >>>---------------------------------------- > >>>>From: jay.krell at cornell.edu > >>>>To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > >>>>Date: Tue, 31 May 2016 17:18:46 +0000 > >>>>Subject: Re: [M3devel] replace build system?? > >>>> > >>>>We should probably learn how to get the number of processors and optionally system load and make this more automatic. > >>>>Posix should standardize more of this. :( > >>>> > >>>>- Jay > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>---------------------------------------- > >>>>>To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com > >>>>>To: dabenavidesd at yahoo.es > >>>>>Date: Tue, 31 May 2016 10:16:05 -0700 > >>>>>From: mika at async.caltech.edu > >>>>>Subject: Re: [M3devel] replace build system?? > >>>>> > >>>>>I mentioned this before a few times on this mailing list.... CM3 is already fairly > >>>>>parallel if you turn on the right options. Every back-end invocation can be done > >>>>>in parallel. > >>>>> > >>>>>Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... > >>>>> > >>>>>"Daniel Alejandro Benavides D." writes: > >>>>>... > >>>>>>A make-based build solution would have other advantages as well like=20 > >>>>>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 > >>>>>>four times faster. > >>>>>... > >>>>>_______________________________________________ > >>>>>M3devel mailing list > >>>>>M3devel at elegosoft.com > >>>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>>> > >>>>_______________________________________________ > >>>>M3devel mailing list > >>>>M3devel at elegosoft.com > >>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >>>_______________________________________________ > >>>M3devel mailing list > >>>M3devel at elegosoft.com > >>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > >>> > >> > >>-- > >>Rodney Bates > >>rodney.m.bates at acm.org > >>_______________________________________________ > >>M3devel mailing list > >>M3devel at elegosoft.com > >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > >_______________________________________________ > >M3devel mailing list > >M3devel at elegosoft.com > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Sat Jun 4 03:08:23 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 4 Jun 2016 01:08:23 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <20160604002337.GA30109@topoi.pooq.com> References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, <20160531171605.13AA61A2068@async.async.caltech.edu>, , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com> Message-ID: Aside: > LINIXLIBC Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD SPARC32_SOLARIS (which internally lets you chose C compiler) ? They have all been present and working for years. These other names are a warty legacy. The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) I wouldn't mind lowercase or omitting the underscore or using a dash though -- possibly short forms recognized by "config.guess". - Jay > Date: Fri, 3 Jun 2016 20:23:37 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Fri, Jun 03, 2016 at 09:44:12AM -0500, Rodney M. Bates wrote: > > BTW, 'though the current build system is considerably sophisticated, > > it does have some bugs, which we should fix. I have the exact setup & > > symptom for one of them written down somewhere. I think it's when adding > > a procedure signature to an interface when a procedure body for it was > > already present in a compiled exporting module, things don't get recompiled > > right. > > > > Also, I occasionally get recompile failures that I don't think are valid, > > and when I clean the package and recompile that, all is well. I haven't > > ever tracked down how to reproduce this or noted its exact symptom. > > I get them too, and I too have not managed to reproduce this. I have > the impression that sonetimes something that should have been > recompiled isn't, but I don't really know. Deleting the LINIXLIBC > directory usually makes everything work. > > -- hendrik > > > > > We also don't keep track of compile options that affect generated code, > > so as to recompile when only the options change. E.g., optimization level, > > debug info presence and format. I did do this for the size of WIDECHAR, > > since mismatches here would be disastrous. > > > > On 06/03/2016 02:22 AM, Jay K wrote: > > >I understand the C model very well, but I'm not certain of all > > >the ramifications for Modula-3. > > > > > > > > >I understand C++ has limited checking via name mangling. > > > > > > > > >And that other systems carry names/signatures forward to runtime > > >and check them there. Even dynamically linked C++. And Modula-3. > > > > > > > > >However function names aren't everything -- i.e. they don't include > > >structural hashes of types. It is name-based, not structure-based. > > > > > > > > >The most common mistake in the loose C systems is not doing things > > >wrong per se, but accidentally mixing old and new versions, > > >and the result is undefined/crash. > > > > > > > > >We have the same problem with our "extern" mechanism -- how many times > > >I debugged stat corrupting stack until I put in the extra layer. > > > > > > > > >C++ systems are getting better and check for "ODR violations" at link time. > > >But checking at link time kinda contradicts my question. > > >Goal being to fit into "generic" build systems but C++ only starts > > >winning when the linker knows about it -- when the system is C++ specific. > > >If C++ requires C++ linker/driver, we shouldn't feel too bad > > >requiring a Modula-3 linker/driver. > > > > > > > > > > Typical front end compilation actual work is similar to > > > > or less than executable program startup/shutdown overhead. > > > > > > > > > Ideally this is not the case. Ideally process startup is just referencing > > > pages already in file system cache. Demand paging, unified file system cache > > > and memory manager and all that (all systems NT, Linux, etc. are the same > > > these days). But I realize in reality there is alway some amount of other CPU > > > to burn at each process start. The ideal is not quite reached. > > > > > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > > > > > I wonder if we can fix this btw. > > > If we can start it just once per library. > > > Process create really is fast on typical Posix systems, so people maybe > > > don't notice. > > > > > > There are a few methods: > > > - alway keep the m3cg ir files > > > - if any out of date, read them all into one large IR and write out one large assembly file > > > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > > > poor: No need to worry about linker/driver support. > > > rich: We can afford the memory these days. Though at some point it wouldn't scale. > > > - I want do similar for C backend, as it seems noticable either the compiler startup > > > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > > > > > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > > > separately as usual, but within one m3cg invocation. > > > > > > > > > > Does/can it retain its incrementality? > > > > > > I think the best but not great we could do is make would decide pessmisically > > > to run the compiler, but the compiler could figure out that the old/existing > > > output is still correct. The world to decide up-to-date is significant but > > > perhaps somewhat less than the work to just do all the work. > > > > > > > see no benefit and almost certainly losses, in putting these in different executables. > > > > > > Less code to maintain. Easier to redistribute. > > > I like the model of distribution of C source. > > > wget source && tar xf && configure && make && make install > > > > > > I want something like that. > > > I am uncertain of "redistribution" and "active development" must/should be different. > > > i.e. We do have something like this, and the two ways of working cooexist. > > > > > > > > > - Jay > > > > > > > > > > > > > > > > > >---------------------------------------- > > >>Date: Wed, 1 Jun 2016 17:08:34 -0500 > > >>From: rodney_bates at lcwb.coop > > >>To: m3devel at elegosoft.com > > >>Subject: Re: [M3devel] replace build system?? > > >> > > >> > > >> > > >>On 06/01/2016 04:30 AM, Jay K wrote: > > >>>We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > > >> > > >>It doesn't, not very well. A deep principle, if you can call it that, of C is that > > >>each compilation has no knowledge of any other compilation. Likewise, when linking > > >>them together, it's all just binary stuff, addresses, constants, etc. To get a > > >>semblance of type-safe compilation, all coders have to correctly follow some variant > > >>of a header file/#include idiom and create a matching makefile that correctly reflects > > >>the actual interdependencies. There is no checking that these are done correctly, and > > >>from direct experience, I can say they are routinely done wrong. > > >> > > >>All the modular languages have direct linguistic support for type-safe separate > > >>compilation. Implementing this correctly while using the C compilation model lies > > >>somewhere between very kludgy and impossible. > > >> > > >>> > > >>>Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > > >> > > >>It couldn't possibly. Typical front end compilation actual work is similar to > > >>or less than executable program startup/shutdown overhead. This effect > > >>might be partially lost in the fact that m3cc is a separate executable and it > > >>has to start up for every .[im]3 file. But it would still about double the > > >>number of program executions and startup overheads. > > >> > > >>>Does/can it retain its incrementality? > > >> > > >>I don't think this is possible all. No matter what is in the makefile, Make > > >>only understands dependencies among whole source files and must rebuild if > > >>a depended-on file has been touched, even no changes to content. Our current > > >>build system works on declaration granularity. (I presume this is what you > > >>mean by incrementality." I am not aware of any language-independent build > > >>infrastructure that keeps track of dependencies on other that source file > > >>granularity. > > >> > > >>> > > >>>Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > > >>> > > >> > > >>In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > > >>frontend, and language-specific link-time functions in one executable. I see no > > >>benefit and almost certainly losses, in putting these in different executables. > > >> > > >>>- Jay > > >>> > > >>> > > >>> > > >>> > > >>>---------------------------------------- > > >>>>From: jay.krell at cornell.edu > > >>>>To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > > >>>>Date: Tue, 31 May 2016 17:18:46 +0000 > > >>>>Subject: Re: [M3devel] replace build system?? > > >>>> > > >>>>We should probably learn how to get the number of processors and optionally system load and make this more automatic. > > >>>>Posix should standardize more of this. :( > > >>>> > > >>>>- Jay > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>---------------------------------------- > > >>>>>To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com > > >>>>>To: dabenavidesd at yahoo.es > > >>>>>Date: Tue, 31 May 2016 10:16:05 -0700 > > >>>>>From: mika at async.caltech.edu > > >>>>>Subject: Re: [M3devel] replace build system?? > > >>>>> > > >>>>>I mentioned this before a few times on this mailing list.... CM3 is already fairly > > >>>>>parallel if you turn on the right options. Every back-end invocation can be done > > >>>>>in parallel. > > >>>>> > > >>>>>Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... > > >>>>> > > >>>>>"Daniel Alejandro Benavides D." writes: > > >>>>>... > > >>>>>>A make-based build solution would have other advantages as well like=20 > > >>>>>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 > > >>>>>>four times faster. > > >>>>>... > > >>>>>_______________________________________________ > > >>>>>M3devel mailing list > > >>>>>M3devel at elegosoft.com > > >>>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>>> > > >>>>_______________________________________________ > > >>>>M3devel mailing list > > >>>>M3devel at elegosoft.com > > >>>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>> > > >>>_______________________________________________ > > >>>M3devel mailing list > > >>>M3devel at elegosoft.com > > >>>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > >>> > > >> > > >>-- > > >>Rodney Bates > > >>rodney.m.bates at acm.org > > >>_______________________________________________ > > >>M3devel mailing list > > >>M3devel at elegosoft.com > > >>https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > >_______________________________________________ > > >M3devel mailing list > > >M3devel at elegosoft.com > > >https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Jun 4 15:32:10 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 4 Jun 2016 09:32:10 -0400 Subject: [M3devel] replace build system?? In-Reply-To: References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> <575197BC.1030100@lcwb.coop> <20160604002337.GA30109@topoi.pooq.com> Message-ID: <20160604133210.GA19285@topoi.pooq.com> On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > Aside: > > LINIXLIBC > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > SPARC32_SOLARIS (which internally lets you chose C compiler) > ? How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is there a way of telling it otherwise? Do I have to reinstall from scratch? Is there an installer that knows I386_LINUX? Last time I looked there didn't seem to be. > They have all been present and working for years. > These other names are a warty legacy. > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) And how do I ask for the C code generator? -- hendrik From jay.krell at cornell.edu Sun Jun 5 09:46:56 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 5 Jun 2016 07:46:56 +0000 Subject: [M3devel] some quake proposals for more string functions? Message-ID: A new abstract type that represents a set of characters: ? It might actually be a string underneath with 0 or 1 at each position, to aid ease of implementation. ? or a 2-array of 64-bit integers (implied that only characters 0-127 can be in a set) And then functions that indicate how many characters from a set a string contains, or starts with, or ends with? a = CharSet("a") fo = CharSet("fo") foo = CharSet("foo") % redundant but ok fo == foo StringCountSet("foo", a) :0 % how many characters in set a are in string "foo" StringStartsSet("foo", fo) : 3 % how many characters in set fo does string "foo" start with StringEndSet("food", fo) : 0 % how many characters in set fo does string "food" end with FindFirstCharInSet("foo", fo): 0 FindFirstCharInSet("foo", CharSet("d"): -1 or length("foo") ? FindNextCharInSet("foo", fo, 0): 1 ? Might want to combine first/next functions, and require initialOffset and maximumOffset parameters? StringOnlyContainsCharsInSet(str, set): ? return?StringStartsSet(str, set) = length(str) StringContainsNoCharsInSet(str, set): ? return?StringCountSet(str, set) = 0 StringRemoveCharsInSetFromStart(str, set): ? return sub(str,?StringStartsSet(str, set), length(str)) StringRemoveCharsInSetFromEnd(str, set): ? return sub(str, lenth(str) -?StringStartsSet(str, set), length(str)) and then, really, we might add regexps find and find+replace, another day/year? There is some overlap here with the stuff Olaf put in, granted. Thank you, ?- Jay From lists at darko.org Mon Jun 6 01:09:35 2016 From: lists at darko.org (Darko Volaric) Date: Mon, 6 Jun 2016 01:09:35 +0200 Subject: [M3devel] [M3commit] [modula3/cm3] 137373: Add grisu support (little dragon) which allows fas... In-Reply-To: <5754aab547142_6cbe3fe7add0b2c0187858@hookshot-fe3-cp1-prd.iad.github.net.mail> References: <5754aab547142_6cbe3fe7add0b2c0187858@hookshot-fe3-cp1-prd.iad.github.net.mail> Message-ID: For those wondering: http://www.ryanjuckett.com/programming/printing-floating-point-numbers/ On Mon, Jun 6, 2016 at 12:41 AM, GitHub wrote: > Branch: refs/heads/master > Home: https://github.com/modula3/cm3 > Commit: 137373312f6dc35115b5070fcd477a376fa007ea > > https://github.com/modula3/cm3/commit/137373312f6dc35115b5070fcd477a376fa007ea > Author: peter mckinna > Date: 2016-06-03 (Fri, 03 Jun 2016) > > Changed paths: > A m3-libs/m3core/src/float/Common/grisu/CachedPowers.i3 > A m3-libs/m3core/src/float/Common/grisu/CachedPowers.m3 > A m3-libs/m3core/src/float/Common/grisu/Grisu.i3 > A m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > A m3-libs/m3core/src/float/Common/grisu/IEEE.i3 > A m3-libs/m3core/src/float/Common/grisu/IEEE.m3 > A m3-libs/m3core/src/float/Common/grisu/SimFP.i3 > A m3-libs/m3core/src/float/Common/grisu/SimFP.m3 > A m3-libs/m3core/src/float/Common/grisu/m3makefile > M m3-libs/m3core/src/float/Common/m3makefile > M m3-libs/m3core/src/float/IEEE/LongFloat.m3 > M m3-libs/m3core/src/float/IEEE/RealFloat.m3 > > Log Message: > ----------- > Add grisu support (little dragon) which allows faster conversion > from longreal (and real) to ascii in 99.4% of cases. In 0.6% one has > to fall back on dragon4. The algorithm uses native integers instead of > bignums to convert up to 4 times faster than dragon4. > > > Commit: 5703fac32dd7f7718c3f8ce70c15eb9af73499b5 > > https://github.com/modula3/cm3/commit/5703fac32dd7f7718c3f8ce70c15eb9af73499b5 > Author: peter mckinna > Date: 2016-06-03 (Fri, 03 Jun 2016) > > Changed paths: > M m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > M m3-libs/m3core/src/float/Common/grisu/IEEE.m3 > M m3-libs/m3core/src/float/Common/grisu/SimFP.m3 > > Log Message: > ----------- > Cosmetic clean. > > > Commit: 72fe0cb894af1eb605350424733d7bf78520dd43 > > https://github.com/modula3/cm3/commit/72fe0cb894af1eb605350424733d7bf78520dd43 > Author: peter mckinna > Date: 2016-06-04 (Sat, 04 Jun 2016) > > Changed paths: > M m3-libs/m3core/src/float/Common/grisu/Grisu.m3 > > Log Message: > ----------- > Fix exponent to agree with dragon > > > Commit: 2dd875488ae9f37fee4473b39802e82e6b9068f3 > > https://github.com/modula3/cm3/commit/2dd875488ae9f37fee4473b39802e82e6b9068f3 > Author: peter mckinna > Date: 2016-06-06 (Mon, 06 Jun 2016) > > Changed paths: > M m3-sys/m3back/src/M3C.m3 > M scripts/python/capture-boot.py > > Log Message: > ----------- > Merge branch 'master' of https://github.com/modula3/cm3 > > > Compare: > https://github.com/modula3/cm3/compare/0f85470335a7...2dd875488ae9 > _______________________________________________ > M3commit mailing list > M3commit at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 6 19:39:49 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 6 Jun 2016 17:39:49 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <20160604133210.GA19285@topoi.pooq.com> References: , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, <20160531171605.13AA61A2068@async.async.caltech.edu>, , <574F5CE2.8080506@lcwb.coop>, <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , <20160604133210.GA19285@topoi.pooq.com> Message-ID: Fair questions. LINUXLIBC6 and I386_LINUX are the same really. It should work to just rename a few files and change the string in the config file. Overkill but should also work, *something like*: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 ./do-cm3-all.py realclean I386_LINUX ./do-cm3-all.py buildship I386_LINUX # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX ./make-dist-cfg.py I386_LINUX # does very little, you could instead edit your config I have to run through this still myself. With the last two steps, it should use I386_LINUX by default you don't have to keep saying it. This is less overkill if e.g. you want to switch from LINUXLIBC6 to AMD64_LINUX. To use the C compiler, similarly: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using -- no need to a second time if did previous ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX c ./make-dist-cfg.py I386_LINUX Now, the last step, if you aren't going to be using the scripts all the time...you have to edit your bin/config/cm3cfg.common to enable the C backend. Where it says: if not defined ("M3_BACKEND_MODE") M3_BACKEND_MODE = "3" % -- defines how the frontend, backend, and assembler interact % "0" -- don't call m3_backend, M3CG produces object code % "1" -- don't call m3_backend, M3CG produces assembly code % "2" -- call m3_backend, it produces object code % "3" -- call m3_backend, it produces assembly code % "C" -- use C backend, and then compile with compile_c() end change: M3_BACKEND_MODE = "3" to: M3_BACKEND_MODE = "C" I haven't tried these *exact* steps though. We should nail the exact minimal steps and check them in to doc or scripts/python. I'm sure these are not minimal -- we should split it up as minimal to get a compiler, and then separate to build the rest of the system. And we might want to try rename or copy instead of rebuild. Something not in-place will also be good, more like make-dist.py. Sorry, I wanted to send more confident directions earlier but got hung up on some stuff. - Jay > Date: Sat, 4 Jun 2016 09:32:10 -0400 > From: hendrik at topoi.pooq.com > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > > Aside: > > > LINIXLIBC > > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > > SPARC32_SOLARIS (which internally lets you chose C compiler) > > ? > > How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is > there a way of telling it otherwise? Do I have to reinstall from > scratch? Is there an installer that knows I386_LINUX? Last time I > looked there didn't seem to be. > > > They have all been present and working for years. > > These other names are a warty legacy. > > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) > > And how do I ask for the C code generator? > > -- hendrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Jun 11 09:45:01 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 11 Jun 2016 07:45:01 +0000 Subject: [M3devel] cm3 -boot? Message-ID: Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. They seem useless currently. ?- Jay From rodney_bates at lcwb.coop Sat Jun 11 21:07:26 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 11 Jun 2016 14:07:26 -0500 Subject: [M3devel] cm3 -boot? In-Reply-To: References: Message-ID: <575C616E.4030608@lcwb.coop> Not I. All I did was try to make some wild guesses what to expand it to do in conjunction with the llvm backend. On 06/11/2016 02:45 AM, Jay K wrote: > Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? > I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. > They seem useless currently. > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Jun 12 09:35:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 12 Jun 2016 07:35:50 +0000 Subject: [M3devel] cm3 -boot? In-Reply-To: <575C616E.4030608@lcwb.coop> References: , <575C616E.4030608@lcwb.coop> Message-ID: There are multiple parts of "boot". I haven't actually looked into all of them, but might soon. There are: 1) "pushing the compilation along per source file". Compilation is a few steps, and boot deliberately stops earlier. Where it stops, I don't view as a scientific thing, but it is based kind of what tools people tend to have where. Our system kind of assumes that C cross compilers -- and the C runtime headers/libraries and the assembler and linker -- are not particularly available. So "boot" stops at C source or assembly source, assuming the target system has a native assembler, C compiler, linker, etc. Cross compilation systems are more common these days, but I don't yet view as adequately easy to produce or acquire. The C compiler part is actually pretty easy with gcc, but constructing the actually required adequate system -- linker, assembler, "crt0.o", headers, libraries (sysroot) is much more/worse. So I think the assumption is true enough and useful. 2) Production of some sort of makefiles or makeflle fragments to drive the later steps. One small dilema though, is to decide if the second part of bootstrap builds only cm3, then uses it to build the rest, or builds everything. If it only builds cm3, it has smaller requirements. For example, dynamic libs don't really matter. Also incrementality at this point matters less. This is what I have been doing, but I'm torn on the right choice. At the very least, boot1.py should tar.gz-up the source tree, so it is sure to match. Currently boot1.py does some makefile construction but I think maybe instead to repair/extend the cm3 -boot makefile production. Basically to output an autoconf/automake/libtool system. Or least to output something more complete here, even if not autoconf/make/libtool. (I finally am using these systems successfully, and they are ok, but e.g. libtool compiles everything twice, either with unnecessary variation or no variation at all, quite disappointing -- basically everyone using libtool can/should cut their build times in half.) ?- Jay ---------------------------------------- > Date: Sat, 11 Jun 2016 14:07:26 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 -boot? > > > Not I. All I did was try to make some wild guesses what to expand it to > do in conjunction with the llvm backend. > > > On 06/11/2016 02:45 AM, Jay K wrote: >> Does anyone, except for scripts/python/boot1.py, have any dependency on cm3 -boot? >> I'd like to change it a bit -- in particular I want to make the generated makefiles to be close to useful. >> They seem useless currently. >> >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Jun 21 17:54:14 2016 From: jay.krell at cornell.edu (Jay) Date: Tue, 21 Jun 2016 08:54:14 -0700 Subject: [M3devel] Commits mails missing? Message-ID: <960DBCE3-8C99-423F-A8BE-D80796F05BA3@gmail.com> Github m3 commits mails missing? Just me? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 19 09:00:31 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jun 2016 07:00:31 +0000 Subject: [M3devel] m3 cvsweb? Message-ID: Is it possible to bring back www.elegosoft.com/cgi-bin/cvsweb.cgi? The history on github is sometimes unclear e.g. of m3-sys/cm3/src/Builder.m3. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 07:12:13 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 05:12:13 +0000 Subject: [M3devel] cm3 -boot Builder.GenObjectList etc. Message-ID: Does anyone have a full story around cm3 -boot? I "fixed" Builder.GenObjectList to produce valid make code,which is what I thought it was doing, but now I realize, I guess,it was generating quake code.The chunking was probably to workaround historical fixed size buffersand buffer overflows in quake, that we have since fixed. Anyway, I could fix this all up in one of a few ways: - do nothing; do everything in python - generate cmake stuff - generate autotools/libtool stuff - generate quake stuff - other Python nearly suffices, but the current direction bugsme in multiple ways: - I duplicate things like cflags/compiler/asssembler between config and python - The python hardcodes knowing what is a program vs. a library In going to try to complete the cm3 code here..like putting in linking,it strikes me that one problem is when to do probing for libraries,the logic around shared vs. static, prefering shared by default,prefering static if build_standalone, and using whatever is found if only one. Running that logic in cm3 -boot is slightly challenging.We'd either assume and hardcode file presence, possibly just assuming static,or defer the logic to the next phase. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 19 10:45:48 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 19 Jun 2016 08:45:48 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , , , <20160604133210.GA19285@topoi.pooq.com>, Message-ID: I'm pretty convinced the thing to do here, is still run cm3 in its "bulk" way, to output a bunch of .mc/.ic or .c files, and then use autotools/libtool, or possibly cmake to run the C compiler, the assembler, and lib/linker, and probably for ship/install as well. It should output configure.ac/Makefile.am files. This would give us both much increased portability and much decreased maintenance. And we'd use make -j for parallelism. The config files would go away. Quake might even go away. I oscillate between, on one side: There exists Posix cc, so one can portably run a C compiler. And ar+ranlib are very well standardized. But we go well beyond that -- we run the assembler, we produce shared libraries. We support many systems, with fairly minimal and efficient layering, but autotools/libtool and cmake go much further out of their way and support more. We basically only have gcc-based, gcc-like, and Solaris. gcc-like, I mean, Apple's clang accepts gcc options. When using the C backend, we could do another trick -- generate 4+ variations all in one file each with #if's around them: #if 32bit and little endian and posix // x86, arm32 ... #elif 64bit and little endian and posix // amd64, arm64 ... #elif 32bit and big endian and posix // sparc32, ppc_darwin ... #elif 64bit and big endian and posix // sparc64, ppc64_darwin ... #endif The #if's would be controlled by autoconf. We'd have one slightly bloated source distribution for all Posix systems. I know the autotools have a very mixed reputation, and it is deserved. Instead of sh/m4/sed/awk/make/python, they should just us C/C++, maybe Python. Their portability is difficult to reject though. - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com Date: Mon, 6 Jun 2016 17:39:49 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] replace build system?? Fair questions. LINUXLIBC6 and I386_LINUX are the same really. It should work to just rename a few files and change the string in the config file. Overkill but should also work, *something like*: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 ./do-cm3-all.py realclean I386_LINUX ./do-cm3-all.py buildship I386_LINUX # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX ./make-dist-cfg.py I386_LINUX # does very little, you could instead edit your config I have to run through this still myself. With the last two steps, it should use I386_LINUX by default you don't have to keep saying it. This is less overkill if e.g. you want to switch from LINUXLIBC6 to AMD64_LINUX. To use the C compiler, similarly: cd scripts/python edit ../pkginfo.txt remove these two lines: odbc database postgres95 database rm ../PKGS # might not be needed but doesn't hurt ./upgrade.py # be current in whatever you are using -- no need to a second time if did previous ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there # Quick test of compiler, should run and error: ../../m3-sys/cm3/I386_LINUX/cm3 If it fails though, we have already shipped most stuff, so not good. I need to rewrite these steps to be more friendly if they fail. # and then some final small steps ./install-cm3-compiler.py I386_LINUX c ./make-dist-cfg.py I386_LINUX Now, the last step, if you aren't going to be using the scripts all the time...you have to edit your bin/config/cm3cfg.common to enable the C backend. Where it says: if not defined ("M3_BACKEND_MODE") M3_BACKEND_MODE = "3" % -- defines how the frontend, backend, and assembler interact % "0" -- don't call m3_backend, M3CG produces object code % "1" -- don't call m3_backend, M3CG produces assembly code % "2" -- call m3_backend, it produces object code % "3" -- call m3_backend, it produces assembly code % "C" -- use C backend, and then compile with compile_c() end change: M3_BACKEND_MODE = "3" to: M3_BACKEND_MODE = "C" I haven't tried these *exact* steps though. We should nail the exact minimal steps and check them in to doc or scripts/python. I'm sure these are not minimal -- we should split it up as minimal to get a compiler, and then separate to build the rest of the system. And we might want to try rename or copy instead of rebuild. Something not in-place will also be good, more like make-dist.py. Sorry, I wanted to send more confident directions earlier but got hung up on some stuff. - Jay > Date: Sat, 4 Jun 2016 09:32:10 -0400 > From: hendrik at topoi.pooq.com > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: > > Aside: > > > LINIXLIBC > > Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 > > SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD > > SPARC32_SOLARIS (which internally lets you chose C compiler) > > ? > > How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is > there a way of telling it otherwise? Do I have to reinstall from > scratch? Is there an installer that knows I386_LINUX? Last time I > looked there didn't seem to be. > > > They have all been present and working for years. > > These other names are a warty legacy. > > The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. 1) The assumption that Linux is x86 only. 2) The fact that libc5 back in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible with libc6. There is more motivation imho for pthreads vs. userthreads targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should be link compatible with one of them though?) > > And how do I ask for the C code generator? > > -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 07:01:33 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 05:01:33 +0000 Subject: [M3devel] m3 cvsweb? In-Reply-To: References: Message-ID: I found it. https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cm3/src/Builder.m3 very useful because e.g. I put comments sometimes in code, e.g.: configure_c_compiler() % why? http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/cminstall/src/config-no-install/Darwin.common.diff?r1=1.41;r2=1.42 - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: m3 cvsweb? Date: Sun, 19 Jun 2016 07:00:31 +0000 Is it possible to bring back www.elegosoft.com/cgi-bin/cvsweb.cgi? The history on github is sometimes unclear e.g. of m3-sys/cm3/src/Builder.m3. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 06:44:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 04:44:51 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: Message-ID: i.e.: diff --git a/m3-sys/m3quake/src/M3Path.m3 b/m3-sys/m3quake/src/M3Path.m3 index b3e0f27..1b3ac2c 100644 --- a/m3-sys/m3quake/src/M3Path.m3 +++ b/m3-sys/m3quake/src/M3Path.m3 @@ -22,8 +22,8 @@ TYPE CONST Suffix = ARRAY OSKind OF SMap { - (* Unix *) SMap { "", ".i3", ".ib", ".ic", ".is", ".io", - ".m3", ".mb", ".mc", ".ms", ".mo", + (* Unix *) SMap { "", ".i3", ".ib", ".ic", "_i.s", "_i.o", + ".m3", ".mb", ".mc", "_m.s", "_m.o", ".ig", ".mg", ".c", ".h", ".bc", ".s", ".o", ".a", ".a", ".m3x", "", ".mx", ".tmpl" }, (* GrumpyUnix *) SMap { "", ".i3", ".ib", ".ic", "_i.s", "_i.o", full system built on Darwin no problem as I expected. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: naming convention unix vs. grumpyunix? Date: Mon, 20 Jun 2016 01:40:21 +0000 I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 20 03:40:21 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 20 Jun 2016 01:40:21 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? Message-ID: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Jun 22 22:45:56 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 22 Jun 2016 15:45:56 -0500 Subject: [M3devel] Fwd: Re: Commits mails missing? In-Reply-To: <576AB04C.8050602@lcwb.coop> References: <576AB04C.8050602@lcwb.coop> Message-ID: <576AF904.40906@lcwb.coop> FWIW: For a long time, I have been getting three copies of these, two starting with [m3commit][modula3/cm3] in the subject, and one without the [m3commit]. For a few days, I was getting only the shorter one. But now the other two have shown up, for each such commit, 3 days later arriving, but showing the identical date and time of origination. On 06/21/2016 10:54 AM, Jay wrote: > Github m3 commits mails missing? Just me? > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at m3lists.elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From lists at darko.org Wed Jun 22 22:53:12 2016 From: lists at darko.org (Darko Volaric) Date: Wed, 22 Jun 2016 22:53:12 +0200 Subject: [M3devel] Fwd: Re: Commits mails missing? In-Reply-To: <576AF904.40906@lcwb.coop> References: <576AB04C.8050602@lcwb.coop> <576AF904.40906@lcwb.coop> Message-ID: When I changed it some time ago to fix an issue related to people using emails on github that were not subscribed to the list, your email was in there along with the list address, so I left it in there. You might also be personally subscribed to the commits, which would explain the third copy. On Wed, Jun 22, 2016 at 10:45 PM, Rodney M. Bates wrote: > > FWIW: > > For a long time, I have been getting three copies of these, two starting > with > [m3commit][modula3/cm3] in the subject, and one without the [m3commit]. > For a few days, I was getting only the shorter one. But now the other two > have shown up, for each such commit, 3 days later arriving, but showing the > identical date and time of origination. > > On 06/21/2016 10:54 AM, Jay wrote: > >> Github m3 commits mails missing? Just me? >> >> >> - Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at m3lists.elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> > -- > Rodney Bates > rodney.m.bates at acm.org > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Jun 23 02:38:18 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jun 2016 00:38:18 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Thu Jun 23 08:33:00 2016 From: dmuysers at hotmail.com (dirk muysers) Date: Thu, 23 Jun 2016 08:33:00 +0200 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay -------------------------------------------------------------------------------- From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------------------------------------------------------------------------- _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From nafkhami at elegosoft.com Thu Jun 23 11:01:04 2016 From: nafkhami at elegosoft.com (navid afkhami) Date: Thu, 23 Jun 2016 11:01:04 +0200 Subject: [M3devel] New Link for M3 Mailing lists and Archives Message-ID: <576BA550.9000403@elegosoft.com> Hi, We would like to inform you that the Mailing lists and Archives links are available again and you can find them in the mentioned link below. https://m3lists.elegosoft.com Thank you elego Software Solutions -- Navid Afkhami IT Services & Support elego Software Solutions GmbH Gustav-Meyer-Allee 25 Building 12.3 (BIG) room 227 13355 Berlin, Germany phone +49 30 23 45 86 96 navid.afkhami at elegosoft.com fax +49 30 23 45 86 95 http://www.elegosoft.com Geschaeftsfuehrer: Olaf Wagner, Sitz Berlin Amtsgericht Berlin-Charlottenburg, HRB 77719, USt-IdNr: DE163214194 From jay.krell at cornell.edu Thu Jun 23 20:33:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 23 Jun 2016 18:33:47 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu>, , Message-ID: Not really. My goal is to be close as possible to: tar xf foo.tar.gz mkdir bar # optional cd bar # or foo ../foo/configure make sudo make install Nagging questions: Is there one foo.tar.gz for everyone and autoconf picks the right part of it, or people pick the "correct" one for their system. Is there a few such files -- target.tar.gz, the-rest-m3.tar.gz, maybe m3cc.tar.gz -- this is how 3.6 was structured Back then quake was written in C however, not sure it matters. Do we first build cm3 and then the rest of the system using it, or do we just use "make" to build everything. I can imagine how to build everything from assembly using autoconf/libtool/make, but I kinda only want to do that only if cm3 also reuses the same infrastructure. Sometimes I also think of giving up on dynamic linking, since that is one of the bigger thorns. - Jay From: dmuysers at hotmail.com To: jay.krell at cornell.edu; hosking at purdue.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Thu, 23 Jun 2016 08:33:00 +0200 Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: I propose making unix match grumpyunix and removing grumpyunix. There is slight upside and should be no downside. The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? I expect everything will just work. - Jay _______________________________________________ M3devel mailing list M3devel at m3lists.elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Jun 24 19:34:16 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 24 Jun 2016 17:34:16 +0000 (UTC) Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu> Message-ID: <359115250.1721526.1466789656512.JavaMail.yahoo@mail.yahoo.com> Hi all: Although I have proposed moving to Java or encouraged al least (both build system and runtime), you have to take for granted that there are sources of security leaks like [1] (Protection in Programming-Language Translations)explains in Java to JVML translation. Specially fault isolation properties defined by Modula-3 UNSAFE modules. Thanks in advance [1] ?Digital Systems Research Center: Report 154?. [Online]. Available on: ftp://ftp.hpl.hp.com/gatekeeper/pub/compaq/SRC/research-reports/abstracts/src-rr-154.html. [Accesed: 24-jun-2016]. El Jueves 23 de junio de 2016 13:33, Jay K escribi?: Not really. My goal is to be close as possible to: tar xf foo.tar.gz mkdir bar # optional cd bar # or foo ../foo/configure make sudo make install Nagging questions: Is there one foo.tar.gz for everyone and autoconf picks the right part of it, or people pick the "correct" one for their system. Is there a few such files -- target.tar.gz, the-rest-m3.tar.gz, maybe m3cc.tar.gz -- this is how 3.6 was structured Back then quake was written in C however, not sure it matters. Do we first build cm3 and then the rest of the system using it, or do we just use "make" to build everything. I can imagine how to build everything from assembly using autoconf/libtool/make, but I kinda only want to do that only if cm3 also reuses the same infrastructure. Sometimes I also think of giving up on dynamic linking, since that is one of the bigger thorns. - Jay ________________________________ From: dmuysers at hotmail.com To: jay.krell at cornell.edu; hosking at purdue.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Thu, 23 Jun 2016 08:33:00 +0200 Did you ever consider 0install as a means of distribution? From: Jay K Sent: Thursday, June 23, 2016 2:38 AM To: Hosking, Antony L Cc: m3devel Subject: Re: [M3devel] naming convention unix vs. grumpyunix? This is a bit long and out of order, sorry. Simple story is for us to get out of the platform-specific build system maintenance business, and reuse larger portability from other projects. I've been wrestling with this in my head a long while. - I don't like maintaining the config files. It is hard to be an expert on dynamic linking across "many" operating systems, linkers, versions. - I don't like that for example an AIX port remains absent. And now I see AIX doesn't have $ORIGIN. - It bothers me just slightly that we aren't portable to the older systems that lack $ORIGIN. $ORIGIN is nice if you are redistributing binaries, that will be moved around, but it was never needed for self-built software, or software installed to an agreed upon place, and it isn't supported in setuid or such programs. (Aside -- and remember how bad it used to be? We used to distribute binaries with random hardcoded paths, and advise people to set LD_LIBRARY_PATH. Even for stuff people self-built, it wasn't good. So I did improve things but I don't think it is worth us doing ourselves.) - Our current bootstrap/cross-build story isn't automated enough. And then, what should it look like? - Generating cmake or autoconf/automake/libtool input provides some potential answers. I'd really like to delegate to folks that did and will continue to port pretty much everywhere. Sometimes I think, hey, we can just do what we need ourselves, but then I see how much gnarly system-specific knowledge autotools/cmake deliver nicely to their users. I had a mental stumbling block for years with cmake/autotools but finally got over it. I have prototyped some simple uses, both with recursive make and non-recursive make. configure is a bit slow, but we'd have a very minimal one. The resulting make invocations are ok. I can almost just generate makefiles myself, but then for example I don't know much about "install". cmake/automake provide me "install" with me knowing nothing. I don't really want to be an expert in make, compiler flags, linker flags, Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but not so much as the make/sed/awk/sh level. There are a few details of autoconf/cmake/libtool I don't like, where the Modula-3 build system is clearly and simply superior. And other areas where I'm not sure what is ideal. Where Modula-3 is clearly superior in that in producing static and dynamic libraries, it only ever compiles once. cmake and libtool are pretty keen on compiling everything twice -- even with identical command lines. Where I'm not sure is our probing for libraries and the build_standalone feature. I think if we did things a little different/better, we wouldn't even have cm3 be standalone. I very much want to offer to users the: tar xf cm3... cd cm3... configure make make install sort of experience. There are slight difficulties. configure probes the C compiler for what it produces. Let's ignore C-backend and LLVM for now and consider cm3cg. The likely best bootstrap format is assembly source. Like the 3.6 release. For just cm3/m3core/libm3, or the entire system? So configure probing vs. having on hand possibly just one assembly source is a bit of a misfit. Perhaps configure would be tailored to hardcode what the distribution contains. Or perhaps the distribution would contain "everything" and configure would pick the right one. It is obviously wasteful, but these days maybe ok, and the result easier for people to install. The C generating backend doesn't fix this much or entirely, since the C is still target-specific. Maybe we can fold the C down to just a few platforms, and then the idea of one cross-platform distribution might work. Maybe eventually the generated C can speak in "integer" and array/struct references, instead of front-end computed offsets, but that is a ways off. autotools/libtool also solve that problem where non-shipped binaries don't run. Something we have hacked on by sprinkling build_standalone around. I'm not sure if cmake fixes this. I'm not sure they solve it the way I want though -- I'd like to have the uninstalled paths hardcoded, then relink or otherwise binary edit in install. One thing I need to study a bit more is how to install all the extra stuff to the pkg directories. As well,...so many things... we have this structure: bin/foo lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) pkg/foo/TARGET/foo.so I have always found it a little suspicious that binaries have implicit TARGET but pkgs have explicit TARGET. I somewhat pine for a layout that can accomodiate all targets including the bin directory. I suppose if bin and lib are what run, and pkg is only for building, this accomodates unshipped cross builds nicely. But ideally you could have a runnable PPC_DARWIN/I386_DARWIN/AMD664_DARWIN system all in structure (caveat that PPC_DARWIN doesn't work in Rosetta because of our preemptive suspend -- cooperative suspend would fix that.) - Jay ________________________________ From: hosking at purdue.edu To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] naming convention unix vs. grumpyunix? Date: Wed, 22 Jun 2016 21:19:12 +0000 Why import dependencies on make and automake? Sent from my iPad On Jun 22, 2016, at 9:41 PM, Jay K wrote: >I propose making unix match grumpyunix and removing grumpyunix. > >There is slight upside and should be no downside. > >The upside is that various tools -- make and automake -- know that .s is assembly and will assemble it. > >Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. foo.m3 => foo.ms/foo.mo? > >I expect everything will just work. > >- Jay > > > > > _______________________________________________ >M3devel mailing list >M3devel at m3lists.elegosoft.com >https://m3lists.elegosoft.com/mailman/listinfo/m3devel > ________________________________ _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel From wagner at elegosoft.com Sat Jun 25 14:24:59 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 25 Jun 2016 14:24:59 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? Message-ID: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Lurking in the shadows and reading all the recent mails about improving the build system and seeing more and more steps 'backwards' to adapt to C, autoconf, makefiles and more overcome tools and concepts, a heretical thought comes to mind: Have you ever considered using docker as a means to provide easy access and distribution of Modula-3? Some historic background: in the eighties and nineties, standardization took place on the language level with structured programming, data types, modules and interfaces, classes etc. I know of several attempts to extend the new concepts and abstractions to operating system level: Smalltalk, ELAN (extended Algol-68) (EUMEL OS ;-) and even Modula-3 come to mind (SPIN OS). All these attempts were partially successful in their implementation, but never succeeded in gaining even 0.01% of acceptance in the OS market. On the OS level, however, all standardization attempts sadly failed (IMHO). POSIX is only a crutch, and all vendors and even the free Unix derivates and lookalikes went their own way wrt. libraries, linking and packaging. As Greg Nelson (IIRC) mentioned to me some years ago, the aspects of linking have never been analyzed in the scope of Modula-3, and so have all the other OS-specific tasks like package building and distribution. When the Modula-3 package system has been designed, there were no Redhat or Debian packages, and perhaps only some first attempts at SunOS packages. In the Java realm at least, packages, repositories and build systems like maven have been largely successful, and they look very much like a redesigned M3 build system. Recently however, a pragmatical approach at distributing easy-to-run software services and applications has been and still is very successful: bundle all the needed stuff together and put it into one 'container'. The free and most successful implementation of this is the Linux docker system. While the implementation is ABI-centric and depends on Linux, nontheless docker containers can be run easily on MacOS and Windows too, which should cover more than 95% of the installation base, I think. At Elego we've been using docker successfully for more than a year now to automate and standardize both our development and system operation procedures. I haven't thought this through, but there could be several ways of using docker together with the CM3 system. For beginners, there could be a standalone docker image with the compiler, all packages installed, and an HTTP-service as GUI, which would be started automatically when running the docker image. Local directories may be mounted at predefined locations to provide access to local files. We already have the HTTP-service GUI in our sources! As a development system, I could envisage a docker image with the compiler and build system in one container image and the package libraries -- perhaps even separated for different target architectures -- in persistent data containers. Initial versions of the latter could also be distributed via existing docker registries in the internet. We would have to define something like a standard M3 docker interface for M3 docker containers, which could be done with container entrypoints and some additional scripts. We could use docker-compose specifications to define complex setups. Finally, the M3 docker system could be extended not only to run new binaries inside its own container, but to build standalone docker containers to run the applications and distribute them via the existing registry system in the internet. It could all be as easy as typing docker run m3-juno to start the Juno-2 constrained based editor on your local system without even installing one Modula-3 package or dependency. You only need to have installed docker or one of its emulation packages. CAVEATS This is not for everybody and every purpose though. First, it does not help in the realm of efficient systems programming for dedicated target platforms. As this was one of the original goals of Modula-3 development, it is somehow at cross-purposes with its intention. Second, it doesn't solve the building and linking problems, but works around them. Third, it adds another level of complexity and probably hundreds of MBs to running an M3 application. This seems to be more and more acceptable, looking at other development though. ;-) PROS It would completely resolve the access and distribution aspect of Modula-3 IMO. No questions anymore about how to install the compiler and how to get it running on system X. At least, if the system X is able to run docker containers, but as I said before, most systems today are. What do you think? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dmuysers at hotmail.com Sat Jun 25 15:46:20 2016 From: dmuysers at hotmail.com (dirk muysers) Date: Sat, 25 Jun 2016 15:46:20 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Message-ID: Another --less heavyweight-- package distribution system is Zero Install. https://0install.de/ -----Original Message----- From: Olaf Wagner Sent: Saturday, June 25, 2016 2:24 PM To: m3devel Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? Lurking in the shadows and reading all the recent mails about improving the build system and seeing more and more steps 'backwards' to adapt to C, autoconf, makefiles and more overcome tools and concepts, a heretical thought comes to mind: Have you ever considered using docker as a means to provide easy access and distribution of Modula-3? Some historic background: in the eighties and nineties, standardization took place on the language level with structured programming, data types, modules and interfaces, classes etc. I know of several attempts to extend the new concepts and abstractions to operating system level: Smalltalk, ELAN (extended Algol-68) (EUMEL OS ;-) and even Modula-3 come to mind (SPIN OS). All these attempts were partially successful in their implementation, but never succeeded in gaining even 0.01% of acceptance in the OS market. On the OS level, however, all standardization attempts sadly failed (IMHO). POSIX is only a crutch, and all vendors and even the free Unix derivates and lookalikes went their own way wrt. libraries, linking and packaging. As Greg Nelson (IIRC) mentioned to me some years ago, the aspects of linking have never been analyzed in the scope of Modula-3, and so have all the other OS-specific tasks like package building and distribution. When the Modula-3 package system has been designed, there were no Redhat or Debian packages, and perhaps only some first attempts at SunOS packages. In the Java realm at least, packages, repositories and build systems like maven have been largely successful, and they look very much like a redesigned M3 build system. Recently however, a pragmatical approach at distributing easy-to-run software services and applications has been and still is very successful: bundle all the needed stuff together and put it into one 'container'. The free and most successful implementation of this is the Linux docker system. While the implementation is ABI-centric and depends on Linux, nontheless docker containers can be run easily on MacOS and Windows too, which should cover more than 95% of the installation base, I think. At Elego we've been using docker successfully for more than a year now to automate and standardize both our development and system operation procedures. I haven't thought this through, but there could be several ways of using docker together with the CM3 system. For beginners, there could be a standalone docker image with the compiler, all packages installed, and an HTTP-service as GUI, which would be started automatically when running the docker image. Local directories may be mounted at predefined locations to provide access to local files. We already have the HTTP-service GUI in our sources! As a development system, I could envisage a docker image with the compiler and build system in one container image and the package libraries -- perhaps even separated for different target architectures -- in persistent data containers. Initial versions of the latter could also be distributed via existing docker registries in the internet. We would have to define something like a standard M3 docker interface for M3 docker containers, which could be done with container entrypoints and some additional scripts. We could use docker-compose specifications to define complex setups. Finally, the M3 docker system could be extended not only to run new binaries inside its own container, but to build standalone docker containers to run the applications and distribute them via the existing registry system in the internet. It could all be as easy as typing docker run m3-juno to start the Juno-2 constrained based editor on your local system without even installing one Modula-3 package or dependency. You only need to have installed docker or one of its emulation packages. CAVEATS This is not for everybody and every purpose though. First, it does not help in the realm of efficient systems programming for dedicated target platforms. As this was one of the original goals of Modula-3 development, it is somehow at cross-purposes with its intention. Second, it doesn't solve the building and linking problems, but works around them. Third, it adds another level of complexity and probably hundreds of MBs to running an M3 application. This seems to be more and more acceptable, looking at other development though. ;-) PROS It would completely resolve the access and distribution aspect of Modula-3 IMO. No questions anymore about how to install the compiler and how to get it running on system X. At least, if the system X is able to run docker containers, but as I said before, most systems today are. What do you think? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel From adacore at marino.st Sat Jun 25 16:06:12 2016 From: adacore at marino.st (John Marino) Date: Sat, 25 Jun 2016 16:06:12 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> Message-ID: <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> On 6/25/2016 2:24 PM, Olaf Wagner wrote: > Lurking in the shadows and reading all the recent mails about improving > the build system and seeing more and more steps 'backwards' to adapt > to C, autoconf, makefiles and more overcome tools and concepts, a > heretical thought comes to mind: Have you ever considered using docker > as a means to provide easy access and distribution of Modula-3? You mean Linux-only Docker? In general this concept would not be compatible with typical s/w build repositories used by various OS. If you went to a primary Docker distribution system, you're basically saying A) M3 is limited to Linux and B) It's distributed by third parties (you) only. At first glance, I'd say it's a bad idea. John --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus From jay.krell at cornell.edu Sat Jun 25 19:43:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Sat, 25 Jun 2016 17:43:07 +0000 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation: automake "compiles" to make cm3 "compiles" to quake Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem. Not the entire problem -- cloud app declaration, software composition. Though it still low level -- how do I describe and join multiple containers? I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work. I think they are great for portabilitity. Even better, generate C++ and likely gain efficient exception handling. Delegating the work to the C++ implementers, which generally do much better than us here (basically inheriting "stack walkers" wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could: - accept our current platforms/ports (i.e. don't fret over AIX, etc.) - accept that the config files aren't bad asis - bootstrap can just automate slightly more and build cm3 - use cm3 to build and install everything else, including cm3cg, dynamic linking, etc. That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core The system it is building would have to be matched with the cm3 itself. It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! - Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 20:17:11 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 18:17:11 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv0826315534 #yiv0826315534 --.yiv0826315534hmmessage P{margin:0px;padding:0px;}#yiv0826315534 body.yiv0826315534hmmessage{font-size:12pt;font-family:Calibri;}#yiv0826315534 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 20:20:51 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 18:20:51 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> hi:besides that cm3 v4 could compile pm3 anyway. El S?bado 25 de junio de 2016 13:17, Daniel Alejandro Benavides D. escribi?: Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv3270809950 --.yiv3270809950hmmessage P{margin:0px;padding:0px;}#yiv3270809950 body.yiv3270809950hmmessage{font-size:12pt;font-family:Calibri;}#yiv3270809950 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Jun 25 21:41:44 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 25 Jun 2016 19:41:44 +0000 (UTC) Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <1909574679.2271708.1466878631674.JavaMail.yahoo@mail.yahoo.com> <1052505204.2280893.1466878851365.JavaMail.yahoo@mail.yahoo.com> Message-ID: <285148656.2323402.1466883704854.JavaMail.yahoo@mail.yahoo.com> Hi all:anyway whether C or assembly should be available as options for bootstrapping, now we just have gcc and not all platforms are gcc ready.Thanks in advance El S?bado 25 de junio de 2016 13:20, Daniel Alejandro Benavides D. escribi?: hi:besides that cm3 v4 could compile pm3 anyway. El S?bado 25 de junio de 2016 13:17, Daniel Alejandro Benavides D. escribi?: Hi Jay, all:do you? remember that pm3 build run in make; do you remember the cross platform assembly generation model.What do you think of it? Thanks in advance El S?bado 25 de junio de 2016 12:43, Jay K escribi?: #yiv4790279345 --.yiv4790279345hmmessage P{margin:0px;padding:0px;}#yiv4790279345 body.yiv4790279345hmmessage{font-size:12pt;font-family:Calibri;}#yiv4790279345 I agree and disagree. Regarding Posix -- I think it did well for low level C. I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation:?automake "compiles" to make?cm3 "compiles" to quake ?Quake was written in C. The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. I'm aware of Docker. I think it is solving a problem.? Not the entire problem -- cloud app declaration, software composition.? Though it still low level -- how do I describe and join multiple containers? ??I'm not sure it is relevant here. However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I don't think C and autoconf/automake are clearly a step backward. I want more portability and, I admit, less work.?I think they are great for portabilitity.?Even better, generate C++ and likely gain efficient exception handling.?Delegating the work to the C++ implementers, which generally?do much better than us here (basically inheriting "stack walkers"?wherever they exist.) I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. There are middle grounds that don't "step backward".We could:?- accept our current platforms/ports (i.e. don't fret over AIX, etc.)??- accept that the config files aren't bad asis??- bootstrap can just automate slightly more and build cm3??- use cm3 to build and install everything else, including cm3cg,? ?dynamic linking, etc.?? ?That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core?The system it is building would have to be matched with the cm3 itself.?It might turn around at the cm3 point and use that cm3 to rebuild. This is only a minor layering/scripting over what we already have. This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! ?- Jay > To: wagner at elegosoft.com; m3devel at elegosoft.com > From: adacore at marino.st > Date: Sat, 25 Jun 2016 16:06:12 +0200 > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > Lurking in the shadows and reading all the recent mails about improving > > the build system and seeing more and more steps 'backwards' to adapt > > to C, autoconf, makefiles and more overcome tools and concepts, a > > heretical thought comes to mind: Have you ever considered using docker > > as a means to provide easy access and distribution of Modula-3? > > You mean Linux-only Docker? > > In general this concept would not be compatible with typical s/w build > repositories used by various OS. If you went to a primary Docker > distribution system, you're basically saying A) M3 is limited to Linux > and B) It's distributed by third parties (you) only. > > At first glance, I'd say it's a bad idea. > > John > > --- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Jun 26 08:34:39 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jun 2016 06:34:39 +0000 Subject: [M3devel] naming convention unix vs. grumpyunix? In-Reply-To: References: , <0AE64914-0724-4423-A551-B7789627B9C9@purdue.edu>, Message-ID: Slight clarification from earlier: "Dependency on gcc" is gcc or cc is a good way to run the correct assembler, which they decide to do for .s suffix, but not .ms/.is And, it turns out, when faced with assembly source, automake likes to run the C compiler. If you imagine a strategy where we write automake intput, then .s helps. Though the next possible step is automake being "native" -- and shiping some form of it for Windows, which I know doesn't go over well, but it really is impressive the extent that GNU build tools (gcc/ld) can target Windows -- it would be a good direction now for an AMD64 Windows port (besides the C backend...and ignoring LLVM...which maybe I should move my focus too...) ?- Jay ________________________________ > From: jay.krell at cornell.edu > To: hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] naming convention unix vs. grumpyunix? > Date: Thu, 23 Jun 2016 00:38:18 +0000 > > This is a bit long and out of order, sorry. > Simple story is for us to get out of the platform-specific build system > maintenance business, and reuse larger portability from other projects. > > > > I've been wrestling with this in my head a long while. > > > - I don't like maintaining the config files. > It is hard to be an expert on dynamic linking across "many" operating > systems, linkers, versions. > > > - I don't like that for example an AIX port remains absent. > And now I see AIX doesn't have $ORIGIN. > > > - It bothers me just slightly that we aren't portable > to the older systems that lack $ORIGIN. > > > $ORIGIN is nice if you are redistributing binaries, > that will be moved around, but it was never needed > for self-built software, or software installed to > an agreed upon place, and it isn't supported in setuid or such > programs. > > (Aside -- and remember how bad it used to be? > We used to distribute binaries with random hardcoded paths, > and advise people to set LD_LIBRARY_PATH. Even for stuff people > self-built, it wasn't good. So I did improve things > but I don't think it is worth us doing ourselves.) > > > - Our current bootstrap/cross-build story isn't automated enough. > And then, what should it look like? > > > - Generating cmake or autoconf/automake/libtool input provides some > potential answers. > > I'd really like to delegate to folks that did and will continue to > port pretty much everywhere. > Sometimes I think, hey, we can just do what we need ourselves, but > then I see how > much gnarly system-specific knowledge autotools/cmake deliver nicely > to their users. > > > I had a mental stumbling block for years with cmake/autotools but finally > got over it. I have prototyped some simple uses, both with recursive > make and non-recursive make. > > > configure is a bit slow, but we'd have a very minimal one. > The resulting make invocations are ok. > > > I can almost just generate makefiles myself, but then for example > I don't know much about "install". cmake/automake provide me "install" > with me knowing nothing. > > > I don't really want to be an expert in make, compiler flags, linker flags, > Posix portability gotchas, etc. -- ok maybe at the libc/m3core level, but > not so much as the make/sed/awk/sh level. > > > There are a few details of autoconf/cmake/libtool I don't like, where > the Modula-3 > build system is clearly and simply superior. And other areas where I'm not > sure what is ideal. > > > Where Modula-3 is clearly superior in that in producing static and dynamic > libraries, it only ever compiles once. cmake and libtool are pretty keen > on compiling everything twice -- even with identical command lines. > > > Where I'm not sure is our probing for libraries and the > build_standalone feature. > I think if we did things a little different/better, we wouldn't even > have cm3 > be standalone. > > > I very much want to offer to users the: > tar xf cm3... > cd cm3... > configure > make > make install > > > sort of experience. > > > There are slight difficulties. > configure probes the C compiler for what it produces. > Let's ignore C-backend and LLVM for now and consider cm3cg. > > > The likely best bootstrap format is assembly source. Like the 3.6 release. > For just cm3/m3core/libm3, or the entire system? > > > So configure probing vs. having on hand possibly just one assembly > source is a bit of a misfit. > > > Perhaps configure would be tailored to hardcode what the distribution > contains. > > > Or perhaps the distribution would contain "everything" and configure > would pick the right one. > It is obviously wasteful, but these days maybe ok, and the result > easier for people to install. > > > The C generating backend doesn't fix this much or entirely, since the > C is still target-specific. > Maybe we can fold the C down to just a few platforms, and then the > idea of one cross-platform distribution > might work. Maybe eventually the generated C can speak in "integer" > and array/struct references, instead > of front-end computed offsets, but that is a ways off. > > > autotools/libtool also solve that problem where non-shipped binaries > don't run. > Something we have hacked on by sprinkling build_standalone around. > I'm not sure if cmake fixes this. > > > I'm not sure they solve it the way I want though -- I'd like to have > the uninstalled > paths hardcoded, then relink or otherwise binary edit in install. > > > One thing I need to study a bit more is how to install all the extra > stuff to the pkg directories. > > As well,...so many things... we have this structure: > bin/foo > lib/foo.so (did I do this? No matter, the layout is wierd w/o it.) > pkg/foo/TARGET/foo.so > > > I have always found it a little suspicious that binaries have implicit > TARGET > but pkgs have explicit TARGET. I somewhat pine for a layout that can > accomodiate > all targets including the bin directory. > > > I suppose if bin and lib are what run, and pkg is only for building, > this accomodates > unshipped cross builds nicely. But ideally you could have a runnable > PPC_DARWIN/I386_DARWIN/AMD664_DARWIN > system all in structure (caveat that PPC_DARWIN doesn't work in > Rosetta because of our > preemptive suspend -- cooperative suspend would fix that.) > > > - Jay > > > > > ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] naming convention unix vs. grumpyunix? > Date: Wed, 22 Jun 2016 21:19:12 +0000 > > Why import dependencies on make and automake? > > Sent from my iPad > > On Jun 22, 2016, at 9:41 PM, Jay K > > wrote: > > I propose making unix match grumpyunix and removing grumpyunix. > > There is slight upside and should be no downside. > > The upside is that various tools -- make and automake -- know that .s > is assembly and will assemble it. > > Is it a downside for base name to change foo.m3 => foo_m.s/foo_m.o vs. > foo.m3 => foo.ms/foo.mo? > > I expect everything will just work. > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at m3lists.elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Sun Jun 26 09:15:37 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 26 Jun 2016 07:15:37 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , , <574F5CE2.8080506@lcwb.coop>, , <575197BC.1030100@lcwb.coop>, <20160604002337.GA30109@topoi.pooq.com>, , , , <20160604133210.GA19285@topoi.pooq.com>, Message-ID: >> How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is? >> there a way of telling it otherwise? Do I have to reinstall from? Ok I went through this exercise. Finally getting at least some cross tools on the Linux/amd64 system to target x86. gcc and libc but not libxaw. I had some trial/error and hits/misses. I started from no x86 tools (not even gcc) and I never got 32bit Xaw working on AMD64 Linux working. So I did remove all the gui stuff from pkginfo.txt. But a native system won't have these troubles. But anyway, based on my experience: ?vi $HOME/cm3.l6/bin/cm3.cfg? ?Change HOST and TARGET to I386_LINUX unconditionally:? ? ? ?jay at debamd64:~/dev2/cm3/m3-libs/m3core$ cat ?/home/jay/cm3.l6/bin/cm3.cfg ? ?if not defined("SL") SL = "/" end? ?HOST = "I386_LINUX"? ?TARGET = "I386_LINUX"? ?INSTALL_ROOT = (path() & SL & "..")? ?include(path() & SL & "config" & SL & TARGET)? ? ? ?You have to change them both for the existing cm3cg to be used.? ?There are ways other than changing HOST, lik3 cm3 -Dm3back=cm3cg.? ?Your tree probably needs to be all in sync first for this to work, due to the reuse of existing cm3 and cm3cg?with a fresh m3core/libm3. ?i.e. first do an upgrade.py.? ? Manually build m3core/libm3. ? ? ? ? cd $ROOT/m3-libs/m3core ? ? cm3 ? ? cm3 -ship ?? ? cd ../libm3 ? ? cm3 ? ? cm3 -ship ? ? ? ? ? ?and then scripts/python/upgrade-full.sh ? That's it. Pretty easy. If you want to skip the upgrade, you can probably first copy the LINUXLIBC6 pkg/m3core, libm3 to I386_LINUX. ?- Jay ? ________________________________ > From: jay.krell at cornell.edu > To: hendrik at topoi.pooq.com > Date: Mon, 6 Jun 2016 17:39:49 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > Fair questions. > > > LINUXLIBC6 and I386_LINUX are the same really. > > It should work to just rename a few files and change the string in the > config file. > > Overkill but should also work, *something like*: > > cd scripts/python > edit ../pkginfo.txt > remove these two lines: > odbc database > postgres95 database > rm ../PKGS # might not be needed but doesn't hurt > ./upgrade.py # be current in whatever you are using i.e. LINUXLIBC6 > ./do-cm3-all.py realclean I386_LINUX > ./do-cm3-all.py buildship I386_LINUX > # Quick test of compiler, should run and error: > ../../m3-sys/cm3/I386_LINUX/cm3 > If it fails though, we have already shipped most stuff, so not good. > I need to rewrite these steps to be more friendly if they fail. > > # and then some final small steps > ./install-cm3-compiler.py I386_LINUX > ./make-dist-cfg.py I386_LINUX # does very little, you could instead > edit your config > > > > I have to run through this still myself. > With the last two steps, it should use I386_LINUX by default you don't > have to keep saying it. > > > This is less overkill if e.g. you want to switch from LINUXLIBC6 to > AMD64_LINUX. > > > To use the C compiler, similarly: > cd scripts/python > edit ../pkginfo.txt > remove these two lines: > odbc database > postgres95 database > rm ../PKGS # might not be needed but doesn't hurt > ./upgrade.py # be current in whatever you are using -- no need to a > second time if did previous > ./do-cm3-all.py realclean I386_LINUX c # notice the c throw in there > ./do-cm3-all.py buildship I386_LINUX c # notice the c throw in there > # Quick test of compiler, should run and error: > ../../m3-sys/cm3/I386_LINUX/cm3 > If it fails though, we have already shipped most stuff, so not good. > I need to rewrite these steps to be more friendly if they fail. > > # and then some final small steps > ./install-cm3-compiler.py I386_LINUX c > ./make-dist-cfg.py I386_LINUX > > Now, the last step, if you aren't going to be using the scripts all > the time...you have to edit your bin/config/cm3cfg.common to enable the > C backend. > Where it says: > > if not defined ("M3_BACKEND_MODE") > M3_BACKEND_MODE = "3" > % -- defines how the frontend, backend, and assembler interact > % "0" -- don't call m3_backend, M3CG produces object code > % "1" -- don't call m3_backend, M3CG produces assembly code > % "2" -- call m3_backend, it produces object code > % "3" -- call m3_backend, it produces assembly code > % "C" -- use C backend, and then compile with compile_c() > end > > > change: > M3_BACKEND_MODE = "3" > > to: > M3_BACKEND_MODE = "C" > > I haven't tried these *exact* steps though. > > > We should nail the exact minimal steps and check them in to doc or > scripts/python. > I'm sure these are not minimal -- we should split it up as minimal to > get a compiler, > and then separate to build the rest of the system. And we might want > to try rename or copy instead > of rebuild. > > > Something not in-place will also be good, more like make-dist.py. > Sorry, I wanted to send more confident directions earlier but got hung > up on some stuff. > > > > - Jay > > > >> Date: Sat, 4 Jun 2016 09:32:10 -0400 >> From: hendrik at topoi.pooq.com >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> On Sat, Jun 04, 2016 at 01:08:23AM +0000, Jay K wrote: >>> Aside: >>>> LINIXLIBC >>> Any chance people will move from: LINUXLIBC6 NT386 FreeBSD4 >>> SOLgnu SOLsun to: I386_LINUX I386_NT I386_FREEBSD >>> SPARC32_SOLARIS (which internally lets you chose C compiler) >>> ? >> >> How do I switch? My compiler just uses LINIXLIBC6 whatever I do. Is >> there a way of telling it otherwise? Do I have to reinstall from >> scratch? Is there an installer that knows I386_LINUX? Last time I >> looked there didn't seem to be. >> >>> They have all been present and working for years. >>> These other names are a warty legacy. >>> The name LINUXLIBC6 has 20+ years of outdated legacy encoded in it. > 1) The assumption that Linux is x86 only. 2) The fact that libc5 back > in kernel 1.x days, pre-pthreads or at least pre-NPTL, was incompatible > with libc6. There is more motivation imho for pthreads vs. userthreads > targets, and C vs. gcc-based vs. LLVM-based targets. (LLVM-based should > be link compatible with one of them though?) >> >> And how do I ask for the C code generator? >> >> -- hendrik > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From wagner at elegosoft.com Mon Jun 27 08:10:09 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 27 Jun 2016 08:10:09 +0200 Subject: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> Message-ID: <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> On Sat, 25 Jun 2016 17:43:07 +0000 Jay K wrote: > I agree and disagree. > > Regarding Posix -- I think it did well for low level C. > > I think we have some portability in Modula-3 as a result.I agree they seemed to have done nothing for shared linking, for packaging,and almost nothing for "make", and I guess X Windows didn't need their help,at the time, people thought. > > There is a clear resemblance among the build systems.Even automake looks like cm3 from a user point of view.Even the implementation: automake "compiles" to make cm3 "compiles" to quake > Quake was written in C. > > The notion of a bunch of builtins and some escape hatch is also common.Quake is a better escape hatch than cmake. > > The notion of writing a build scripting language and/or a set of build functionsaccessible to a scripting language is common -- in scons it is all just Python,and they provide a library. The Unity?Unreal (I get these mixed up) build systemhas all the "makefiles" written in C#, and actually adding a single source fileimplies some automatic recompile, but it is a small/fast recompile, andthe resulting build system is fast, AND you can step through your build systemusing an ordinary C# debugger -- trying doing that with cm3/quake/make! > > cm3 kind of delegates to something kind of like autoconf, if you consider cminstall.But the cminstall part I never liked -- it was interactive and we still haveto maintain the choices, and it was often wrong. It was too close to the "user edits it" model. > > I'm aware of Docker. > > I think it is solving a problem. Not the entire problem -- cloud app declaration, software composition. Though it still low level -- how do I describe and join multiple containers? They invented docker-compose for that. > I'm not sure it is relevant here. > > However, Docker-induced bloat alludes to static link bloat.If we give up on dynamic linking, we get Posix-portable linking --ar + ranlib + cc/ld are very portable. I'm reluctant to do that though. I wouldn't like to give us dynamic linking, too, nor anything we have now. > I don't think C and autoconf/automake are clearly a step backward. See below. > I want more portability and, I admit, less work. I think they are great for portabilitity. Even better, generate C++ and likely gain efficient exception handling. Delegating the work to the C++ implementers, which generally do much better than us here (basically inheriting "stack walkers" wherever they exist.) > > I want to be able to declare large swaths of the system ported/portableand not have to constantly revisit them (or worry that they are neglected) > > I don't want clang vs. gccto be something we have to worry about. I want want minor forkslike Dragonfly or the new OpenBSD fork to be something that requiresour attention to work, or perhaps even anyones -- if these systems resemblePosix enough and/or use gcc, autoconf can just figure stuff out. Or maybewe just have to pick up newer libtool sometimes. > > If I write a substantial program in C or C++, and I use cmake or autoconf/libtool,I can redistribute it easily as source and people can easily rebuild and installand package it. I want our system to be similar. > > There are middle grounds that don't "step backward".We could: - accept our current platforms/ports (i.e. don't fret over AIX, etc.) - accept that the config files aren't bad asis - bootstrap can just automate slightly more and build cm3 - use cm3 to build and install everything else, including cm3cg, dynamic linking, etc. That is, we can get close to configure && make && make install w/o muchchange. Mostly like 3.6. > > One nagging point is to distribute target specific cm3 bootstraps, or bundlethem all together and have something like "configure" pick which one.You know -- does user have to pick among many downloads or just one. > We can maintain the "configure && make && make install" model, butthey wouldn't use any of autoconf/automake. > configure would determine what target to usemake would cd to that target and build cm3, statically linkedmake would then use that cm3 to the entire system -- starting with cm3cg, and then m3core The system it is building would have to be matched with the cm3 itself. It might turn around at the cm3 point and use that cm3 to rebuild. > > This is only a minor layering/scripting over what we already have. > This configure might be written in portable sh or portable C/C++, but it doesn't have to do much.The make step, boot1.py already generates makefiles, but that could be pushed into shor C/C++ if desired. > Honestly, there is a problem that I just find sh very difficult to adopt for anything.Python or C or C++ are much preferable to me.This construction does delegate mostly to Modula-3, but for some bootstrapping. > Still though, I still wonder about replacing the config files with libtool...Generating Makefile.am from the m3makefiles..and btw, we'd run "make -j" -- parallelism! > - Jay > > > To: wagner at elegosoft.com; m3devel at elegosoft.com > > From: adacore at marino.st > > Date: Sat, 25 Jun 2016 16:06:12 +0200 > > Subject: Re: [M3devel] M3 Portabilbity, Installation, Distribution -- use Docker Containers for Easy Access? > > > > On 6/25/2016 2:24 PM, Olaf Wagner wrote: > > > Lurking in the shadows and reading all the recent mails about improving > > > the build system and seeing more and more steps 'backwards' to adapt > > > to C, autoconf, makefiles and more overcome tools and concepts, a > > > heretical thought comes to mind: Have you ever considered using docker > > > as a means to provide easy access and distribution of Modula-3? > > > > You mean Linux-only Docker? > > > > In general this concept would not be compatible with typical s/w build > > repositories used by various OS. If you went to a primary Docker > > distribution system, you're basically saying A) M3 is limited to Linux > > and B) It's distributed by third parties (you) only. > > > > At first glance, I'd say it's a bad idea. > > > > John Hi Jay and everybody else, I think I've been slightly misunderstood; but that was to be expected. I agree with most of the things you say. For now I'd just like to add some more or less random comments: * "backwards" didn't mean a step back for cm3, but only that I think that the approach it takes is just pragmatical: testing for everything that might be there or not. And it was certainly not intended to criticise anybody's recent work on cm3. * I didn't really like cminstall, too, though I understand it might seem so. Replacing it with a more sophisticated configure/automake at installation time would probably help the installation very much. * I doubt that generating C or C++ will be the right way to go, though I'm not against it as an alternative way/backend. The first versions of M3 used C as intermediate language, and after some years it was said to have been "a mediocre choice at best". Been there, done that. * I think build systems need to provide a purely declarative interface for the user; m3makefiles/quake is a rather good implementation. * I don't think we should try to make the M3 compiler behave more like a C compiler; rather than stepping from handling one package at a time to compiling single sources we need to look at building complete systems at a time. * I don't think we should try to unify/combine the m3 package concept with OS installation packages. * I don't think that using docker will help with providing a better build system, it might just help to make M3 more easily available to more users and thus help to keep the community alive. * I don't think that the M3 community currently has the ressources to really solve the issues of a better build system and easy distribution with all existing OS platforms. There are just too few people and the tasks are too huge. So I thought it might be worth thinking of somehing completely different to attract more people. * I guess that to come up with a reasonable declarative new or better build system, we'd need one common stable platform for it, and separate this task from those of achieving portability for all platforms. The platform might be an interpreter (like UCSD Pascal used or the JVM) or a really stable system and library interface (like an extended POSIX layer) or something completely new ;-) LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctly. Using an interpreter would also give up one of the big advantages of current CM3, that it can be compiled and linked natively on OSs. Finally: There are too many sentences starting with "I". Please excuse me. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 27 08:31:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 06:31:47 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: redirecting... > Olaf > LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl I would like to take this up, maybe soon. I do have a bit of an agenda. Maybe my priorities are mixed up. 1 Provide a very portable system. 2 Provide an easy to install and use system. 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least for their supported backends). 4 Maybe write our own backends. Where is the LLVM support at? Mostly working? Barely working? I know LLVM is big and changing, and maybe they don't value compatibility of bitcode. But look at what we have with the gcc backend. Even if we didn't have to patch it at all, I expect we'd still have to keep and build a local copy. Perhaps we should just do that? With LLVM, with its different licensing, perhaps we could get our "frontend" merged upstream, but this would then give us a compatibility burden in the persisted m3cg. Is that ok? It is hypothetical at this point. I know everyone here doesn't really like C/C++ (except me). And, more significantly, I know the system written in itself is a great test case, but I wonder if we shouldn't write a new "real" frontend in C or C++, and see if we can't merge that upstream with gcc and/or clang. It also worth mentioning that I believe gcc's Ada front end is written in Ada -- you don't actually have to write the frontend in C/C++ to merge upstream. But there might remain licensing concern. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 27 08:39:12 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 06:39:12 +0000 Subject: [M3devel] parse.c licensing question, dual? Message-ID: - I basically understand licensing. - I understand GPL - I understand more liberal BSD license - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. What I do not understand: - dual licensing - who owns parse.c - can parse.c be dual licensed? In particular: jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ m3-parse.h parse.c m3-def.h m3cg.h Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly in-process ones, either call-based or "linearized IR in memory". In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. m3cg.h is output by m3cggen. m3-def.h I wrote. These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. Can we also BSD license them or such? (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Mon Jun 27 08:53:33 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Mon, 27 Jun 2016 08:53:33 +0200 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: Message-ID: <20160627085333.916aae7db6aa078d5300c739@elegosoft.com> On Mon, 27 Jun 2016 06:39:12 +0000 Jay K wrote: > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. > > > What I do not understand: > - dual licensing > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? > > (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) My understand is that you can put any license on things you wrote yourself. I'm not really sure, but I doubt that there is any legal entity left that cares for the M3 sources from DEC SRC (if it is that old). So I _think_ that we (you) might change the copyright for those. To be more compatible with the GNU stuff, it might be better to use LGPL together with the gcc backend. I am not a lawyer though. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Mon Jun 27 10:16:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 08:16:06 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> Message-ID: I'm going to read Olaf's assertion as 1 "go from GPL to LGPL", not 2 "go from DEC to LGPL". Imho we should use the OpenBSD or FreeBSD or NetBSD licenses. The OpenBSD folks are..funny but right-seeming. Their take for example on the Apache 2.0 license is -- why another long license? They'd need to pay a lawyer to allow for it. So just don't allow for it. They have stayed back with Apache 1.x for this reason. So you should reuse an existing short license. So, sorry, another question I forgot to ask -- who owns parse.c? Can we relicense it? And, ok, I own m3-def.h. I can just paste two license into it? I'll research the old Qt story here I guess or otherwise research. There is also a claim that the FreeBSD license is GPL-compatible, which implies we can use it on parse.c/m3-def.h -- just a single license. That is clearer to me. I just understand what it means to have two licences, unless, e.g. the license is context-dependent -- different people get different license depending on situation, like if they paid, or if they are getting paid. I think I was going to use m3-def.h/parse.c in the C backend, writing it in C or C++, and the intervening layer that allows easily writing multiple passes over the IR. The result instead was incredibly tedious and makes changing the IR more difficult/tedious. If I embark on an LLVM backend, I'll again be tempted to do that. It would be nice if we could relicense all the DEC SRC stuff as slightly more liberal BSD. I see DEC SRC seems to have an optional give back clause -- if you give your changes back to DEC, then you also license it to them liberally. But give back doesn't seem mandatory. 4. Improvements. LICENSEE hereby grants to DIGITAL a non-exclusive, non-transferable, royalty free right to use, modify, reproduce and distribute with the right to sublicense at any tier, any improvements, enhancements, extensions, or modifications that LICENSEE make to SOFTWARE, provided such are returned to DIGITAL by LICENSEE. - Jay From: hosking at purdue.edu To: wagner at elegosoft.com CC: jay.krell at cornell.edu; m3devel at elegosoft.com Subject: Re: [M3devel] parse.c licensing question, dual? Date: Mon, 27 Jun 2016 06:55:17 +0000 Please be careful here. Going from the current license to LGPL is probably not the best route for CM3! On 27 Jun 2016, at 4:53 PM, Olaf Wagner wrote: On Mon, 27 Jun 2016 06:39:12 +0000 Jay K wrote: - I basically understand licensing. - I understand GPL - I understand more liberal BSD license - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. What I do not understand: - dual licensing - who owns parse.c - can parse.c be dual licensed? In particular: jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ m3-parse.h parse.c m3-def.h m3cg.h Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly in-process ones, either call-based or "linearized IR in memory". In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. m3cg.h is output by m3cggen. m3-def.h I wrote. These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. Can we also BSD license them or such? (and broken record, but m3-def.h...we could really use some sort of preprocessor for Modula-3, maybe...this form of C/C++ is super useful...) My understand is that you can put any license on things you wrote yourself. I'm not really sure, but I doubt that there is any legal entity left that cares for the M3 sources from DEC SRC (if it is that old). So I _think_ that we (you) might change the copyright for those. To be more compatible with the GNU stuff, it might be better to use LGPL together with the gcc backend. I am not a lawyer though. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Jun 27 10:37:12 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 08:37:12 +0000 Subject: [M3devel] getting all registers in gc? Message-ID: I just noticed this in the Boehm GC documentation: ?- Changed the alpha port to use the generic register scanning code instead ? ?of alpha_mach_dep.s. ?Alpha_mach_dep.s doesn't look for pointers in fp ? ?registers, but gcc sometimes spills pointers there. ?(Thanks to Manuel ? ?Serrano for helping me debug this by email.) ?Changed the IA64 code to ? ?do something similar for similar reasons. This would seem like a hazard for us too. And not convincingly Alpha/IA64-specific. We basically assume setjmp stores a context, or at least all live pointers. In hindsight I see two problems: ?- one alluded to -- jmpbuf might not have floating point registers, ? ?and floating point registers might have pointers. ?- Same thing but more general: jmpbuf might not even have all integer ? ?registers? ? ? So that leaves the question "What is generic register scanning code"? I don't know yet but..thinking... ?Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? ? ?I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, ?as I think that is a lingering thing we need to handle to finish our portability. Ignoring IA64 for now, maybe here: void __cdecl ThreadPThread__sigsuspend(void) { ? struct { ? ? sigjmp_buf jb; ? } s; ? ZERO_MEMORY(s); ? if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ #ifdef M3_REGISTER_WINDOWS ? ? siglongjmp(s.jb, 1); /* flush register windows */ ? else #endif ? ? sigsuspend(&mask); } ? ? ?and here: ? ?void __cdecl ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) { ? struct { ? ? sigjmp_buf jb; ? } s; ? ZERO_MEMORY(s); ? if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ we should use getcontext/RtlCaptureContext/GetThreadContext? I'll look more at the Boehm code. ?- Jay ? From jay.krell at cornell.edu Mon Jun 27 11:14:13 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 09:14:13 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: References: Message-ID: ok, the Boehm code: For the current live thread, merely: ? ? ? ? ? ? ? ? ? ? ? ? /* Push enough of the current stack eagerly to ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* ensure that callee-save registers saved in ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* GC frames are scanned. ? ? ? ? ? ? ? ? ? ? ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* In the non-threads case, schedule entire ? ? */ ? ? ? ? ? ? ? ? ? ? ? ? /* stack for scanning. ? ? ? ? ? ? ? ? ? ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* The second argument is a pointer to the ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* (possibly null) thread context, for ? ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* (currently hypothetical) more precise ? ? ? ?*/ ? ? ? ? ? ? ? ? ? ? ? ? /* stack scanning. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ /* ?* In the absence of threads, push the stack contents. ?* In the presence of threads, push enough of the current stack ?* to ensure that callee-save registers saved in collector frames have been ?* seen. ?* FIXME: Merge with per-thread stuff. ?*/ /*ARGSUSED*/ STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) { # ? if defined(THREADS) ? ? ? ? if (0 == cold_gc_frame) return; # ? ? ? ifdef STACK_GROWS_DOWN ? ? ? ? ? GC_push_all_eager(GC_approx_sp(), cold_gc_frame); ? ? ? ? ? /* For IA64, the register stack backing store is handled ? ? ?*/ ? ? ? ? ? /* in the thread-specific code. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? */ # ? ? ? else ? ? ? ? ? GC_push_all_eager(cold_gc_frame, GC_approx_sp()); # ? ? ? endif # ? else ... # ? endif /* !THREADS */ GC_INNER ptr_t GC_approx_sp(void) { ? ? volatile word sp; ? ? sp = (word)&sp; ? ? ? ? ? ? ? ? /* Also force stack to grow if necessary. Otherwise the */ ? ? ? ? ? ? ? ? /* later accesses might cause the kernel to think we're */ ? ? ? ? ? ? ? ? /* doing something wrong. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? */ ? ? return((ptr_t)sp); ? ? ? ? ? ? ? ? /* GNU C: alternatively, we may return the value of ? ? */ ? ? ? ? ? ? ? ? /*__builtin_frame_address(0). ? ? ? ? ? ? ? ? ? ? ? ? ? */ } Notice that it doesn't even do what it says -- no attempt to save registers to stack. but for suspended threads it is more convincing: /* Ensure that either registers are pushed, or callee-save registers ? ?*/ /* are somewhere on the stack, and then call fn(arg, ctxt). ? ? ? ? ? ? */ /* ctxt is either a pointer to a ucontext_t we generated, or NULL. ? ? ?*/ GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ptr_t arg) { ? ? volatile int dummy; ? ? void * context = 0; .. .... ?a mix of methods:? ? - sometimes processor specific assembly? ? - sometimes getcontext, and then workaround a bug on Linux/amd64? ? - and then _setjmp on Unix? ? - setjmp on Windows? ?getcontext to me seems more promising than setjmp, ?and you can use both ? ? for Win32, I suggest RtlCaptureContext (for live thread too) ?? ?? ? We maybe should copy getcontext from various BSDs? ? i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need ? to worry about the glibc getcontext bug). or maybe just getcontext. The gradually expanding register set on x86 makes me nervous that this isn't a maintenance problem, but I'm guessing you never get pointers spilled to the ymm/zmm registers. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Mon, 27 Jun 2016 08:37:12 +0000 > Subject: [M3devel] getting all registers in gc? > > I just noticed this in the Boehm GC documentation: > > - Changed the alpha port to use the generic register scanning code instead > of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp > registers, but gcc sometimes spills pointers there. (Thanks to Manuel > Serrano for helping me debug this by email.) Changed the IA64 code to > do something similar for similar reasons. > > > This would seem like a hazard for us too. > > And not convincingly Alpha/IA64-specific. > > We basically assume setjmp stores a context, or at least all live pointers. > > > In hindsight I see two problems: > - one alluded to -- jmpbuf might not have floating point registers, > and floating point registers might have pointers. > > > - Same thing but more general: jmpbuf might not even have all integer > registers? > > > > So that leaves the question "What is generic register scanning code"? > > I don't know yet but..thinking... > > Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? > > I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, > as I think that is a lingering thing we need to handle to finish our portability. > > Ignoring IA64 for now, maybe here: > > void > __cdecl > ThreadPThread__sigsuspend(void) > { > struct { > sigjmp_buf jb; > } s; > > ZERO_MEMORY(s); > > if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ > #ifdef M3_REGISTER_WINDOWS > siglongjmp(s.jb, 1); /* flush register windows */ > else > #endif > sigsuspend(&mask); > } > > > and here: > > void > __cdecl > ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) > { > struct { > sigjmp_buf jb; > } s; > > ZERO_MEMORY(s); > > if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ > > > we should use getcontext/RtlCaptureContext/GetThreadContext? > > > I'll look more at the Boehm code. > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Mon Jun 27 16:59:20 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 10:59:20 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: <20160627145920.GA10803@topoi.pooq.com> On Mon, Jun 27, 2016 at 06:31:47AM +0000, Jay K wrote: > I know everyone here doesn't really like C/C++ (except me). > And, more significantly, I know the system written in itself > is a great test case, but I wonder if we shouldn't write a new > "real" frontend in C or C++, and see if we can't merge that > upstream with gcc and/or clang. Replacing the entire front end with a new one, written from scratch, and not just a machine-translation or a hand-translation of the old one, would enable us to deal with the licincing issues about the M3 compiler. Then only the libraries will be stuck with the wrong licence. OCaml might also be a good language in which to write a new compiler. > > It also worth mentioning that I believe gcc's Ada front end > is written in Ada -- you don't actually have to write > the frontend in C/C++ to merge upstream. > > But there might remain licensing concern. -- hendrik From hendrik at topoi.pooq.com Mon Jun 27 17:35:12 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 11:35:12 -0400 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: Message-ID: <20160627153512.GB10803@topoi.pooq.com> On Mon, Jun 27, 2016 at 06:39:12AM +0000, Jay K wrote: > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but everyone seems to define it "like how static libraries work", "maybe with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL licensed, linked to other GPL licensed code, and does not "link" to cm3, so does not infect the cm3 runtime, so does not infect all the Modula-3 code. > > > What I do not understand: > - dual licensing Here's how dual licensing works. If you release a work and grant several public licences to it, the public can use it under any of tthe licences that have been granted to it. So if you've granted rights under license A and under license B, someone for who doesn't like th rerms of licence A can if she chooses, use the rights granted under license B instead. Don't know what's clear about that. nIn particular, I believe any new components of CM3 should be licensed under both the existing licence and the GPL or LGPL, and perhaps under MIT as well. Then gradually, as we write new components, the entire ecosystem will become more and more compatible with the commonly used free licenses. It isi the copyright owner who determines what license(s) to release stuff under. If you write it, you own the copyright, unless it is work-for-hire, in which case your employer does. We do not own the copyright of the SRC Modula 3 system, so we cannot relicence it. What we can do, given enough time, is write a new one. -- hendrik > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? The copyright owner can do this. The copyright owner can license them under as many licences as he wishes. No one else can. -- hendrik From jay.krell at cornell.edu Mon Jun 27 22:00:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:00:06 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: Can the owner relicense parse.c, or it is stuck with GPL because it links to gcc? And then -- who owns it? Digital and Critical Mass? ?- Jay ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: wagner at elegosoft.com; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 09:51:31 +0000 > > parse.c is contaminated by GPL. > > On 27 Jun 2016, at 6:16 PM, Jay K > > wrote: > > I'm going to read Olaf's assertion as 1 "go from GPL to LGPL", not 2 > "go from DEC to LGPL". > > > Imho we should use the OpenBSD or FreeBSD or NetBSD licenses. > > > The OpenBSD folks are..funny but right-seeming. > Their take for example on the Apache 2.0 license is -- why another long > license? They'd need to pay a lawyer to allow for it. So just don't > allow for it. They have stayed back with Apache 1.x for this reason. > > > So you should reuse an existing short license. > > > So, sorry, another question I forgot to ask -- who owns parse.c? > Can we relicense it? > > > And, ok, I own m3-def.h. I can just paste two license into it? > I'll research the old Qt story here I guess or otherwise research. > > There is also a claim that the FreeBSD license is GPL-compatible, which > implies we can use it on parse.c/m3-def.h -- just a single license. > That is clearer to me. I just understand what it means to have two > licences, unless, e.g. the license is context-dependent -- different > people get different license depending on situation, like if they paid, > or if they are getting paid. > > > I think I was going to use m3-def.h/parse.c in the C backend, writing > it in C or C++, and the intervening layer that allows easily writing > multiple passes over the IR. The result instead was incredibly tedious > and makes changing the IR more difficult/tedious. > If I embark on an LLVM backend, I'll again be tempted to do that. > > > It would be nice if we could relicense all the DEC SRC stuff as > slightly more liberal BSD. > I see DEC SRC seems to have an optional give back clause -- if you give > your changes back to DEC, then you also license it to them liberally. > But give back doesn't seem mandatory. > > 4. Improvements. LICENSEE hereby grants to DIGITAL a non-exclusive, > non-transferable, royalty free right to use, modify, reproduce > and distribute with the right to sublicense at any tier, any > improvements, enhancements, extensions, or modifications that > LICENSEE make to SOFTWARE, provided such are returned to DIGITAL > by LICENSEE. > > > - Jay > > > ________________________________ > From: hosking at purdue.edu > To: wagner at elegosoft.com > CC: jay.krell at cornell.edu; > m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 06:55:17 +0000 > > Please be careful here. > Going from the current license to LGPL is probably not the best route > for CM3! > > On 27 Jun 2016, at 4:53 PM, Olaf Wagner > > wrote: > > On Mon, 27 Jun 2016 06:39:12 +0000 > Jay K > wrote: > > - I basically understand licensing. > - I understand GPL > - I understand more liberal BSD license > - I understand that the notion of "linking" hasn't been defined, but > everyone seems to define it "like how static libraries work", "maybe > with dynamic linking", and definitely not with "process boundaries". > - So cm3 calls gcc across a process boundary, and parse.c is GPL > licensed, linked to other GPL licensed code, and does not "link" to > cm3, so does not infect the cm3 runtime, so does not infect all the > Modula-3 code. > > > What I do not understand: > - dual licensing > - who owns parse.c > - can parse.c be dual licensed? > > In particular: > jair:mips jay$ edit /dev2/cm3.4/m3-sys/m3cc/gcc/gcc/m3cg/ > m3-parse.h parse.c > m3-def.h m3cg.h > > > Some of these files would be useful in other backends, structured like > the cm3cg backend at least, and possibly > in-process ones, either call-based or "linearized IR in memory". > > In particular m3-def.h and m3cg.h. I would like to maybe reuse these in > non-GPL code. > > m3cg.h is output by m3cggen. > m3-def.h I wrote. > > These files need to be at least be GPL licensed since they are used by > parse.c and linked to the overall gcc backend. > Can we also BSD license them or such? > > (and broken record, but m3-def.h...we could really use some sort of > preprocessor for Modula-3, maybe...this form of C/C++ is super > useful...) > > My understand is that you can put any license on things you wrote yourself. > I'm not really sure, but I doubt that there is any legal entity left that > cares for the M3 sources from DEC SRC (if it is that old). So I _think_ > that we (you) might change the copyright for those. > > To be more compatible with the GNU stuff, it might be better to use > LGPL together with the gcc backend. > > I am not a lawyer though. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From jay.krell at cornell.edu Mon Jun 27 22:04:27 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:04:27 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu> References: , , <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu> Message-ID: How do we ensure that? What about other backends, e.g. C or LLVM? The Boehm collector code implies the situation is under control, if we are like it, at least. I think getcontext will go far toward providing peace of mind on many platforms (and RtlCaptureContext on NT). I might still be convinced that setjmp suffices. In that, compiler will spill volatiles ahead of it anyway. I wonder if that suffices, but I haven't convinced myself. ?- Jay ---------------------------------------- > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] getting all registers in gc? > Date: Mon, 27 Jun 2016 09:50:06 +0000 > > A thought: can we simply ensure that the gcc-based backend never spills pointers to FP regs? > >> On 27 Jun 2016, at 7:14 PM, Jay K wrote: >> >> ok, the Boehm code: >> >> For the current live thread, merely: >> >> /* Push enough of the current stack eagerly to */ >> /* ensure that callee-save registers saved in */ >> /* GC frames are scanned. */ >> /* In the non-threads case, schedule entire */ >> /* stack for scanning. */ >> /* The second argument is a pointer to the */ >> /* (possibly null) thread context, for */ >> /* (currently hypothetical) more precise */ >> /* stack scanning. */ >> /* >> * In the absence of threads, push the stack contents. >> * In the presence of threads, push enough of the current stack >> * to ensure that callee-save registers saved in collector frames have been >> * seen. >> * FIXME: Merge with per-thread stuff. >> */ >> /*ARGSUSED*/ >> STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) >> { >> # if defined(THREADS) >> if (0 == cold_gc_frame) return; >> # ifdef STACK_GROWS_DOWN >> GC_push_all_eager(GC_approx_sp(), cold_gc_frame); >> /* For IA64, the register stack backing store is handled */ >> /* in the thread-specific code. */ >> # else >> GC_push_all_eager(cold_gc_frame, GC_approx_sp()); >> # endif >> # else >> ... >> # endif /* !THREADS */ >> >> >> GC_INNER ptr_t GC_approx_sp(void) >> { >> volatile word sp; >> sp = (word)&sp; >> /* Also force stack to grow if necessary. Otherwise the */ >> /* later accesses might cause the kernel to think we're */ >> /* doing something wrong. */ >> return((ptr_t)sp); >> /* GNU C: alternatively, we may return the value of */ >> /*__builtin_frame_address(0). */ >> } >> >> >> Notice that it doesn't even do what it says -- no attempt >> to save registers to stack. >> >> >> but for suspended threads it is more convincing: >> >> >> /* Ensure that either registers are pushed, or callee-save registers */ >> /* are somewhere on the stack, and then call fn(arg, ctxt). */ >> /* ctxt is either a pointer to a ucontext_t we generated, or NULL. */ >> GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), >> ptr_t arg) >> { >> volatile int dummy; >> void * context = 0; >> >> >> .. >> .... >> >> >> a mix of methods: >> - sometimes processor specific assembly >> - sometimes getcontext, and then workaround a bug on Linux/amd64 >> - and then _setjmp on Unix >> - setjmp on Windows >> >> >> getcontext to me seems more promising than setjmp, >> and you can use both >> >> for Win32, I suggest RtlCaptureContext (for live thread too) >> >> >> We maybe should copy getcontext from various BSDs? >> i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need >> to worry about the glibc getcontext bug). >> >> or maybe just getcontext. The gradually expanding register set on x86 makes me nervous >> that this isn't a maintenance problem, but I'm guessing you never get pointers >> spilled to the ymm/zmm registers. >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Date: Mon, 27 Jun 2016 08:37:12 +0000 >>> Subject: [M3devel] getting all registers in gc? >>> >>> I just noticed this in the Boehm GC documentation: >>> >>> - Changed the alpha port to use the generic register scanning code instead >>> of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp >>> registers, but gcc sometimes spills pointers there. (Thanks to Manuel >>> Serrano for helping me debug this by email.) Changed the IA64 code to >>> do something similar for similar reasons. >>> >>> >>> This would seem like a hazard for us too. >>> >>> And not convincingly Alpha/IA64-specific. >>> >>> We basically assume setjmp stores a context, or at least all live pointers. >>> >>> >>> In hindsight I see two problems: >>> - one alluded to -- jmpbuf might not have floating point registers, >>> and floating point registers might have pointers. >>> >>> >>> - Same thing but more general: jmpbuf might not even have all integer >>> registers? >>> >>> >>> >>> So that leaves the question "What is generic register scanning code"? >>> >>> I don't know yet but..thinking... >>> >>> Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? >>> >>> I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, >>> as I think that is a lingering thing we need to handle to finish our portability. >>> >>> Ignoring IA64 for now, maybe here: >>> >>> void >>> __cdecl >>> ThreadPThread__sigsuspend(void) >>> { >>> struct { >>> sigjmp_buf jb; >>> } s; >>> >>> ZERO_MEMORY(s); >>> >>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>> #ifdef M3_REGISTER_WINDOWS >>> siglongjmp(s.jb, 1); /* flush register windows */ >>> else >>> #endif >>> sigsuspend(&mask); >>> } >>> >>> >>> and here: >>> >>> void >>> __cdecl >>> ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) >>> { >>> struct { >>> sigjmp_buf jb; >>> } s; >>> >>> ZERO_MEMORY(s); >>> >>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>> >>> >>> we should use getcontext/RtlCaptureContext/GetThreadContext? >>> >>> >>> I'll look more at the Boehm code. >>> >>> >>> - Jay >>> >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From lemming at henning-thielemann.de Mon Jun 27 22:17:38 2016 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 27 Jun 2016 22:17:38 +0200 (CEST) Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: On Mon, 27 Jun 2016, Jay K wrote: > Can the owner relicense parse.c, Sure, the owner can always relicense. > or it is stuck with GPL because it links to gcc? He can choose any license that is compatible with GPL. From jay.krell at cornell.edu Mon Jun 27 22:19:35 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:19:35 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , Message-ID: Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). But parse.c might be ownerless and stuck? ? - Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 22:17:38 +0200 > From: lemming at henning-thielemann.de > To: jay.krell at cornell.edu > CC: hosking at purdue.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > > > On Mon, 27 Jun 2016, Jay K wrote: > >> Can the owner relicense parse.c, > > Sure, the owner can always relicense. > >> or it is stuck with GPL because it links to gcc? > > He can choose any license that is compatible with GPL. From rodney_bates at lcwb.coop Mon Jun 27 22:19:30 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 15:19:30 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> Message-ID: <57718A52.3030609@lcwb.coop> On 06/27/2016 01:31 AM, Jay K wrote: > redirecting... > > > Olaf > >> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl > My big lament about llvm regards producing Dwarf debug info. Having debug info in this vastly superior format is one of my main reasons for wanting an llvm back end. From the llvm documentation, it sounded like it would do this, including altering debug info as needed to match any optimizations done. Then I hit a brick wall, finding out llvm only handles a severe subset of Dwarf, apparently what they felt was needed for C & C++. More below. > > I would like to take this up, maybe soon. > > > I do have a bit of an agenda. > Maybe my priorities are mixed up. > > > 1 Provide a very portable system. > 2 Provide an easy to install and use system. > 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least > for their supported backends). > 4 Maybe write our own backends. > > > Where is the LLVM support at? > Mostly working? Barely working? I could swear I remember getting the llvm back end to recompile everything in the m3front group twice and converge to identical-sized compiled code, and that I said so in either a commit message or an m3devel list post. But I couldn't find any such statement. This is on AMD64_LINUX, and if you are careful not to ask m3llvm for debug info at all. Otherwise, there are crashes in m3llvm and/or llc, trying to provide Dwarf. This is what I was working on when I made the disappointing discoveries. If my memory is correct here, this means the llvm back end is close to usable for building. Also, again from memory, I think the llvm back end has no failures of test cases that do not also fail using m3cg. There are still other questions, though. So far, there are no changes required to the stock llvm version being used (3.6.1) But it still requires a build of that on the machine where the M3 compilation is done, and it is big and slow to build. Lots of llvm libraries have to be linked in to m3llvm, and separate executable llc needs to be available. The llvm folk are constantly making major API changes, with no explanation other than diffing successive versions of header files, so there is no practical chance of using whatever llvm version may be already installed on a particular host machine. In light of that, I suppose we could fork and modify a specific version of llvm and put its source code in our own repository, similar to m3cg. But maintenance work on llvm is, to my standards, a real nightmare. Just about every single identifier and operator you see involves a needle-in-haystack search to locate its declaration, which could be just about anywhere, in order to know what it is. And no, the names and operator spellings are not close to adequate to clue you in. They have gone to every length possible to use every clever new C++ "feature" that comes out in the latest C++- standard, which always just increases the complexity of the search to a declaration. So I don't fancy doing any of this. (BTW, =17 in recent discussions.) As for the debug info problem, there has been talk in the llvm list of a rework that allows a front end to directly produce true Dwarf (presumably not a subset) for type info only, which would not be changed by optimizations and so llvm would not need to pass it through in its own, different, representation. I don't know how far this has gone. If/when it happens, utilizing it would entail constructing new bindings to altered llvm APIs. This in itself is an extremely tedious and error-prone task that I hoped, after doing it for llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, there is no type checking possible, so nitty little errors will only show up as hard-to-diagnose runtime errors. That is made even worse by the lack of a debugger that handles both languages. Or, we could use Dragisha's rtinfo library to get type info for a debugger from the .M3WEB files, and Dwarf for the rest. Off hand, this sounds like the best approach to me. Stick with llvm 3.6.1 as long a possible, and avoid more pain. > > > I know LLVM is big and changing, and maybe they don't value > compatibility of bitcode. Actually, keeping their bitcode stable across llvm releases is one place they do talk about compatibility. But m3llvm uses calls to llvm APIs to construct llvm IR as in-memory data, then another call to get llvm to convert it to bitcode. So bitcode's stability is irrelevant to us. I once thought about producing llvm bitcode directly, but that seems like a pretty big job. It would, however, obviate creating most of those wretched bindings. > > > But look at what we have with the gcc backend. > Even if we didn't have to patch it at all, I expect > we'd still have to keep and build a local copy. > > Perhaps we should just do that? > > With LLVM, with its different licensing, perhaps we could > get our "frontend" merged upstream, but this would > then give us a compatibility burden in the persisted m3cg. > Is that ok? > It is hypothetical at this point. > > > I know everyone here doesn't really like C/C++ (except me). > And, more significantly, I know the system written in itself > is a great test case, but I wonder if we shouldn't write a new > "real" frontend in C or C++, and see if we can't merge that > upstream with gcc and/or clang. > > > It also worth mentioning that I believe gcc's Ada front end > is written in Ada -- you don't actually have to write > the frontend in C/C++ to merge upstream. Yes, I once worked on this front end. As I understood, SRC M3 was not merged into gcc only because RMS was annoyed that SRC made it a separate executable, so its license was not poisoned by GPL. > > But there might remain licensing concern. > > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Jun 27 22:26:39 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 15:26:39 -0500 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , Message-ID: <57718BFF.1070501@lcwb.coop> On 06/27/2016 03:19 PM, Jay K wrote: > Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). > > But parse.c might be ownerless and stuck? Isn't there some law that if more than 25% of a work is changed, it ceases to be a derived work but becomes a new work by the changer? If so, I would guess the current parse.c would meet this criterion, and I think Jay would own it, since I think he has done the majority of the changes. IANAL. > > - Jay > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 22:17:38 +0200 >> From: lemming at henning-thielemann.de >> To: jay.krell at cornell.edu >> CC: hosking at purdue.edu; m3devel at elegosoft.com >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> >> On Mon, 27 Jun 2016, Jay K wrote: >> >>> Can the owner relicense parse.c, >> >> Sure, the owner can always relicense. >> >>> or it is stuck with GPL because it links to gcc? >> >> He can choose any license that is compatible with GPL. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Jun 27 22:28:57 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:28:57 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> Message-ID: Regarding debug info. I have thought about this. Various optimizing compilers make some effort to produce some debug info. And as much as they don't optimize, they should do that. However I believe in general, optimization and describability with any debug format, is not a solvable problem. There are many points in a program where things aren't particularly transformed/lost by the optimizer. Things tend to be clearly defined at the start of a function. Variable locations can be described over ranges of code as either being in a particular register, or register + offset (frame), or nowhere. But line numbers, as critical as they are, I think are the most untenable aspect. Compilers can move code around arbitrarily, including removing it. The C backend also has good debugging as a goal. We should be able to have structs with fields, instead of just byte arrays. And all the parameters/locals should have good names. ? This I messed up slightly and will try to fix. Every identifier ? gets an appended number for uniqueness, which is sometimes but rarely needed. ?? Does the current LLVM backend produce any debug info or none? I probably avoid any approach that requires writing bindings. Unless it is to our own code, like I did for Posix. Otherwise keeping things up to date is tedious and error prone and errors are fatal. To our own code is just slightly better. Maybe I'll write bitcode. Or go through the persisted cm3cg IR that m3cc uses, but it sounds like you are super far along and we should build on that. I have other things to do first though and I can't promise that I won't reinvent eventually. ?- Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 15:19:30 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 01:31 AM, Jay K wrote: >> redirecting... >> >>> Olaf >> >>> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl >> > > My big lament about llvm regards producing Dwarf debug info. Having > debug info in this vastly superior format is one of my main reasons > for wanting an llvm back end. From the llvm documentation, it sounded > like it would do this, including altering debug info as needed to > match any optimizations done. Then I hit a brick wall, finding out > llvm only handles a severe subset of Dwarf, apparently what they felt > was needed for C & C++. More below. > >> >> I would like to take this up, maybe soon. >> >> >> I do have a bit of an agenda. >> Maybe my priorities are mixed up. >> >> >> 1 Provide a very portable system. >> 2 Provide an easy to install and use system. >> 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least >> for their supported backends). >> 4 Maybe write our own backends. >> >> >> Where is the LLVM support at? >> Mostly working? Barely working? > > I could swear I remember getting the llvm back end to recompile > everything in the m3front group twice and converge to identical-sized > compiled code, and that I said so in either a commit message or > an m3devel list post. But I couldn't find any such statement. > > This is on AMD64_LINUX, and if you are careful not to ask m3llvm > for debug info at all. Otherwise, there are crashes in m3llvm > and/or llc, trying to provide Dwarf. This is what I was working > on when I made the disappointing discoveries. If my memory is > correct here, this means the llvm back end is close to usable > for building. > > Also, again from memory, I think the llvm back end has no failures > of test cases that do not also fail using m3cg. > > There are still other questions, though. So far, there are no > changes required to the stock llvm version being used (3.6.1) > But it still requires a build of that on the machine where the > M3 compilation is done, and it is big and slow to build. Lots > of llvm libraries have to be linked in to m3llvm, and separate > executable llc needs to be available. > > The llvm folk are constantly making major API changes, with no > explanation other than diffing successive versions of header > files, so there is no practical chance of using whatever llvm > version may be already installed on a particular host machine. > > In light of that, I suppose we could fork and modify a specific > version of llvm and put its source code in our own repository, > similar to m3cg. But maintenance work on llvm is, to my > standards, a real nightmare. Just about every single identifier > and operator you see involves a needle-in-haystack search to > locate its declaration, which could be just about anywhere, > in order to know what it is. > > And no, the names and operator spellings are not close to adequate > to clue you in. They have gone to every length possible to use > every clever new C++ "feature" that comes out in the latest > C++- standard, which always just increases the complexity > of the search to a declaration. So I don't fancy doing any of > this. (BTW, =17 in recent discussions.) > > As for the debug info problem, there has been talk in the llvm > list of a rework that allows a front end to directly produce > true Dwarf (presumably not a subset) for type info only, which > would not be changed by optimizations and so llvm would not > need to pass it through in its own, different, representation. > I don't know how far this has gone. > > If/when it happens, utilizing it would entail constructing new > bindings to altered llvm APIs. This in itself is an extremely > tedious and error-prone task that I hoped, after doing it for > llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, > there is no type checking possible, so nitty little errors will > only show up as hard-to-diagnose runtime errors. That is made > even worse by the lack of a debugger that handles both languages. > > Or, we could use Dragisha's rtinfo library to get type info for > a debugger from the .M3WEB files, and Dwarf for the rest. Off > hand, this sounds like the best approach to me. Stick with > llvm 3.6.1 as long a possible, and avoid more pain. > >> >> >> I know LLVM is big and changing, and maybe they don't value >> compatibility of bitcode. > > Actually, keeping their bitcode stable across llvm releases is > one place they do talk about compatibility. But m3llvm uses calls > to llvm APIs to construct llvm IR as in-memory data, then another > call to get llvm to convert it to bitcode. So bitcode's stability > is irrelevant to us. I once thought about producing llvm bitcode > directly, but that seems like a pretty big job. It would, however, > obviate creating most of those wretched bindings. > >> >> >> But look at what we have with the gcc backend. >> Even if we didn't have to patch it at all, I expect >> we'd still have to keep and build a local copy. >> >> Perhaps we should just do that? >> >> With LLVM, with its different licensing, perhaps we could >> get our "frontend" merged upstream, but this would >> then give us a compatibility burden in the persisted m3cg. >> Is that ok? >> It is hypothetical at this point. >> >> >> I know everyone here doesn't really like C/C++ (except me). >> And, more significantly, I know the system written in itself >> is a great test case, but I wonder if we shouldn't write a new >> "real" frontend in C or C++, and see if we can't merge that >> upstream with gcc and/or clang. >> >> >> It also worth mentioning that I believe gcc's Ada front end >> is written in Ada -- you don't actually have to write >> the frontend in C/C++ to merge upstream. > > Yes, I once worked on this front end. As I understood, SRC M3 > was not merged into gcc only because RMS was annoyed that SRC made > it a separate executable, so its license was not poisoned by GPL. > >> >> But there might remain licensing concern. >> >> >> >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From lemming at henning-thielemann.de Mon Jun 27 22:29:08 2016 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Mon, 27 Jun 2016 22:29:08 +0200 (CEST) Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: On Mon, 27 Jun 2016, Rodney M. Bates wrote: > And no, the names and operator spellings are not close to adequate > to clue you in. They have gone to every length possible to use > every clever new C++ "feature" that comes out in the latest > C++- standard, which always just increases the complexity > of the search to a declaration. So I don't fancy doing any of > this. (BTW, =17 in recent discussions.) A teacher of mine called this behavior "version junkie". > Actually, keeping their bitcode stable across llvm releases is > one place they do talk about compatibility. But m3llvm uses calls > to llvm APIs to construct llvm IR as in-memory data, then another > call to get llvm to convert it to bitcode. So bitcode's stability > is irrelevant to us. I once thought about producing llvm bitcode > directly, but that seems like a pretty big job. It would, however, > obviate creating most of those wretched bindings. An alternative would be to create .ll text files. But its format changes, too. From jay.krell at cornell.edu Mon Jun 27 22:30:57 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:30:57 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: <57718BFF.1070501@lcwb.coop> References: , ,,<20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, ,,<653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, ,,, , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , , , , <57718BFF.1070501@lcwb.coop> Message-ID: Some number should be reasonable yes, but I think you exaggerate how much I've done. :) I'll look later. ?- Jay ---------------------------------------- > Date: Mon, 27 Jun 2016 15:26:39 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > > > > On 06/27/2016 03:19 PM, Jay K wrote: >> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >> >> But parse.c might be ownerless and stuck? > > Isn't there some law that if more than 25% of a work is changed, it ceases > to be a derived work but becomes a new work by the changer? If so, I would > guess the current parse.c would meet this criterion, and I think Jay would > own it, since I think he has done the majority of the changes. > > IANAL. > >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>> From: lemming at henning-thielemann.de >>> To: jay.krell at cornell.edu >>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>> Subject: Re: [M3devel] parse.c licensing question, dual? >>> >>> >>> On Mon, 27 Jun 2016, Jay K wrote: >>> >>>> Can the owner relicense parse.c, >>> >>> Sure, the owner can always relicense. >>> >>>> or it is stuck with GPL because it links to gcc? >>> >>> He can choose any license that is compatible with GPL. >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Mon Jun 27 22:32:29 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:32:29 +0000 Subject: [M3devel] getting all registers in gc? In-Reply-To: References: , , <0D39D7E6-0C71-4C19-8A25-2399AF510C34@purdue.edu>, Message-ID: ?> I think getcontext will go far toward providing peace of mind on many platform That is surprisingly a syscall where I've now looked. Surprising and slightly bad. In the name of thread suspension, it might be affordable. Still might copy that underlying code. We need to solve this even with cooperative suspend btw -- which I really want... ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: hosking at purdue.edu > CC: m3devel at elegosoft.com > Subject: RE: [M3devel] getting all registers in gc? > Date: Mon, 27 Jun 2016 20:04:27 +0000 > > How do we ensure that? > > What about other backends, e.g. C or LLVM? > > The Boehm collector code implies the situation is under control, if we are like it, at least. > > I think getcontext will go far toward providing peace of mind on many platforms (and RtlCaptureContext > on NT). > > I might still be convinced that setjmp suffices. > In that, compiler will spill volatiles ahead of it anyway. > I wonder if that suffices, but I haven't convinced myself. > > - Jay > > ---------------------------------------- >> From: hosking at purdue.edu >> To: jay.krell at cornell.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] getting all registers in gc? >> Date: Mon, 27 Jun 2016 09:50:06 +0000 >> >> A thought: can we simply ensure that the gcc-based backend never spills pointers to FP regs? >> >>> On 27 Jun 2016, at 7:14 PM, Jay K wrote: >>> >>> ok, the Boehm code: >>> >>> For the current live thread, merely: >>> >>> /* Push enough of the current stack eagerly to */ >>> /* ensure that callee-save registers saved in */ >>> /* GC frames are scanned. */ >>> /* In the non-threads case, schedule entire */ >>> /* stack for scanning. */ >>> /* The second argument is a pointer to the */ >>> /* (possibly null) thread context, for */ >>> /* (currently hypothetical) more precise */ >>> /* stack scanning. */ >>> /* >>> * In the absence of threads, push the stack contents. >>> * In the presence of threads, push enough of the current stack >>> * to ensure that callee-save registers saved in collector frames have been >>> * seen. >>> * FIXME: Merge with per-thread stuff. >>> */ >>> /*ARGSUSED*/ >>> STATIC void GC_push_current_stack(ptr_t cold_gc_frame, void * context) >>> { >>> # if defined(THREADS) >>> if (0 == cold_gc_frame) return; >>> # ifdef STACK_GROWS_DOWN >>> GC_push_all_eager(GC_approx_sp(), cold_gc_frame); >>> /* For IA64, the register stack backing store is handled */ >>> /* in the thread-specific code. */ >>> # else >>> GC_push_all_eager(cold_gc_frame, GC_approx_sp()); >>> # endif >>> # else >>> ... >>> # endif /* !THREADS */ >>> >>> >>> GC_INNER ptr_t GC_approx_sp(void) >>> { >>> volatile word sp; >>> sp = (word)&sp; >>> /* Also force stack to grow if necessary. Otherwise the */ >>> /* later accesses might cause the kernel to think we're */ >>> /* doing something wrong. */ >>> return((ptr_t)sp); >>> /* GNU C: alternatively, we may return the value of */ >>> /*__builtin_frame_address(0). */ >>> } >>> >>> >>> Notice that it doesn't even do what it says -- no attempt >>> to save registers to stack. >>> >>> >>> but for suspended threads it is more convincing: >>> >>> >>> /* Ensure that either registers are pushed, or callee-save registers */ >>> /* are somewhere on the stack, and then call fn(arg, ctxt). */ >>> /* ctxt is either a pointer to a ucontext_t we generated, or NULL. */ >>> GC_INNER void GC_with_callee_saves_pushed(void (*fn)(ptr_t, void *), >>> ptr_t arg) >>> { >>> volatile int dummy; >>> void * context = 0; >>> >>> >>> .. >>> .... >>> >>> >>> a mix of methods: >>> - sometimes processor specific assembly >>> - sometimes getcontext, and then workaround a bug on Linux/amd64 >>> - and then _setjmp on Unix >>> - setjmp on Windows >>> >>> >>> getcontext to me seems more promising than setjmp, >>> and you can use both >>> >>> for Win32, I suggest RtlCaptureContext (for live thread too) >>> >>> >>> We maybe should copy getcontext from various BSDs? >>> i.e. Win32 RtlCaptureContext, else carry the assembly with us (no need >>> to worry about the glibc getcontext bug). >>> >>> or maybe just getcontext. The gradually expanding register set on x86 makes me nervous >>> that this isn't a maintenance problem, but I'm guessing you never get pointers >>> spilled to the ymm/zmm registers. >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Date: Mon, 27 Jun 2016 08:37:12 +0000 >>>> Subject: [M3devel] getting all registers in gc? >>>> >>>> I just noticed this in the Boehm GC documentation: >>>> >>>> - Changed the alpha port to use the generic register scanning code instead >>>> of alpha_mach_dep.s. Alpha_mach_dep.s doesn't look for pointers in fp >>>> registers, but gcc sometimes spills pointers there. (Thanks to Manuel >>>> Serrano for helping me debug this by email.) Changed the IA64 code to >>>> do something similar for similar reasons. >>>> >>>> >>>> This would seem like a hazard for us too. >>>> >>>> And not convincingly Alpha/IA64-specific. >>>> >>>> We basically assume setjmp stores a context, or at least all live pointers. >>>> >>>> >>>> In hindsight I see two problems: >>>> - one alluded to -- jmpbuf might not have floating point registers, >>>> and floating point registers might have pointers. >>>> >>>> >>>> - Same thing but more general: jmpbuf might not even have all integer >>>> registers? >>>> >>>> >>>> >>>> So that leaves the question "What is generic register scanning code"? >>>> >>>> I don't know yet but..thinking... >>>> >>>> Maybe we should instead use Posix-deprecated getcontext and Win32 RtlCaptureContext? >>>> >>>> I'm actually looking for how Boehm gc gets the "second half" of the IA64 stack, >>>> as I think that is a lingering thing we need to handle to finish our portability. >>>> >>>> Ignoring IA64 for now, maybe here: >>>> >>>> void >>>> __cdecl >>>> ThreadPThread__sigsuspend(void) >>>> { >>>> struct { >>>> sigjmp_buf jb; >>>> } s; >>>> >>>> ZERO_MEMORY(s); >>>> >>>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>>> #ifdef M3_REGISTER_WINDOWS >>>> siglongjmp(s.jb, 1); /* flush register windows */ >>>> else >>>> #endif >>>> sigsuspend(&mask); >>>> } >>>> >>>> >>>> and here: >>>> >>>> void >>>> __cdecl >>>> ThreadPThread__ProcessLive(char *bottom, void (*p)(void *start, void *limit)) >>>> { >>>> struct { >>>> sigjmp_buf jb; >>>> } s; >>>> >>>> ZERO_MEMORY(s); >>>> >>>> if (sigsetjmp(s.jb, 0) == 0) /* save registers to stack */ >>>> >>>> >>>> we should use getcontext/RtlCaptureContext/GetThreadContext? >>>> >>>> >>>> I'll look more at the Boehm code. >>>> >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > From jay.krell at cornell.edu Mon Jun 27 22:42:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 20:42:51 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , , , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , , , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, ,,, , , , , , <57718BFF.1070501@lcwb.coop>, Message-ID: Tony changed the license from DEC to FSF here (possibly someone else did in another branch) https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 There is this growth: jair:~ jay$ wc -l parse.c.current? ? ? 6516 parse.c.current jair:~ jay$ wc -l parse.c.old? ? ? 4190 parse.c.old though derivation is still hard to argue with. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Date: Mon, 27 Jun 2016 20:30:57 +0000 > Subject: Re: [M3devel] parse.c licensing question, dual? > > Some number should be reasonable yes, but I think you exaggerate how much I've done. :) > I'll look later. > > - Jay > > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 15:26:39 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> >> >> On 06/27/2016 03:19 PM, Jay K wrote: >>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>> >>> But parse.c might be ownerless and stuck? >> >> Isn't there some law that if more than 25% of a work is changed, it ceases >> to be a derived work but becomes a new work by the changer? If so, I would >> guess the current parse.c would meet this criterion, and I think Jay would >> own it, since I think he has done the majority of the changes. >> >> IANAL. >> >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>> From: lemming at henning-thielemann.de >>>> To: jay.krell at cornell.edu >>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>> >>>> >>>> On Mon, 27 Jun 2016, Jay K wrote: >>>> >>>>> Can the owner relicense parse.c, >>>> >>>> Sure, the owner can always relicense. >>>> >>>>> or it is stuck with GPL because it links to gcc? >>>> >>>> He can choose any license that is compatible with GPL. >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Mon Jun 27 23:44:13 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 17:44:13 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <57718A52.3030609@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: <20160627214413.GA25148@topoi.pooq.com> On Mon, Jun 27, 2016 at 03:19:30PM -0500, Rodney M. Bates wrote: > > My big lament about llvm regards producing Dwarf debug info. Having > debug info in this vastly superior format is one of my main reasons > for wanting an llvm back end. From the llvm documentation, it sounded > like it would do this, including altering debug info as needed to > match any optimizations done. Then I hit a brick wall, finding out > llvm only handles a severe subset of Dwarf, apparently what they felt > was needed for C & C++. More below. llvm is an intermediate code for C and C++. Its address arithmetic is C/C++ arithmetic. Any usability for other languages is a happy accident, whatever they may say. -- hendrik From hendrik at topoi.pooq.com Mon Jun 27 23:48:49 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 27 Jun 2016 17:48:49 -0400 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: <20160627085333.916aae7db6aa078d5300c739@elegosoft.com> <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu> <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu> Message-ID: <20160627214849.GB25148@topoi.pooq.com> On Mon, Jun 27, 2016 at 10:17:38PM +0200, Henning Thielemann wrote: > > On Mon, 27 Jun 2016, Jay K wrote: > > >Can the owner relicense parse.c, > > Sure, the owner can always relicense. > > >or it is stuck with GPL because it links to gcc? > > He can choose any license that is compatible with GPL. And any other licence at al. But it would ony be directly useful if one of the licences is conpatible with GPL. -- hendrik k From jay.krell at cornell.edu Tue Jun 28 01:44:19 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 27 Jun 2016 23:44:19 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160627214413.GA25148@topoi.pooq.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop>, <20160627214413.GA25148@topoi.pooq.com> Message-ID: gcc has these other frontends -- ada and java at least.If they are much used -- and I don't know if they are, but if they are, LLVM should expand its goals to replace them.Just my opinion.But either way, other people are using LLVM, like for OCaml and Rust I'm pretty sure.And GPU stuff.So it isn't just C/C++, but may be a happy accident, indeed. How much of gcc/dwarf is that way?I understand gcc does actually cater to Pascal/Modula-3?Ada some -- the multiple forms of divide, the nested functions (eventhough we have to patch gcc for nested functions, it does have some nested function support already, includinga certain optimization you won't see any any narrow C/C++ compiler -- ok, granted, they have nested functions in their C frontend.., but still, the divide stuff..) - Jay > Date: Mon, 27 Jun 2016 17:44:13 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Mon, Jun 27, 2016 at 03:19:30PM -0500, Rodney M. Bates wrote: > > > > My big lament about llvm regards producing Dwarf debug info. Having > > debug info in this vastly superior format is one of my main reasons > > for wanting an llvm back end. From the llvm documentation, it sounded > > like it would do this, including altering debug info as needed to > > match any optimizations done. Then I hit a brick wall, finding out > > llvm only handles a severe subset of Dwarf, apparently what they felt > > was needed for C & C++. More below. > > llvm is an intermediate code for C and C++. Its address arithmetic is > C/C++ arithmetic. Any usability for other languages is a happy > accident, whatever they may say. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Jun 28 02:29:47 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 19:29:47 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> Message-ID: <5771C4FB.8040009@lcwb.coop> On 06/27/2016 03:28 PM, Jay K wrote: > Regarding debug info. > > > I have thought about this. > > > Various optimizing compilers make some effort to produce > some debug info. > > > And as much as they don't optimize, they should do that. > > > However I believe in general, optimization and describability > with any debug format, is not a solvable problem. > > > There are many points in a program where things aren't particularly > transformed/lost by the optimizer. > > > Things tend to be clearly defined at the start of a function. > Variable locations can be described over ranges of code as either > being in a particular register, or register + offset (frame), or nowhere. > > > But line numbers, as critical as they are, I think are the most > untenable aspect. Compilers can move code around arbitrarily, including removing it. > Yes, this is a bundle of hard problems. llvm's official goal is that, in the face of optimizations, it should still be possible for a debugger to observe all program state of the original program, but not necessarily alter any state. Without optimizations, it should also be possible to alter any state as well. How well they are achieving this, I don't know, but I would guess it comes closer than gcc, especially an old gcc. > > The C backend also has good debugging as a goal. > We should be able to have structs with fields, instead of just byte arrays. > > And all the parameters/locals should have good names. > This I messed up slightly and will try to fix. Every identifier > gets an appended number for uniqueness, which is sometimes but rarely needed. > > > Does the current LLVM backend produce any debug info or none? It has lots of code for producing debug info, probably as much code as for producing compilable output, maybe slightly more. I think it is pretty complete. But it has not had a lot of testing, and there are known cases where it has crashes. The tests expose several of these. I was working on this when I hit the problem I have complained about. If you don't ask for debug output, these don't happen. If I could decide what to do when llvm's debug info is inadequate, I would work on fixing them--probably not too big a job. > > I probably avoid any approach that requires writing bindings. > Unless it is to our own code, like I did for Posix. > Otherwise keeping things up to date is tedious and error prone and errors are fatal. > To our own code is just slightly better. > > Maybe I'll write bitcode. Or go through the persisted cm3cg IR that m3cc uses, but it sounds > like you are super far along and we should build on that. > I have other things to do first though and I can't promise that I won't reinvent eventually. > At this point, I think this might be nice to have. The existing m3llvm would be a good starting point. I think mostly, it would just need the calls on llvm APIs replaced by bitcode-emitting calls, plus the low-level things for them to call. > - Jay > > > ---------------------------------------- >> Date: Mon, 27 Jun 2016 15:19:30 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 llvm backend? >> >> >> >> On 06/27/2016 01:31 AM, Jay K wrote: >>> redirecting... >>> >>>> Olaf >>> >>>> LLVM didn't seem to satisfy M3's needs, however, if I understood Rodney's laments correctl >>> >> >> My big lament about llvm regards producing Dwarf debug info. Having >> debug info in this vastly superior format is one of my main reasons >> for wanting an llvm back end. From the llvm documentation, it sounded >> like it would do this, including altering debug info as needed to >> match any optimizations done. Then I hit a brick wall, finding out >> llvm only handles a severe subset of Dwarf, apparently what they felt >> was needed for C & C++. More below. >> >>> >>> I would like to take this up, maybe soon. >>> >>> >>> I do have a bit of an agenda. >>> Maybe my priorities are mixed up. >>> >>> >>> 1 Provide a very portable system. >>> 2 Provide an easy to install and use system. >>> 3 Switch from gcc backend to LLVM backend, at least optionally (i.e. at least >>> for their supported backends). >>> 4 Maybe write our own backends. >>> >>> >>> Where is the LLVM support at? >>> Mostly working? Barely working? >> >> I could swear I remember getting the llvm back end to recompile >> everything in the m3front group twice and converge to identical-sized >> compiled code, and that I said so in either a commit message or >> an m3devel list post. But I couldn't find any such statement. >> >> This is on AMD64_LINUX, and if you are careful not to ask m3llvm >> for debug info at all. Otherwise, there are crashes in m3llvm >> and/or llc, trying to provide Dwarf. This is what I was working >> on when I made the disappointing discoveries. If my memory is >> correct here, this means the llvm back end is close to usable >> for building. >> >> Also, again from memory, I think the llvm back end has no failures >> of test cases that do not also fail using m3cg. >> >> There are still other questions, though. So far, there are no >> changes required to the stock llvm version being used (3.6.1) >> But it still requires a build of that on the machine where the >> M3 compilation is done, and it is big and slow to build. Lots >> of llvm libraries have to be linked in to m3llvm, and separate >> executable llc needs to be available. >> >> The llvm folk are constantly making major API changes, with no >> explanation other than diffing successive versions of header >> files, so there is no practical chance of using whatever llvm >> version may be already installed on a particular host machine. >> >> In light of that, I suppose we could fork and modify a specific >> version of llvm and put its source code in our own repository, >> similar to m3cg. But maintenance work on llvm is, to my >> standards, a real nightmare. Just about every single identifier >> and operator you see involves a needle-in-haystack search to >> locate its declaration, which could be just about anywhere, >> in order to know what it is. >> >> And no, the names and operator spellings are not close to adequate >> to clue you in. They have gone to every length possible to use >> every clever new C++ "feature" that comes out in the latest >> C++- standard, which always just increases the complexity >> of the search to a declaration. So I don't fancy doing any of >> this. (BTW, =17 in recent discussions.) >> >> As for the debug info problem, there has been talk in the llvm >> list of a rework that allows a front end to directly produce >> true Dwarf (presumably not a subset) for type info only, which >> would not be changed by optimizations and so llvm would not >> need to pass it through in its own, different, representation. >> I don't know how far this has gone. >> >> If/when it happens, utilizing it would entail constructing new >> bindings to altered llvm APIs. This in itself is an extremely >> tedious and error-prone task that I hoped, after doing it for >> llvm 3.6.1, never to repeat. Moreover, at the C/M3 interface, >> there is no type checking possible, so nitty little errors will >> only show up as hard-to-diagnose runtime errors. That is made >> even worse by the lack of a debugger that handles both languages. >> >> Or, we could use Dragisha's rtinfo library to get type info for >> a debugger from the .M3WEB files, and Dwarf for the rest. Off >> hand, this sounds like the best approach to me. Stick with >> llvm 3.6.1 as long a possible, and avoid more pain. >> >>> >>> >>> I know LLVM is big and changing, and maybe they don't value >>> compatibility of bitcode. >> >> Actually, keeping their bitcode stable across llvm releases is >> one place they do talk about compatibility. But m3llvm uses calls >> to llvm APIs to construct llvm IR as in-memory data, then another >> call to get llvm to convert it to bitcode. So bitcode's stability >> is irrelevant to us. I once thought about producing llvm bitcode >> directly, but that seems like a pretty big job. It would, however, >> obviate creating most of those wretched bindings. >> >>> >>> >>> But look at what we have with the gcc backend. >>> Even if we didn't have to patch it at all, I expect >>> we'd still have to keep and build a local copy. >>> >>> Perhaps we should just do that? >>> >>> With LLVM, with its different licensing, perhaps we could >>> get our "frontend" merged upstream, but this would >>> then give us a compatibility burden in the persisted m3cg. >>> Is that ok? >>> It is hypothetical at this point. >>> >>> >>> I know everyone here doesn't really like C/C++ (except me). >>> And, more significantly, I know the system written in itself >>> is a great test case, but I wonder if we shouldn't write a new >>> "real" frontend in C or C++, and see if we can't merge that >>> upstream with gcc and/or clang. >>> >>> >>> It also worth mentioning that I believe gcc's Ada front end >>> is written in Ada -- you don't actually have to write >>> the frontend in C/C++ to merge upstream. >> >> Yes, I once worked on this front end. As I understood, SRC M3 >> was not merged into gcc only because RMS was annoyed that SRC made >> it a separate executable, so its license was not poisoned by GPL. >> >>> >>> But there might remain licensing concern. >>> >>> >>> >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 28 02:31:53 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 27 Jun 2016 19:31:53 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> Message-ID: <5771C579.2000607@lcwb.coop> On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > >> And no, the names and operator spellings are not close to adequate >> to clue you in. They have gone to every length possible to use >> every clever new C++ "feature" that comes out in the latest >> C++- standard, which always just increases the complexity >> of the search to a declaration. So I don't fancy doing any of >> this. (BTW, =17 in recent discussions.) > > A teacher of mine called this behavior "version junkie". > Yes, yes. > >> Actually, keeping their bitcode stable across llvm releases is >> one place they do talk about compatibility. But m3llvm uses calls >> to llvm APIs to construct llvm IR as in-memory data, then another >> call to get llvm to convert it to bitcode. So bitcode's stability >> is irrelevant to us. I once thought about producing llvm bitcode >> directly, but that seems like a pretty big job. It would, however, >> obviate creating most of those wretched bindings. > > An alternative would be to create .ll text files. But its format changes, too. Yes. But, according to the list talk, they don't have the intention to make it as stable as the bitcode format. > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 28 03:03:47 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 01:03:47 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5771C579.2000607@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: So..rambling a bit..but I have discussed some with people and considered it. "the interface to the compiler backend" and my half serious answer: All of the compilers have a well documented very stable interface to their backends, and it is in fact the same, roughly, interface to all the backends: source code. It is true it isn't the most efficient representation. Maybe the least efficient. It might not expose all the internals at least portable (e.g. tail call). But it works, is heavily used, is well known, documented, has high compatibility requirements, somewhat readable with standard tools, etc. I would advocate that C and C++ be evolved a little bit for these purposes. In particular, C needs exception handling. C and C++ need a tail call notion. _alloca should maybe be standardized. I should be able to generate image-relative pointers/offsets. (trivial in Microsoft assembly with "imagerel"), to help me layout position indepenent data structures. C-- is kind of this, and there was some C-resembling assembly,' but I think really C should be the starting point, as it is pretty close. Probably need some extensions like non-int bitfields, and rotate, and shift right with both sign copying and zero fill. > A teacher of mine called this behavior "version junkie". There are at least two big reasons for this. - The language really is improving. Programs written in the newer language are easier to read and often easier to optimize and sometimes easier to implement a compiler for. - Dogfooding -- using the language helps inform the language implementer where they have done things right, or wrong, and what to improve. I believe in fact that under-dogfooding of C++ led to some early omissions. The need for auto for example. Granted, Stroustrup put it in in the 1980s and had to remove it. But with more dogfooding by more implementers, it would have been added earlier. Similarly "template typedefs". Are obviously needed once you use templates for about a day. Modula-3 has similar failings imho. For example, the fact that with/var imply a nesting that "needs" indentation and needs "end". This is something that C++ and much later even C fixed. Perhaps though, perhaps the Modula-3 designers were balancing the specification and implementation against user convenience, as the current design is obviously simpler to specify and implement. But tedious to use. So the binary form of LLVM bit code is more stable than the text form? - Jay > Date: Mon, 27 Jun 2016 19:31:53 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > >> And no, the names and operator spellings are not close to adequate > >> to clue you in. They have gone to every length possible to use > >> every clever new C++ "feature" that comes out in the latest > >> C++- standard, which always just increases the complexity > >> of the search to a declaration. So I don't fancy doing any of > >> this. (BTW, =17 in recent discussions.) > > > > A teacher of mine called this behavior "version junkie". > > > > Yes, yes. > > > > >> Actually, keeping their bitcode stable across llvm releases is > >> one place they do talk about compatibility. But m3llvm uses calls > >> to llvm APIs to construct llvm IR as in-memory data, then another > >> call to get llvm to convert it to bitcode. So bitcode's stability > >> is irrelevant to us. I once thought about producing llvm bitcode > >> directly, but that seems like a pretty big job. It would, however, > >> obviate creating most of those wretched bindings. > > > > An alternative would be to create .ll text files. But its format changes, too. > > Yes. But, according to the list talk, they don't have the intention to > make it as stable as the bitcode format. > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Jun 28 07:00:49 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 05:00:49 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , , , , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, , , , <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , , , , , , , , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , , , , , , , , , , <57718BFF.1070501@lcwb.coop>, , Message-ID: Links paste funny. I'll try again: 1.17 https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.17;content-type=text%2Fplain vs. v1.18 https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.18;content-type=text%2Fplain Tony, you changed from..er...hold on.. that was not really a change. here vs. 1.12: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.12;content-type=text%2Fplain /* Copyright (C) 1993, Digital Equipment Corporation ? ? ? ? ? ? ? ? ? ? ? ? */ /* All rights reserved. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ /* See the file COPYRIGHT for a full description. ? ? ? ? ? ? ? ? ? ? ? ? ? ?*/ v 1.13: ? ? Copyright (C) 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. ? ? This program is free software; you can redistribute it and/or modify it ? ? under the terms of the GNU General Public License as published by the Free ? ? Software Foundation; either version 2, or (at your option) any later ? ? version. ... Merge with gcc-core-3.4.5. ?This is now the default backend. in either case, there is a notion of "GPL compatible". For example the BSD licenses are GPL compatible. They can be linked with GPL, or such. And then..there is the question of derivation and ownership. There is only a few thousand lines here. The slightly interesting part is the reading of the IR. The building up of gcc tree's is gcc specific. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: [M3devel] parse.c licensing question, dual? > Date: Mon, 27 Jun 2016 20:42:51 +0000 > > Tony changed the license from DEC to FSF here (possibly someone else did in another branch) > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 > > > There is this growth: > > jair:~ jay$ wc -l parse.c.current > 6516 parse.c.current > > jair:~ jay$ wc -l parse.c.old > 4190 parse.c.old > > though derivation is still hard to argue with. > > - Jay > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Date: Mon, 27 Jun 2016 20:30:57 +0000 >> Subject: Re: [M3devel] parse.c licensing question, dual? >> >> Some number should be reasonable yes, but I think you exaggerate how much I've done. :) >> I'll look later. >> >> - Jay >> >> >> >> ---------------------------------------- >>> Date: Mon, 27 Jun 2016 15:26:39 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] parse.c licensing question, dual? >>> >>> >>> >>> On 06/27/2016 03:19 PM, Jay K wrote: >>>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>>> >>>> But parse.c might be ownerless and stuck? >>> >>> Isn't there some law that if more than 25% of a work is changed, it ceases >>> to be a derived work but becomes a new work by the changer? If so, I would >>> guess the current parse.c would meet this criterion, and I think Jay would >>> own it, since I think he has done the majority of the changes. >>> >>> IANAL. >>> >>>> >>>> - Jay >>>> >>>> >>>> ---------------------------------------- >>>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>>> From: lemming at henning-thielemann.de >>>>> To: jay.krell at cornell.edu >>>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>> >>>>> >>>>> On Mon, 27 Jun 2016, Jay K wrote: >>>>> >>>>>> Can the owner relicense parse.c, >>>>> >>>>> Sure, the owner can always relicense. >>>>> >>>>>> or it is stuck with GPL because it links to gcc? >>>>> >>>>> He can choose any license that is compatible with GPL. >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From jay.krell at cornell.edu Tue Jun 28 07:37:17 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 05:37:17 +0000 Subject: [M3devel] parse.c licensing question, dual? In-Reply-To: References: , <20160627085333.916aae7db6aa078d5300c739@elegosoft.com>, <653B1703-DC7E-42BB-8404-423800498EAA@purdue.edu>, , <8052E9AE-2CDA-467D-B58D-125F59365330@purdue.edu>, , , <57718BFF.1070501@lcwb.coop>, , , , Message-ID: I can understand RMS's reaction, since this subverts the spirit of the license, by introducing a process boundary to break the linkage, where normally and conceptually there is a linkage, but I"m wondering about the letter not the spirit. :) > As I understand it, because parse.c is an integrated front-end to gcc it must carry the GPL license This isn't clear to me. There is some notion of "GPL compatible", which includes BSD licenses. ?> Since it is based on other gcc code it becomes GPL Also slightly unclear -- "based on" could be 99% resembling or 1% resembling. In either case...on my big list, maybe, is write a new front end in C or C++ and integrate with gcc or LLVM. Or write out C (which is, still, an integrate path for gcc/LLVM). I can translate *most* of Modula-3 to C in my head -- it almost trivially isomorphic. There are a few parts of the ABI that aren't obvious to me -- open arrays and exceptions. I understand part of them -- but e.g. the "dynamic construction" of exceptions, seems to almost resemble C++, but that doesn't really compute for me in the context of Modula-3. I also suspect it matters less now. There are larger enemies now. ?- Jay ---------------------------------------- > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: [M3devel] parse.c licensing question, dual? > Date: Tue, 28 Jun 2016 05:14:53 +0000 > > As I understand it, because parse.c is an integrated front-end to gcc it must carry the GPL license. Since it is based on other gcc code it becomes GPL-contaminated. I recall that Stallman hit the roof when he found out that M3 was using a shim front-end to gcc the allowed M3 to avoid GPL. > >> On 28 Jun 2016, at 3:00 PM, Jay K wrote: >> >> Links paste funny. >> >> I'll try again: >> >> 1.17 >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.17;content-type=text%2Fplain >> vs. >> >> v1.18 >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.18;content-type=text%2Fplain >> >> >> Tony, you changed from..er...hold on.. >> that was not really a change. >> >> >> here vs. 1.12: >> >> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c?rev=1.12;content-type=text%2Fplain >> >> /* Copyright (C) 1993, Digital Equipment Corporation */ >> /* All rights reserved. */ >> /* See the file COPYRIGHT for a full description. */ >> >> >> v 1.13: >> >> >> Copyright (C) 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc. >> >> This program is free software; you can redistribute it and/or modify it >> under the terms of the GNU General Public License as published by the Free >> Software Foundation; either version 2, or (at your option) any later >> version. >> ... >> >> Merge with gcc-core-3.4.5. This is now the default backend. >> >> >> in either case, there is a notion of "GPL compatible". >> >> For example the BSD licenses are GPL compatible. >> They can be linked with GPL, or such. >> >> And then..there is the question of derivation and ownership. >> >> There is only a few thousand lines here. >> >> The slightly interesting part is the reading of the IR. >> The building up of gcc tree's is gcc specific. >> >> - Jay >> >> >> >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: [M3devel] parse.c licensing question, dual? >>> Date: Mon, 27 Jun 2016 20:42:51 +0000 >>> >>> Tony changed the license from DEC to FSF here (possibly someone else did in another branch) >>> >>> https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3cc/gcc/gcc/m3cg/parse.c.diff?r1=1.12;r2=1.13 >>> >>> >>> There is this growth: >>> >>> jair:~ jay$ wc -l parse.c.current >>> 6516 parse.c.current >>> >>> jair:~ jay$ wc -l parse.c.old >>> 4190 parse.c.old >>> >>> though derivation is still hard to argue with. >>> >>> - Jay >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>>> Date: Mon, 27 Jun 2016 20:30:57 +0000 >>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>> >>>> Some number should be reasonable yes, but I think you exaggerate how much I've done. :) >>>> I'll look later. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> Date: Mon, 27 Jun 2016 15:26:39 -0500 >>>>> From: rodney_bates at lcwb.coop >>>>> To: m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>> >>>>> >>>>> >>>>> On 06/27/2016 03:19 PM, Jay K wrote: >>>>>> Thank you. I was hoping that. So parse.c isn't stuck if it had an answer, and m3-def isn't stuck because I wrote it (possibly derived from DEC -- it is closely related to the rest of m3cg). >>>>>> >>>>>> But parse.c might be ownerless and stuck? >>>>> >>>>> Isn't there some law that if more than 25% of a work is changed, it ceases >>>>> to be a derived work but becomes a new work by the changer? If so, I would >>>>> guess the current parse.c would meet this criterion, and I think Jay would >>>>> own it, since I think he has done the majority of the changes. >>>>> >>>>> IANAL. >>>>> >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> ---------------------------------------- >>>>>>> Date: Mon, 27 Jun 2016 22:17:38 +0200 >>>>>>> From: lemming at henning-thielemann.de >>>>>>> To: jay.krell at cornell.edu >>>>>>> CC: hosking at purdue.edu; m3devel at elegosoft.com >>>>>>> Subject: Re: [M3devel] parse.c licensing question, dual? >>>>>>> >>>>>>> >>>>>>> On Mon, 27 Jun 2016, Jay K wrote: >>>>>>> >>>>>>>> Can the owner relicense parse.c, >>>>>>> >>>>>>> Sure, the owner can always relicense. >>>>>>> >>>>>>>> or it is stuck with GPL because it links to gcc? >>>>>>> >>>>>>> He can choose any license that is compatible with GPL. >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>>>> >>>>> >>>>> -- >>>>> Rodney Bates >>>>> rodney.m.bates at acm.org >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From rodney_bates at lcwb.coop Tue Jun 28 18:28:01 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 28 Jun 2016 11:28:01 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: <5772A591.3080305@lcwb.coop> On 06/27/2016 08:03 PM, Jay K wrote: > So..rambling a bit..but I have discussed some with people > and considered it. > > "the interface to the compiler backend" > > > and my half serious answer: > > > All of the compilers have a well documented very stable > interface to their backends, and it is in fact the same, > roughly, interface to all the backends: source code. > > > It is true it isn't the most efficient representation. > Maybe the least efficient. > > > It might not expose all the internals at least portable (e.g. tail call). > But it works, is heavily used, is well known, documented, > has high compatibility requirements, somewhat readable with > standard tools, etc. > > > I would advocate that C and C++ be evolved a little bit > for these purposes. In particular, C needs exception handling. > C and C++ need a tail call notion. > _alloca should maybe be standardized. > I should be able to generate image-relative pointers/offsets. > (trivial in Microsoft assembly with "imagerel"), to help > me layout position indepenent data structures. > > > C-- is kind of this, and there was some C-resembling assembly,' > but I think really C should be the starting point, as it is pretty close. > > > Probably need some extensions like non-int bitfields, and > rotate, and shift right with both sign copying and zero fill. > > > > A teacher of mine called this behavior "version junkie". > > > There are at least two big reasons for this. > > - The language really is improving. Programs > written in the newer language are easier to read > and often easier to optimize and sometimes easier > to implement a compiler for. > > - Dogfooding -- using the language helps inform the > language implementer where they have done things right, > or wrong, and what to improve. > > > I believe in fact that under-dogfooding of C++ led > to some early omissions. The need for auto for example. > Granted, Stroustrup put it in in the 1980s and had to remove it. > But with more dogfooding by more implementers, it would > have been added earlier. Similarly "template typedefs". > Are obviously needed once you use templates for about a day. > > > Modula-3 has similar failings imho. > For example, the fact that with/var imply > a nesting that "needs" indentation and needs "end". > This is something that C++ and much later even C fixed. > > > Perhaps though, perhaps the Modula-3 designers were > balancing the specification and implementation against > user convenience, as the current design is obviously > simpler to specify and implement. But tedious to use. > > So the binary form of LLVM bit code is more stable than the text form? > That's what they are saying. A terminology point: llvm would consider, I think, that "bit code" refers only to the binary form, and the text form would be called something like "llvm assembly code". Files of binary form are conventionally named .bc and assembly code .ll. In the M3 llvm back end binary files are .ib and .mb, to distinguish interfaces and modules. I am also using .ill and .mll for assembly files, but I'm not sure these suffixes are coded in anywhere. > > - Jay > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > >> And no, the names and operator spellings are not close to adequate > > >> to clue you in. They have gone to every length possible to use > > >> every clever new C++ "feature" that comes out in the latest > > >> C++- standard, which always just increases the complexity > > >> of the search to a declaration. So I don't fancy doing any of > > >> this. (BTW, =17 in recent discussions.) > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > Yes, yes. > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > >> one place they do talk about compatibility. But m3llvm uses calls > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > >> is irrelevant to us. I once thought about producing llvm bitcode > > >> directly, but that seems like a pretty big job. It would, however, > > >> obviate creating most of those wretched bindings. > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > Yes. But, according to the list talk, they don't have the intention to > > make it as stable as the bitcode format. > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Jun 28 18:40:06 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 28 Jun 2016 11:40:06 -0500 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, , <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, , <5771C579.2000607@lcwb.coop> Message-ID: <5772A866.7020601@lcwb.coop> On 06/27/2016 08:03 PM, Jay K wrote: > So..rambling a bit..but I have discussed some with people > and considered it. > > "the interface to the compiler backend" > > > and my half serious answer: > > > All of the compilers have a well documented very stable > interface to their backends, and it is in fact the same, > roughly, interface to all the backends: source code. > > > It is true it isn't the most efficient representation. > Maybe the least efficient. > > > It might not expose all the internals at least portable (e.g. tail call). > But it works, is heavily used, is well known, documented, > has high compatibility requirements, somewhat readable with > standard tools, etc. > > > I would advocate that C and C++ be evolved a little bit > for these purposes. In particular, C needs exception handling. > C and C++ need a tail call notion. > _alloca should maybe be standardized. > I should be able to generate image-relative pointers/offsets. > (trivial in Microsoft assembly with "imagerel"), to help > me layout position indepenent data structures. > > > C-- is kind of this, and there was some C-resembling assembly,' > but I think really C should be the starting point, as it is pretty close. > > > Probably need some extensions like non-int bitfields, and > rotate, and shift right with both sign copying and zero fill. > > > > A teacher of mine called this behavior "version junkie". > > > There are at least two big reasons for this. > > - The language really is improving. Programs > written in the newer language are easier to read > and often easier to optimize and sometimes easier > to implement a compiler for. Sometimes so, sometimes not. Sadly, many language "features" reflect an implicit but very misguided belief that fewer keystrokes/characters means increased readability. Or at least that writability is more important than readability. So often, this means actual code is less explicit. But this makes maintenance far worse. E.g., Ada decided use parentheses for both actual parameter lists to function calls and subscript lists to arrays. Along with optional implicit operations like dereferencing, there are somewhere in the teens of possible meanings for the innocent looking "f(x)". I have forgotten the exact number, but once had to do the semantic analysis. That was Ada 83. Maybe more have been added since. For the poor schmuck who gets called at 3:00 AM to fix a bug in half a million lines of code she didn't write, this is a readability disaster. The savings of keystrokes in not making the operations explicit is penny-wise and million-pound foolish. > > - Dogfooding -- using the language helps inform the > language implementer where they have done things right, > or wrong, and what to improve. > > > I believe in fact that under-dogfooding of C++ led > to some early omissions. The need for auto for example. > Granted, Stroustrup put it in in the 1980s and had to remove it. > But with more dogfooding by more implementers, it would > have been added earlier. Similarly "template typedefs". > Are obviously needed once you use templates for about a day. > > > Modula-3 has similar failings imho. > For example, the fact that with/var imply > a nesting that "needs" indentation and needs "end". > This is something that C++ and much later even C fixed. > > > Perhaps though, perhaps the Modula-3 designers were > balancing the specification and implementation against > user convenience, as the current design is obviously > simpler to specify and implement. But tedious to use. > > So the binary form of LLVM bit code is more stable than the text form? > > > - Jay > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > >> And no, the names and operator spellings are not close to adequate > > >> to clue you in. They have gone to every length possible to use > > >> every clever new C++ "feature" that comes out in the latest > > >> C++- standard, which always just increases the complexity > > >> of the search to a declaration. So I don't fancy doing any of > > >> this. (BTW, =17 in recent discussions.) > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > Yes, yes. > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > >> one place they do talk about compatibility. But m3llvm uses calls > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > >> is irrelevant to us. I once thought about producing llvm bitcode > > >> directly, but that seems like a pretty big job. It would, however, > > >> obviate creating most of those wretched bindings. > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > Yes. But, according to the list talk, they don't have the intention to > > make it as stable as the bitcode format. > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Jun 28 20:12:53 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 18:12:53 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5772A866.7020601@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, ,,<12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, ,,, , , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop>, , , , <5771C579.2000607@lcwb.coop> ,<5772A866.7020601@lcwb.coop> Message-ID: > The savings of keystrokes in not making the operations explicit is penny-wise and million-pound foolish. I hear this a lot. There is a flip side. There is such a thing as too verbose, or explicitly saying everything all the time is inefficient. In fact, if we had to say everything, all the time, we'd instantly run out of space. Requiring too much verbosity can be tedious and hide the important details in the noise of obvious repeated stuff. Some balance must be found. Consider: C: a = b vs. asssembly hypothetical: mov ra, rb "=" has become "mov r r" -- a large multiplication of noise that the human reader has to sift through to see the meaning or worse mov r1, rsp[b] mov rsp[a], r1 Is the assembly clearer because it is more explicit? operator= can be overloaded It means assignment, however the author of the types of a and b deemed appropriate it might be on the order of 1-3 instructions, or it might be a function call. Ignoring performance, it doesn't matter. It is the right notation for "assignment". We must pick local vocabularies or "context" and communicate within it. It must be easy for a newcomer to step back and learn the vocabulary -- that is probably the unsolved part. And then I often read code, that reads like: // foo the bar foo(bar) I don't know what a "bar" is, I don't know what it means to "foo" it, or why/when you would want to. The pattern works in Modula-3, C, C++, etc. even assembly ; foo the bar mov rcx, rsp[bar] call foo all equally gibberish to the newcomer. Though to the initiated, assuming foo is at least a few lines of code, this is probably the right level to code at. - Jay > Date: Tue, 28 Jun 2016 11:40:06 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > > > On 06/27/2016 08:03 PM, Jay K wrote: > > So..rambling a bit..but I have discussed some with people > > and considered it. > > > > "the interface to the compiler backend" > > > > > > and my half serious answer: > > > > > > All of the compilers have a well documented very stable > > interface to their backends, and it is in fact the same, > > roughly, interface to all the backends: source code. > > > > > > It is true it isn't the most efficient representation. > > Maybe the least efficient. > > > > > > It might not expose all the internals at least portable (e.g. tail call). > > But it works, is heavily used, is well known, documented, > > has high compatibility requirements, somewhat readable with > > standard tools, etc. > > > > > > I would advocate that C and C++ be evolved a little bit > > for these purposes. In particular, C needs exception handling. > > C and C++ need a tail call notion. > > _alloca should maybe be standardized. > > I should be able to generate image-relative pointers/offsets. > > (trivial in Microsoft assembly with "imagerel"), to help > > me layout position indepenent data structures. > > > > > > C-- is kind of this, and there was some C-resembling assembly,' > > but I think really C should be the starting point, as it is pretty close. > > > > > > Probably need some extensions like non-int bitfields, and > > rotate, and shift right with both sign copying and zero fill. > > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > There are at least two big reasons for this. > > > > - The language really is improving. Programs > > written in the newer language are easier to read > > and often easier to optimize and sometimes easier > > to implement a compiler for. > > Sometimes so, sometimes not. Sadly, many language "features" reflect > an implicit but very misguided belief that fewer keystrokes/characters > means increased readability. Or at least that writability is more > important than readability. So often, this means actual code is less > explicit. But this makes maintenance far worse. > > E.g., Ada decided use parentheses for both actual parameter lists to > function calls and subscript lists to arrays. Along with optional > implicit operations like dereferencing, there are somewhere in the > teens of possible meanings for the innocent looking "f(x)". I have forgotten > the exact number, but once had to do the semantic analysis. That was > Ada 83. Maybe more have been added since. For the poor schmuck who > gets called at 3:00 AM to fix a bug in half a million lines of code > she didn't write, this is a readability disaster. The savings of > keystrokes in not making the operations explicit is penny-wise and > million-pound foolish. > > > > > - Dogfooding -- using the language helps inform the > > language implementer where they have done things right, > > or wrong, and what to improve. > > > > > > I believe in fact that under-dogfooding of C++ led > > to some early omissions. The need for auto for example. > > Granted, Stroustrup put it in in the 1980s and had to remove it. > > But with more dogfooding by more implementers, it would > > have been added earlier. Similarly "template typedefs". > > Are obviously needed once you use templates for about a day. > > > > > > Modula-3 has similar failings imho. > > For example, the fact that with/var imply > > a nesting that "needs" indentation and needs "end". > > This is something that C++ and much later even C fixed. > > > > > > Perhaps though, perhaps the Modula-3 designers were > > balancing the specification and implementation against > > user convenience, as the current design is obviously > > simpler to specify and implement. But tedious to use. > > > > So the binary form of LLVM bit code is more stable than the text form? > > > > > > - Jay > > > > > > > > > > > Date: Mon, 27 Jun 2016 19:31:53 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] cm3 llvm backend? > > > > > > > > > > > > On 06/27/2016 03:29 PM, Henning Thielemann wrote: > > > > > > > > On Mon, 27 Jun 2016, Rodney M. Bates wrote: > > > > > > > >> And no, the names and operator spellings are not close to adequate > > > >> to clue you in. They have gone to every length possible to use > > > >> every clever new C++ "feature" that comes out in the latest > > > >> C++- standard, which always just increases the complexity > > > >> of the search to a declaration. So I don't fancy doing any of > > > >> this. (BTW, =17 in recent discussions.) > > > > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > > Yes, yes. > > > > > > > > > > >> Actually, keeping their bitcode stable across llvm releases is > > > >> one place they do talk about compatibility. But m3llvm uses calls > > > >> to llvm APIs to construct llvm IR as in-memory data, then another > > > >> call to get llvm to convert it to bitcode. So bitcode's stability > > > >> is irrelevant to us. I once thought about producing llvm bitcode > > > >> directly, but that seems like a pretty big job. It would, however, > > > >> obviate creating most of those wretched bindings. > > > > > > > > An alternative would be to create .ll text files. But its format changes, too. > > > > > > Yes. But, according to the list talk, they don't have the intention to > > > make it as stable as the bitcode format. > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Tue Jun 28 22:15:25 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 28 Jun 2016 22:15:25 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <5772A866.7020601@lcwb.coop> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> Message-ID: <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> On Tue, 28 Jun 2016 11:40:06 -0500 "Rodney M. Bates" wrote: > On 06/27/2016 08:03 PM, Jay K wrote: > > > A teacher of mine called this behavior "version junkie". > > > > > > There are at least two big reasons for this. > > > > - The language really is improving. Programs > > written in the newer language are easier to read > > and often easier to optimize and sometimes easier > > to implement a compiler for. > > Sometimes so, sometimes not. Sadly, many language "features" reflect > an implicit but very misguided belief that fewer keystrokes/characters > means increased readability. Or at least that writability is more > important than readability. So often, this means actual code is less > explicit. But this makes maintenance far worse. > > E.g., Ada decided use parentheses for both actual parameter lists to > function calls and subscript lists to arrays. Along with optional > implicit operations like dereferencing, there are somewhere in the > teens of possible meanings for the innocent looking "f(x)". I have forgotten > the exact number, but once had to do the semantic analysis. That was > Ada 83. Maybe more have been added since. For the poor schmuck who > gets called at 3:00 AM to fix a bug in half a million lines of code > she didn't write, this is a readability disaster. The savings of > keystrokes in not making the operations explicit is penny-wise and > million-pound foolish. I couldn't agree more with this. My focus seems to have moved away from programming to reading and analyzing code, too. Often it is almost next to impossible for me to find the exact meaning or definition of an expression or call without knowing all the other code which is not obviously related to the location in question. I would favour longer explicit syntax and clear meaning above shorter expressions any time. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Tue Jun 28 23:34:35 2016 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 Jun 2016 21:34:35 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, <57718A52.3030609@lcwb.coop>, <5771C579.2000607@lcwb.coop>, <5772A866.7020601@lcwb.coop>, <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: I find myself on the other side of this. There are many people on both sides. There are countless examples. Here are two. People don't like operator loading. They don't want me to write string operator+(string, string); but nobody seems to mind: float f = 1.0 + 2.0; int i = 1 + 2; why is + ok on floats and ints, overloaded, but not user defined types such as string? People don' t like type inferencing. auto a = 1.0 + 2.0; auto b = 1 + 2; Modula-3 has this more than C and C++98 already, so maybe people here don't mind. VAR a := 1.0 + 2.0; VAR b := 1 + 2; In either case, nobody seems to mind temporaries without explicit types in function calls or more generally subexpressions. F(1 + 2); F2(1.0 + 2.0); - Jay > Date: Tue, 28 Jun 2016 22:15:25 +0200 > From: wagner at elegosoft.com > To: rodney.m.bates at acm.org > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Tue, 28 Jun 2016 11:40:06 -0500 > "Rodney M. Bates" wrote: > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > There are at least two big reasons for this. > > > > > > - The language really is improving. Programs > > > written in the newer language are easier to read > > > and often easier to optimize and sometimes easier > > > to implement a compiler for. > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > an implicit but very misguided belief that fewer keystrokes/characters > > means increased readability. Or at least that writability is more > > important than readability. So often, this means actual code is less > > explicit. But this makes maintenance far worse. > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > function calls and subscript lists to arrays. Along with optional > > implicit operations like dereferencing, there are somewhere in the > > teens of possible meanings for the innocent looking "f(x)". I have forgotten > > the exact number, but once had to do the semantic analysis. That was > > Ada 83. Maybe more have been added since. For the poor schmuck who > > gets called at 3:00 AM to fix a bug in half a million lines of code > > she didn't write, this is a readability disaster. The savings of > > keystrokes in not making the operations explicit is penny-wise and > > million-pound foolish. > > I couldn't agree more with this. My focus seems to have moved away from > programming to reading and analyzing code, too. Often it is almost next > to impossible for me to find the exact meaning or definition of an > expression or call without knowing all the other code which is not > obviously related to the location in question. > > I would favour longer explicit syntax and clear meaning above > shorter expressions any time. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Jun 29 00:39:17 2016 From: lists at darko.org (Darko Volaric) Date: Wed, 29 Jun 2016 00:39:17 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: On Tue, Jun 28, 2016 at 11:34 PM, Jay K wrote: > I find myself on the other side of this. > There are many people on both sides. > > There are countless examples. > > > Here are two. > > > People don't like operator loading. > > They don't want me to write > > string operator+(string, string); > > > but nobody seems to mind: > > > float f = 1.0 + 2.0; > int i = 1 + 2; > > why is + ok on floats and ints, overloaded, but not user defined types > such as string? > > > The notion that limited fixed operator overloading justifies unlimited user operator overloading is an asinine argument. The former you can easily keep in your head, since the rules are simple and clear. You know given a + b + c + 1.0 that a, b and c are floating point values, or if you didn't have the literal you'd know by knowing the type of one value. But given user overloading, what is the type of a + b + c + 1.0 or some other expression? You have to know what the type of each value is, you need to know the definitions of the overloads for each type pair, you have to work out what the precedence and associativity rules are so you're evaluating the expression correctly and you have to know the type that each overloaded operator returns and work out what the new type pairs are at each stage of evaluation of the expression. > People don' t like type inferencing. > > auto a = 1.0 + 2.0; > auto b = 1 + 2; > > Modula-3 has this more than C and C++98 already, so maybe people here > don't mind. > > VAR a := 1.0 + 2.0; > VAR b := 1 + 2; > > In either case, nobody seems to mind temporaries without explicit types > in function calls or more generally subexpressions. > > F(1 + 2); > F2(1.0 + 2.0); > > Again, this is "type inference" in the most simplistic case and hardly justifies unbounded type inferencing. The static type pf every expression (and subexpression) is known in M3, merely using it introduces no complexity. The static type is easily determined because the rules for determining the type of any expression are small, fixed, simple and strict. If you really want to understand type inferencing and overloading, have a look at the Swift programming language. You'll find yourself writing an innocuous expression and the compiler will tell you that the expression is "ambiguous" or "too complex" even though there doesn't seem any other way to interpret the expression or it involves only one operator/type combo used more than once. What the compiler is really telling you is that it can't infer the type of some subexpression or choose the right method and you must now start applying casts to make it obvious (this doesn't always work and sometimes you have to rewrite it wholesale). Why does this happen? Because Swift allows you to overload functions and operators based on parameter type and return type. You say "people are on both sides of this" but good design isn't a democracy or a popularity contest. If you walk around downtown San Francisco you can find many people who will agree with you. You will also find as many people who claim that the government is beaming thoughts into their heads via satellite. How people feel about language features and the fact that they are untroubled by C++ and its misfeatures is not a logical, convincing argument. The stated design goals for M3 is simplicity and safety, not feature richness or novelty. That should be what guides any discussion of M3 language design. > > - Jay > > > > Date: Tue, 28 Jun 2016 22:15:25 +0200 > > From: wagner at elegosoft.com > > To: rodney.m.bates at acm.org > > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > > Subject: Re: [M3devel] cm3 llvm backend? > > > > On Tue, 28 Jun 2016 11:40:06 -0500 > > "Rodney M. Bates" wrote: > > > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > > > > There are at least two big reasons for this. > > > > > > > > - The language really is improving. Programs > > > > written in the newer language are easier to read > > > > and often easier to optimize and sometimes easier > > > > to implement a compiler for. > > > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > > an implicit but very misguided belief that fewer keystrokes/characters > > > means increased readability. Or at least that writability is more > > > important than readability. So often, this means actual code is less > > > explicit. But this makes maintenance far worse. > > > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > > function calls and subscript lists to arrays. Along with optional > > > implicit operations like dereferencing, there are somewhere in the > > > teens of possible meanings for the innocent looking "f(x)". I have > forgotten > > > the exact number, but once had to do the semantic analysis. That was > > > Ada 83. Maybe more have been added since. For the poor schmuck who > > > gets called at 3:00 AM to fix a bug in half a million lines of code > > > she didn't write, this is a readability disaster. The savings of > > > keystrokes in not making the operations explicit is penny-wise and > > > million-pound foolish. > > > > I couldn't agree more with this. My focus seems to have moved away from > > programming to reading and analyzing code, too. Often it is almost next > > to impossible for me to find the exact meaning or definition of an > > expression or call without knowing all the other code which is not > > obviously related to the location in question. > > > > I would favour longer explicit syntax and clear meaning above > > shorter expressions any time. > > > > Olaf > > -- > > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 > 95 > > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Jun 29 01:54:06 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 28 Jun 2016 23:54:06 +0000 (UTC) Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com> <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st> <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com> <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: <53036089.5289165.1467158046154.JavaMail.yahoo@mail.yahoo.com> Hi all:it depends, Cardelli has stated in an interview the type system was the top part thing of language designers at its time. I recall SPwM3 states also that they video recorded all design sessions, though provably these would not be very attracting to the new modula-3 programmer or computer scientists (instead of it, discouraging). Probably the safety and simplicity were user/programmer priorities Thanks in advance El Martes 28 de junio de 2016 17:39, Darko Volaric escribi?: On Tue, Jun 28, 2016 at 11:34 PM, Jay K wrote: ?I find myself on the other side of this. ?There are many people on both sides. ?There are countless examples. ? Here are two. ?People don't like operator loading.? ?They don't want me to write? ?string operator+(string, string); ?but nobody seems to mind: ???float f = 1.0 + 2.0;? ? int?i = 1 + 2; ? why is + ok on floats and ints, overloaded, but not user defined types such as string??? The notion that limited fixed operator overloading justifies unlimited user operator overloading is an asinine argument. The former you can easily keep in your head, since the rules are simple and clear. You know given a + b + c + 1.0 that a, b and c are floating point values, or if you didn't have the literal you'd know by knowing the type of one value. But given user overloading, what is the type of a + b + c + 1.0 or some other expression? You have to know what the type of each value is, you need to know the definitions of the overloads for each type pair, you have to work out what the precedence and associativity rules are so you're evaluating the expression correctly and you have to know the type that each overloaded operator returns and work out what the new type pairs are at each stage of evaluation of the expression. ? ?People don' t like type inferencing.? ??? auto a = 1.0 + 2.0;? ???auto b = 1 + 2; ??Modula-3 has this more than C and C++98 already, so maybe people here don't mind. ???VAR a := 1.0 + 2.0; ??VAR b := 1 + 2; ?In either case, nobody seems to mind temporaries without explicit types in function calls or more generally subexpressions. ?F(1 + 2); ?F2(1.0 + 2.0); Again, this is "type inference" in the most simplistic case and hardly justifies unbounded type inferencing. The static type pf every expression (and subexpression) is known in M3, merely using it introduces no complexity. The static type is easily determined because the rules for determining the type of any expression are small, fixed, simple and strict. If you really want to understand type inferencing and overloading, have a look at the Swift programming language. You'll find yourself writing an innocuous expression and the compiler will tell you that the expression is "ambiguous" or "too complex" even though there doesn't seem any other way to interpret the expression or it involves only one operator/type combo used more than once. What the compiler is really telling you is that it can't infer the type of some subexpression or choose the right method and you must now start applying casts to make it obvious (this doesn't always work and sometimes you have to rewrite it wholesale). Why does this happen? Because Swift allows you to overload functions and operators based on parameter type and return type. You say "people are on both sides of this" but good design isn't a democracy or a popularity contest. If you walk around downtown San Francisco you can find many people who will agree with you. You will also find as many people who claim that the government is beaming thoughts into their heads via satellite. How people feel about language features and the fact that they are untroubled by C++ and its misfeatures is not a logical, convincing argument.? The stated design goals for M3 is simplicity and safety, not feature richness or novelty. That should be what guides any discussion of M3 language design. ? ?- Jay > Date: Tue, 28 Jun 2016 22:15:25 +0200 > From: wagner at elegosoft.com > To: rodney.m.bates at acm.org > CC: rodney_bates at lcwb.coop; jay.krell at cornell.edu; m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 llvm backend? > > On Tue, 28 Jun 2016 11:40:06 -0500 > "Rodney M. Bates" wrote: > > > On 06/27/2016 08:03 PM, Jay K wrote: > > > > A teacher of mine called this behavior "version junkie". > > > > > > > > > There are at least two big reasons for this. > > > > > > - The language really is improving. Programs > > > written in the newer language are easier to read > > > and often easier to optimize and sometimes easier > > > to implement a compiler for. > > > > Sometimes so, sometimes not. Sadly, many language "features" reflect > > an implicit but very misguided belief that fewer keystrokes/characters > > means increased readability. Or at least that writability is more > > important than readability. So often, this means actual code is less > > explicit. But this makes maintenance far worse. > > > > E.g., Ada decided use parentheses for both actual parameter lists to > > function calls and subscript lists to arrays. Along with optional > > implicit operations like dereferencing, there are somewhere in the > > teens of possible meanings for the innocent looking "f(x)". I have forgotten > > the exact number, but once had to do the semantic analysis. That was > > Ada 83. Maybe more have been added since. For the poor schmuck who > > gets called at 3:00 AM to fix a bug in half a million lines of code > > she didn't write, this is a readability disaster. The savings of > > keystrokes in not making the operations explicit is penny-wise and > > million-pound foolish. > > I couldn't agree more with this. My focus seems to have moved away from > programming to reading and analyzing code, too. Often it is almost next > to impossible for me to find the exact meaning or definition of an > expression or call without knowing all the other code which is not > obviously related to the location in question. > > I would favour longer explicit syntax and clear meaning above > shorter expressions any time. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Jun 29 04:02:44 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 02:02:44 +0000 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <20160625142459.7f79d9f7dd72fa403106248c@elegosoft.com>, <12c8fa4c-c60e-e77f-b2d9-8409498524b8@marino.st>, , <20160627081009.0fdc224eb02f78bf5e54f5ea@elegosoft.com>, , <57718A52.3030609@lcwb.coop> ,<5771C579.2000607@lcwb.coop> , <5772A866.7020601@lcwb.coop>, <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com>, , Message-ID: Integers and floats are not the universal types that all programmers should know about,?and no others. Each code base has types it deals with and functions over those types. The type of ?a + b? is just as clear or unclear as the type of: ?Add(a, b)? ?or, close:? ?Foo.Add(a, b)? ?The last is arguably most clear, the type is *probably* Foo.T.? ?but none of the cases are particularly clear, and far from all code? ?is written in this Foo_Add or Foo.Add fashion.? ?Plus the problem of mixing types.? ?VAR a,b,c:Foo.T;? ?b := Foo.Add(a, b);? ?c := Foo.AddI(a, 1);? ?(See there, I expanded to name overloading, this should be:? ?c := Foo.Add(a, 1);? ?or simply? ?c := a + 1; ?) ? Keep in mind, that not only are integers and floating point numbers not the sole and central types, but I can implement other "numbers". What if I have a multiprecision arithmetic library? Using "+" is very natural there. Giving it a longer name like Add hurts, doesn't help. Context is critical, and there is no one small context that suffices. What if I could say: a.+(b) ? Is that also bad? Now, I'm actually very open minded and somewhat torn on the matter. I am faced with a very large code base, and plain text search with very little language awareness. In that situation, C is actually advantageous. Indeed I don't search for "+" or "add", but specifically Foo_Add. Some argue that an IDE lets me just hover over some identifier and it shows me the type. But this does not scale. IDEs do not scale. ?(Not to mention that I don't use one...) Plain text search does scale. I believe language aware search can scale, but not everybody is using that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. It should have been pitched, imho, for its browsing/reading, not for its editing/development, at least as far as it got -- apparently browser-based development is viable these days. And also preprocessor token pasting is a bit of an enemy of plain text search. Also, I would have found Modula-3 and/or Quake much clear if they had used "+" for string concatenation, instead of ampersand. Context is kind of a sliding scale. Sometimes I have a lot, sometimes very little. Code presentation should perhaps adapt based on its viewer, but that is a fantasy, as code presentation is stuck in plain text, as directly typed by a human. Nobody is yet editing or viewing an abstract representation. Anyway, I'm well aware that there are a fair number of people on both sides of these positions.? ?- Jay ________________________________ > From: lists at darko.org > Date: Wed, 29 Jun 2016 00:39:17 +0200 > To: jay.krell at cornell.edu > Subject: Re: [M3devel] cm3 llvm backend? > CC: m3devel at elegosoft.com; rodney.m.bates at acm.org > > > > On Tue, Jun 28, 2016 at 11:34 PM, Jay K > > wrote: > I find myself on the other side of this. > There are many people on both sides. > > There are countless examples. > > > Here are two. > > > People don't like operator loading. > > They don't want me to write > > string operator+(string, string); > > > but nobody seems to mind: > > > float f = 1.0 + 2.0; > int i = 1 + 2; > > why is + ok on floats and ints, overloaded, but not user defined > types such as string? > > > > > The notion that limited fixed operator overloading justifies unlimited > user operator overloading is an asinine argument. The former you can > easily keep in your head, since the rules are simple and clear. You > know given a + b + c + 1.0 that a, b and c are floating point values, > or if you didn't have the literal you'd know by knowing the type of one > value. > > But given user overloading, what is the type of a + b + c + 1.0 or some > other expression? You have to know what the type of each value is, you > need to know the definitions of the overloads for each type pair, you > have to work out what the precedence and associativity rules are so > you're evaluating the expression correctly and you have to know the > type that each overloaded operator returns and work out what the new > type pairs are at each stage of evaluation of the expression. > > > > > People don' t like type inferencing. > > auto a = 1.0 + 2.0; > auto b = 1 + 2; > > Modula-3 has this more than C and C++98 already, so maybe people here > don't mind. > > VAR a := 1.0 + 2.0; > VAR b := 1 + 2; > > In either case, nobody seems to mind temporaries without explicit > types in function calls or more generally subexpressions. > > F(1 + 2); > F2(1.0 + 2.0); > > > > Again, this is "type inference" in the most simplistic case and hardly > justifies unbounded type inferencing. The static type pf every > expression (and subexpression) is known in M3, merely using it > introduces no complexity. The static type is easily determined because > the rules for determining the type of any expression are small, fixed, > simple and strict. > > > If you really want to understand type inferencing and overloading, have > a look at the Swift programming language. You'll find yourself writing > an innocuous expression and the compiler will tell you that the > expression is "ambiguous" or "too complex" even though there doesn't > seem any other way to interpret the expression or it involves only one > operator/type combo used more than once. What the compiler is really > telling you is that it can't infer the type of some subexpression or > choose the right method and you must now start applying casts to make > it obvious (this doesn't always work and sometimes you have to rewrite > it wholesale). Why does this happen? Because Swift allows you to > overload functions and operators based on parameter type and return > type. > > > > You say "people are on both sides of this" but good design isn't a > democracy or a popularity contest. If you walk around downtown San > Francisco you can find many people who will agree with you. You will > also find as many people who claim that the government is beaming > thoughts into their heads via satellite. How people feel about language > features and the fact that they are untroubled by C++ and its > misfeatures is not a logical, convincing argument. > > The stated design goals for M3 is simplicity and safety, not feature > richness or novelty. That should be what guides any discussion of M3 > language design. > > > > > - Jay > > >> Date: Tue, 28 Jun 2016 22:15:25 +0200 >> From: wagner at elegosoft.com >> To: rodney.m.bates at acm.org >> CC: rodney_bates at lcwb.coop; > jay.krell at cornell.edu; > m3devel at elegosoft.com >> Subject: Re: [M3devel] cm3 llvm backend? >> >> On Tue, 28 Jun 2016 11:40:06 -0500 >> "Rodney M. Bates" > > wrote: >> >>> On 06/27/2016 08:03 PM, Jay K wrote: >>>>> A teacher of mine called this behavior "version junkie". >>>> >>>> >>>> There are at least two big reasons for this. >>>> >>>> - The language really is improving. Programs >>>> written in the newer language are easier to read >>>> and often easier to optimize and sometimes easier >>>> to implement a compiler for. >>> >>> Sometimes so, sometimes not. Sadly, many language "features" reflect >>> an implicit but very misguided belief that fewer keystrokes/characters >>> means increased readability. Or at least that writability is more >>> important than readability. So often, this means actual code is less >>> explicit. But this makes maintenance far worse. >>> >>> E.g., Ada decided use parentheses for both actual parameter lists to >>> function calls and subscript lists to arrays. Along with optional >>> implicit operations like dereferencing, there are somewhere in the >>> teens of possible meanings for the innocent looking "f(x)". I have > forgotten >>> the exact number, but once had to do the semantic analysis. That was >>> Ada 83. Maybe more have been added since. For the poor schmuck who >>> gets called at 3:00 AM to fix a bug in half a million lines of code >>> she didn't write, this is a readability disaster. The savings of >>> keystrokes in not making the operations explicit is penny-wise and >>> million-pound foolish. >> >> I couldn't agree more with this. My focus seems to have moved away from >> programming to reading and analyzing code, too. Often it is almost next >> to impossible for me to find the exact meaning or definition of an >> expression or call without knowing all the other code which is not >> obviously related to the location in question. >> >> I would favour longer explicit syntax and clear meaning above >> shorter expressions any time. >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 >> Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: > DE163214194 > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > _______________________________________________ M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Wed Jun 29 04:18:03 2016 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 28 Jun 2016 22:18:03 -0400 Subject: [M3devel] cm3 llvm backend? In-Reply-To: References: <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> Message-ID: <20160629021803.GC6086@topoi.pooq.com> On Wed, Jun 29, 2016 at 02:02:44AM +0000, Jay K wrote: > I believe language aware search can scale, but not everybody is using > that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. > It should have been pitched, imho, for its browsing/reading, not for its > editing/development, at least as far as it got -- apparently browser-based > development is viable these days. I remember reading Modula 3 code via a browser. Expecially following classes and their inheritance from module to module. I don't recall being able to edit it there. Are those viewing tools still around? Might tat have been a tool with PM3? -- hendrik From jay.krell at cornell.edu Wed Jun 29 05:54:15 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 03:54:15 +0000 Subject: [M3devel] m3front scanner div wierdness? Message-ID: Does anyone understand this stuff in m3front/Scanner.m3: ?Here vs. LocalHere? ?SameFile? ? ?I understand only this nuance: ? offset MOD MaxLines ?? ? MaxLines ?= 100000; ?is to crudely handle that when asserts fail, ?they pack the line number in with the assertion failure code, ?potentially loosing bits. ? ?I don't think this is a good design, they should just be separate INTEGERS, ?but this is besides the point. ? ?What doesn't makes sense to me is the machinations around file name. Here:? ? ? file := files [offset DIV MaxLines]; vs. LocalHere: ? ? file := local_files [fnum]; LocalHere makes sense. Here does not. PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = ? BEGIN ? ? RETURN (a DIV MaxLines) = (b DIV MaxLines); ? END SameFile; Shouldn't this just be a = b? Well, anyway, this SameFile function isn't called. Here and LocalHere are used. I'm looking here because I want to add a temporary measure such that the file names are leaf-only. In particular, because generic modules have target names in their paths and I want to temporarly remove target names from output, so I can prove that a few targets are identical. I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. This has to percolate quite a bit through the system -- the backends and the runtime. And then this Here vs. LocalHere difference should fall away. But still, what is it trying to do? Thank you, ?- Jay From jay.krell at cornell.edu Wed Jun 29 06:08:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 04:08:07 +0000 Subject: [M3devel] release 3.6, and history Message-ID: I'm wondering: ?- if anyone has the 3.6 release, can make it available? ?- I can probably find it. We can put it on github? And more so, history prior to 3.6? And between 3.6 and 4.0? Or if Bill Kalsow is still around and available to ask questions? The Scanner thing is wierd to me. I do see some changes..maybe I did this.. old version: PROCEDURE Here (VAR file: TEXT; VAR line: INTEGER) = BEGIN file := files [offset DIV MaxLines]; line := offset MOD MaxLines; END Here; PROCEDURE LocalHere (VAR file: TEXT; VAR line: INTEGER) = VAR fnum := offset DIV MaxLines; BEGIN IF (local_files[fnum] = NIL) THEN local_files[fnum] := Host.FileTail (files[fnum]); END; file := local_files [fnum]; line := offset MOD MaxLines; END LocalHere; There is this div/mod relationship, but I still find it wierd to use div there. You can see the FileTail which is maybe the change I want anyway. Here, yes, me: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3front/src/misc/Scanner.m3.diff?r1=1.5;r2=1.6 Revision?1.6:?download?- view:?text,?markup,?annotated?-?select?for?diffs Thu Feb 28 09:23:22 2008 MET?(8 years, 4 months ago) by?jkrell Branches:?MAIN Diff to: previous 1.5:?preferred,?unified Changes since revision 1.5: +1 -1 lines put full paths to source files in debug info This has the minor downsides of: 1) grows the debug info (it is already huge; who is counting?) 2) reveals file system layout in debug info (privacy?) 3) does it inhibit debugging files from other people's machines or does gdb dir still work? but definitely makes for a more pleasant debugging experience when debugging stuff you have built yourself. The linear searching to see if a name has been allocated a number yet will obviously slow way down due to a large increase in common prefixes, but that should be a hash table anyway. Linear search is lame. (or a trie, but working from the ends of the strings, minus the last one or few characters, due to common prefixes as well as common suffixes) Note that both m3front and m3cc changes are needed as m3front has paths relative to the current working directory or such. For most packages, you can get by without the m3front change and just prepend "../src/" to the path in m3cc, but that doesn't work for hierarchical packages such as libm3 and m3core which I am debugging. I still believe debug info should have full paths. It always does on Windows for C and C++ (and this isn't a language thing). It didn't always but they fixed it at some point. But generic modules maybe something else -- like line directives and full paths back to the .mg files. ?- Jay From jay.krell at cornell.edu Wed Jun 29 07:58:30 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 05:58:30 +0000 Subject: [M3devel] release 3.6, and history In-Reply-To: <08431E36-275B-474B-8D9C-A8517F287B8E@purdue.edu> References: , <08431E36-275B-474B-8D9C-A8517F287B8E@purdue.edu> Message-ID: The cvs history seems to be there, but isn't always clear. ?I found the elegosoft cvsweb sometimes easier to navigate. ? And other things -- it isn't obvious in github how to get file contents either before or after a change -- one is easy and for the other I look at the next/previous change and its before/after file contents. I guess I'll wait, eventually git might be as good as perforce.. I there is nothing before 5.0 or 4.0 though I think. I should have 3.6 somewhere around. ?- Jay ________________________________ > From: hosking at purdue.edu > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] release 3.6, and history > Date: Wed, 29 Jun 2016 05:22:33 +0000 > > All the history should still be there on github. > > On 29 Jun 2016, at 2:08 PM, Jay K > > wrote: > > I'm wondering: > - if anyone has the 3.6 release, can make it available? > - I can probably find it. We can put it on github? > From wagner at elegosoft.com Wed Jun 29 08:48:05 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 29 Jun 2016 08:48:05 +0200 Subject: [M3devel] cm3 llvm backend? In-Reply-To: <20160629021803.GC6086@topoi.pooq.com> References: <57718A52.3030609@lcwb.coop> <5771C579.2000607@lcwb.coop> <5772A866.7020601@lcwb.coop> <20160628221525.b79cc92fbd3b42e3af451aaa@elegosoft.com> <20160629021803.GC6086@topoi.pooq.com> Message-ID: <20160629084805.31275c2f310cb4a614628bf2@elegosoft.com> On Tue, 28 Jun 2016 22:18:03 -0400 Hendrik Boom wrote: > On Wed, Jun 29, 2016 at 02:02:44AM +0000, Jay K wrote: > > > I believe language aware search can scale, but not everybody is using > > that yet. Modula-3 was ahead here, with the .M3WEB/Reactor stuff. > > It should have been pitched, imho, for its browsing/reading, not for its > > editing/development, at least as far as it got -- apparently browser-based > > development is viable these days. > > I remember reading Modula 3 code via a browser. Expecially following > classes and their inheritance from module to module. I don't recall > being able to edit it there. Are those viewing tools still around? > > Might tat have been a tool with PM3? No, it's still there. The code is in * https://github.com/modula3/cm3/tree/master/m3-tools/m3browser * https://github.com/modula3/cm3/tree/master/m3-tools/m3tohtml * https://github.com/modula3/cm3/tree/master/m3-sys/cm3ide ("Reactor") Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jay.krell at cornell.edu Wed Jun 29 11:41:52 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 09:41:52 +0000 Subject: [M3devel] full source paths in debug info, environment variable to contain switches? Message-ID: I raised this matter years ago and people disagreed. Debug info, and probably compiler diagnostics, should have full source paths. Ease of use of compiler diagnostics varies, better and worse, with full paths. Debuggers similar. gcc and clang on Darwin use full paths. Observe: jair:~ jay$ cat 1.c int main() { } jair:~ jay$ rm a.out 1.o jair:~ jay$ /Users/jay/gcc-6.1.0-min/bin/gcc -g -c 1.c jair:~ jay$ dwarfdump 1.o | fgrep 1.c ?? ? ? ? ? ? AT_name( "1.c" ) ?? ? ? ? ? ? ? ? AT_decl_file( "/Users/jay/1.c" ) jair:~ jay$ rm a.out 1.o rm: a.out: No such file or directory jair:~ jay$ clang -c -g 1.c jair:~ jay$ dwarfdump 1.o | fgrep 1.c ?? ? ? ? ? ? AT_name( "1.c" ) ?? ? ? ? ? ? ? ? AT_decl_file( "/Users/jay/1.c" ) Now, for my immediate task at hand, I actually don't want this, at least for instantiated generic modules/interfaces. I propose cm3 should have some command line switches to control this behavior. The default should be full paths in debug info. I further propose that some environment variables be implemented, which shall contain cm3 switches. There is some precedent for this in the world. In the Microsoft C/C++ toolset: INCLUDES is include paths LIB is library paths CL, _CL_, LINK, and _LINK_ environment variables all contribute to compiler/linker command line (I think prepending and appending). I don't propose that many or generally that short of names, but maybe CM3_OPTIONS or CM3_ENV ? I don't want to use just CM3, because people might want that to contain the path to cm3 (without switches( Probably some command line option should quash this too, like cm3 -noenv Which maybe should also work, if it is in CM3_ENV, means to ignore the rest ? This is at once crude and powerfuli and elegant and hacky and efficient and inefficient. It is good because it is easy to use and will usually do what people want. It is bad because it is not scoped -- it travels through entire process trees. It is inefficient because it travels into processes that don't use it. For my current work, I went to shorten paths such that the target doesn't occur in them, so I can show that various targets generate identical C code, so I can fold them down to fewer targets. The paths to generic instantiations messes this up, at least. ?- Jay From jay.krell at cornell.edu Wed Jun 29 11:44:51 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 09:44:51 +0000 Subject: [M3devel] source paths to generics? Message-ID: I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, the source file I am told about is the instantiated i3/m3 file. This isn't particularly useful for the programmer or convenient for the compiler, right? The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? I guess it is slightly easier in the compiler, but easy to do better? I should look into make it so? My real agenda is I want to see: ? ../src/foo.mg instead of ?I386_DARWIN/foo.m3 I don't want the target variation, but other points seem true also, right? Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? I'll try to poke around more. ?- Jay From jay.krell at cornell.edu Wed Jun 29 17:22:14 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 15:22:14 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: References: Message-ID: mfront/src/misc/M3Header.m3 has this: ? ? (* build a synthetic file name & start reading *) ? ? filename := old_filename & " => " & filename; ? ? Scanner.Push (filename, s.generic, is_main := Scanner.in_main); and instantiated generics aren't what I thought. I thought the build system or compiler did a textual substitution of the generic parameters into the entire implementation, and saved that to the file system, and had the assert/debug paths point to it. So could step through what looks exactly like an .m3 file. But it doesn't. The instantiated file is a little stub, like: jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 (*generated by m3build*) MODULE ?TextAtomTbl = Table (Text, Atom) END TextAtomTbl. so I have one or two proposals. 1) old_filename above contains the target, it looks like: "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" or "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" These both occur. I'm not sure what the second is, seems like a mistake. I suggest the first string in both pairs be changed to be the leaf element, like: "TextAtomTbl.m3 => ../src/table/Table.mg" or "TextAtomTbl.m3 => ../src/table" and maybe the second string in both cases should be as it is in the second pair. 2) Possibly it should reall just be: ?../src/table/Table.mg? ? ?and developer can step through that, knowing that it is a somewhat abstracted ?form of what is running ? ?I'm willing to do this under like: ? ?CONST TemporaryForTargetConvergence = TRUE ? ?or VAR TemporaryForTargetConvergence: BOOLEAN; and a command line switch, but I suspect it isn't really useful to anyone asis, so might as well remove target unconditionally. ? ?? ? - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Wed, 29 Jun 2016 09:44:51 +0000 > Subject: [M3devel] source paths to generics? > > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > the source file I am told about is the instantiated i3/m3 file. > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > I guess it is slightly easier in the compiler, but easy to do better? > > I should look into make it so? > > My real agenda is I want to see: > ../src/foo.mg > > instead of > I386_DARWIN/foo.m3 > > I don't want the target variation, but other points seem true also, right? > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > I'll try to poke around more. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Wed Jun 29 18:09:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 11:09:34 -0500 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: Message-ID: <5773F2BE.3050902@lcwb.coop> On 06/28/2016 10:54 PM, Jay K wrote: > Does anyone understand this stuff in m3front/Scanner.m3: > > Here vs. LocalHere? > SameFile? > > > I understand only this nuance: > offset MOD MaxLines > > MaxLines = 100000; > > > is to crudely handle that when asserts fail, > they pack the line number in with the assertion failure code, > potentially loosing bits. > > > I don't think this is a good design, they should just be separate INTEGERS, > but this is besides the point. > This is just pure speculation, but I am very confident of it. These offsets have a very high occurrence count. There is code all over m3front that copies Scanner.offset into various data structures. So the small space saving of one INTEGER instead of two would be multiplied by a big number. I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. Today, I have 8 Gig in the one I bought, and could easily afford more, if I thought I needed it. Two integers would be cleaner, but this design is not too bad *if* you know the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one field. As pure documentation, there really should be a distinct type name (say FileNoAndLineNo?) for all values that use this representation, even though it just equates to INTEGER. There are a lot of variables lying around all over the front end that use this invariant, but are just declared as INTEGER. That's maintainer-hostile. > > What doesn't makes sense to me is the machinations around file name. > > > Here: > file := files [offset DIV MaxLines]; > > vs. LocalHere: > file := local_files [fnum]; > > > LocalHere makes sense. Here does not. > > > PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = > BEGIN > RETURN (a DIV MaxLines) = (b DIV MaxLines); > END SameFile; > > > > Shouldn't this just be a = b? > As coded, this will return TRUE if a and b are different line numbers within the same file. The name "SameFile" at least suggests that is what is intended. A good example of a place where it would have been clearer if a & b were declared as the type name I proposed above. > > Well, anyway, this SameFile function isn't called. > > Here and LocalHere are used. > > > I'm looking here because I want to add a temporary measure > such that the file names are leaf-only. > > > In particular, because generic modules have target names in their paths > and I want to temporarly remove target names from output, so I can prove > that a few targets are identical. > > > I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. > This has to percolate quite a bit through the system -- the backends and the runtime. > > > And then this Here vs. LocalHere difference should fall away. > But still, what is it trying to do? > > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Jun 29 18:18:18 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 11:18:18 -0500 Subject: [M3devel] release 3.6, and history In-Reply-To: References: Message-ID: <5773F4CA.2030309@lcwb.coop> On 06/28/2016 11:08 PM, Jay K wrote: > I'm wondering: > - if anyone has the 3.6 release, can make it available? > - I can probably find it. We can put it on github? > > And more so, history prior to 3.6? And between 3.6 and 4.0? > > Or if Bill Kalsow is still around and available to ask questions? > > The Scanner thing is wierd to me. > > I do see some changes..maybe I did this.. > > old version: > PROCEDURE Here (VAR file: TEXT; VAR line: INTEGER) = > BEGIN > file := files [offset DIV MaxLines]; > line := offset MOD MaxLines; > END Here; > > PROCEDURE LocalHere (VAR file: TEXT; VAR line: INTEGER) = > VAR fnum := offset DIV MaxLines; > BEGIN > IF (local_files[fnum] = NIL) THEN > local_files[fnum] := Host.FileTail (files[fnum]); ^------------------------ This explains it. As currently coded, Here and LocalHere always just return the same VAR parameter results, with LocalHere having the side effect of copying the file name from array files to local_files. But local_files isn't used any other way, so the only effect is that a subsequent call to LocalHere, for the same file, computes the same "file" result slightly differently. But as in this old version, Here gives the full path and LocalHere gives a shortened one, as Host.FileTail computes. local_files just caches this, so it won't be recomputed. on a subsequent call. I think this should give you what you want. There are calls to LocalHere in CG, WebInfo, and InfoThisFile, plus several elsewhere to Here. So if you put the Host.FileTail call back, Here & LocalHere make sense and give you a choice at each place. > END; > file := local_files [fnum]; > line := offset MOD MaxLines; > END LocalHere; > > There is this div/mod relationship, but I still find it wierd to use div there. > You can see the FileTail which is maybe the change I want anyway. > > > Here, yes, me: > > > > https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3front/src/misc/Scanner.m3.diff?r1=1.5;r2=1.6 > > Revision 1.6: download - view: text, markup, annotated - select for diffs > Thu Feb 28 09:23:22 2008 MET (8 years, 4 months ago) by jkrell > Branches: MAIN > Diff to: previous 1.5: preferred, unified > Changes since revision 1.5: +1 -1 lines > put full paths to source files in debug info > > This has the minor downsides of: > 1) grows the debug info (it is already huge; who is counting?) > 2) reveals file system layout in debug info (privacy?) > 3) does it inhibit debugging files from other people's machines or does gdb dir still work? > > but definitely makes for a more pleasant debugging experience when > debugging stuff you have built yourself. > > The linear searching to see if a name has been allocated a number yet > will obviously slow way down due to a large increase in common prefixes, > but that should be a hash table anyway. Linear search is lame. > (or a trie, but working from the ends of the strings, minus the last one or few > characters, due to common prefixes as well as common suffixes) > > Note that both m3front and m3cc changes are needed as m3front has paths > relative to the current working directory or such. > > For most packages, you can get by without the m3front change and just prepend > "../src/" to the path in m3cc, but that doesn't work for hierarchical packages > such as libm3 and m3core which I am debugging. > > I still believe debug info should have full paths. > It always does on Windows for C and C++ (and this isn't a language thing). > It didn't always but they fixed it at some point. > > > But generic modules maybe something else -- like line directives and full paths back to the .mg files. > > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 29 18:31:19 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 16:31:19 +0000 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: <5773F2BE.3050902@lcwb.coop> References: , <5773F2BE.3050902@lcwb.coop> Message-ID: My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. ?(in either case, the address space was 16 bit and ROM and periphal memory was in there, so various bank switching employed to gain access; later similar with the 128k RAM machines...) I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine as you suggest). I know that 32bits is overkill for a line number. I also know 16 bits isn't overkill -- but more than 16 are used here so ok. There is/was a warning in the Visual C++ compiler about truncating line numbers or terminating debug information after 64k lines. Midl output would trigger it. I still don't follow completely. It seems there is an aliasing situation, where lines very far apart can be deemed the same. I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... In general you have to balance: ?1 System still compilable on 32bit.? ?2 vs. 64bit system can do more? For the first case, you want to limit integers to 32bits and for the second you do not. Also convenience and perf suggest *not* having to sprinkle div/mod all around, though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... ?> FileNoAndLineNo Lately this is called "location" and things even have starts and ends, so the error messages can output a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill but clang is there and I think gcc went there. Yes the data is larger. Maybe shorter FileAndLine? I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) ?- Jay ---------------------------------------- > Date: Wed, 29 Jun 2016 11:09:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3front scanner div wierdness? > > > > On 06/28/2016 10:54 PM, Jay K wrote: >> Does anyone understand this stuff in m3front/Scanner.m3: >> >> Here vs. LocalHere? >> SameFile? >> >> >> I understand only this nuance: >> offset MOD MaxLines >> >> MaxLines = 100000; >> >> >> is to crudely handle that when asserts fail, >> they pack the line number in with the assertion failure code, >> potentially loosing bits. >> >> >> I don't think this is a good design, they should just be separate INTEGERS, >> but this is besides the point. >> > > This is just pure speculation, but I am very confident of it. These > offsets have a very high occurrence count. There is code all over m3front > that copies Scanner.offset into various data structures. So the small space > saving of one INTEGER instead of two would be multiplied by a big number. > I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. > Today, I have 8 Gig in the one I bought, and could easily afford more, if I > thought I needed it. > > Two integers would be cleaner, but this design is not too bad *if* you know > the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one > field. As pure documentation, there really should be a distinct type name > (say FileNoAndLineNo?) for all values that use this representation, even > though it just equates to INTEGER. There are a lot of variables lying around > all over the front end that use this invariant, but are just declared as > INTEGER. That's maintainer-hostile. > > >> >> What doesn't makes sense to me is the machinations around file name. >> >> >> Here: >> file := files [offset DIV MaxLines]; >> >> vs. LocalHere: >> file := local_files [fnum]; >> >> >> LocalHere makes sense. Here does not. >> >> >> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >> BEGIN >> RETURN (a DIV MaxLines) = (b DIV MaxLines); >> END SameFile; >> >> >> >> Shouldn't this just be a = b? >> > > As coded, this will return TRUE if a and b are different line numbers within > the same file. The name "SameFile" at least suggests that is what is intended. > A good example of a place where it would have been clearer if a & b were > declared as the type name I proposed above. > >> >> Well, anyway, this SameFile function isn't called. >> >> Here and LocalHere are used. >> >> >> I'm looking here because I want to add a temporary measure >> such that the file names are leaf-only. >> >> >> In particular, because generic modules have target names in their paths >> and I want to temporarly remove target names from output, so I can prove >> that a few targets are identical. >> >> >> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >> This has to percolate quite a bit through the system -- the backends and the runtime. >> >> >> And then this Here vs. LocalHere difference should fall away. >> But still, what is it trying to do? >> >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Wed Jun 29 18:34:06 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 16:34:06 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: References: , Message-ID: There is similar code in Module.m3 and that is the code producing the data "I don't like". M3Header isn't dead but I haven't seen it run yet. I changed them from "=>" to "1=>" and "2=>" and looked where those occur in the output .s files under -keep. This is kinda something I wish were easier, like more strings need to be longer for easier finding w/o ambiguity (the flip side of my context arguments!) As things stand, I can't check that in. I suppose maybe with a CG.comment but nevermind. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Subject: RE: [M3devel] source paths to generics? > Date: Wed, 29 Jun 2016 15:22:14 +0000 > > mfront/src/misc/M3Header.m3 has this: > > > (* build a synthetic file name & start reading *) > filename := old_filename & " => " & filename; > Scanner.Push (filename, s.generic, is_main := Scanner.in_main); > > > and instantiated generics aren't what I thought. > I thought the build system or compiler did a textual > substitution of the generic parameters into the entire implementation, > and saved that to the file system, > and had the assert/debug paths point to it. > > So could step through what looks exactly like an .m3 file. > > But it doesn't. The instantiated file is a little stub, like: > > jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 > (*generated by m3build*) > > MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. > > > so I have one or two proposals. > > 1) old_filename above contains the target, it looks like: > > "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" > or > "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" > > These both occur. > I'm not sure what the second is, seems like a mistake. > > I suggest the first string in both pairs be changed to be the leaf element, like: > > "TextAtomTbl.m3 => ../src/table/Table.mg" > or > "TextAtomTbl.m3 => ../src/table" > > and maybe the second string in both cases should be as it is in the second pair. > > 2) Possibly it should reall just be: > ../src/table/Table.mg > > and developer can step through that, knowing that it is a somewhat abstracted > form of what is running > > I'm willing to do this under like: > > CONST TemporaryForTargetConvergence = TRUE > > or > VAR TemporaryForTargetConvergence: BOOLEAN; > > and a command line switch, but I suspect it isn't really useful to anyone asis, > so might as well remove target unconditionally. > > > ? > > - Jay > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Date: Wed, 29 Jun 2016 09:44:51 +0000 >> Subject: [M3devel] source paths to generics? >> >> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >> the source file I am told about is the instantiated i3/m3 file. >> >> This isn't particularly useful for the programmer or convenient for the compiler, right? >> >> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >> I guess it is slightly easier in the compiler, but easy to do better? >> >> I should look into make it so? >> >> My real agenda is I want to see: >> ../src/foo.mg >> >> instead of >> I386_DARWIN/foo.m3 >> >> I don't want the target variation, but other points seem true also, right? >> >> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >> >> I'll try to poke around more. >> >> - Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > From rodney_bates at lcwb.coop Wed Jun 29 19:01:08 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 12:01:08 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: Message-ID: <5773FED4.7020509@lcwb.coop> On 06/29/2016 04:44 AM, Jay K wrote: > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > the source file I am told about is the instantiated i3/m3 file. > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > I guess it is slightly easier in the compiler, but easy to do better? Yes. m3gdb does this, in many cases: ------------------------------------------------------------------------------------------------------------ (m3gdb) b VarArray.mg:136 Breakpoint 2 at 0x4073c3: file ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg, line 136. (m3gdb) c Continuing. Breakpoint 2, New (InitElemValue=-9223372036854775808, InitialAlloc= RECORD Lo = 9223372036854775807; Hi = -9223372036854775808; END, ExpansionFactor=1.10000002) at ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg:136 136 LExpansionFactorR := MAX ( ExpansionFactor , MinExpansionFactorR ) (m3gdb) ------------------------------------------------------------------------------------------------------------ It has a bug though. Setting a breakpoint to a .mg file before any execution has started appears to work, but the breakpoint won't trigger. I have not looked at cases like runtime errors without m3gdb. This does raise the question, however, that you might very well want to distinguish different instantiations of the same generic when setting a breakpoint. This is a good example of where a debugger needs to have awareness of your language. Modula-3 instantiations, being separate compilation units and having separate generic and instantiation files is a model that, AFAIK, does not occur in other languages. > > I should look into make it so? > > My real agenda is I want to see: > ../src/foo.mg > > instead of > I386_DARWIN/foo.m3 > > I don't want the target variation, but other points seem true also, right? > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > I'll try to poke around more. > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Jun 29 20:10:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 29 Jun 2016 18:10:50 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <5773FED4.7020509@lcwb.coop> References: , <5773FED4.7020509@lcwb.coop> Message-ID: Good point, and C++ systems do handle this.When you step into std::vector::foo, the debugger knows that the source line occurs many places and can prompt you for which you want,or it can assume you mean the current one. If you haven't yet stepped into it, it can prompt for which. Now, 1) this stuff is often inlined and 2) std::vector and std::vector, usually end up being identical code, so you can't break on them separately. The linker does these optimization, and some compilers.Even Modula-3 is subject to this optimization -- since the linker does it. It is *mostly* good, mostly just makes things smaller, but it also confuses a lot of people. The linker I believe has to be a little pessimistic, like if you take the address of a function, that can inhibit optimization, lest someone is comparing function pointers for equality, which is rare and frowned upon, but is sort of in the languages. it also has to get function comparison correct, which is a little tricky, what with functions referencing data and other functions. m3gdb is interpreting these strings I guess. what is with the strings that have only a directory and not a file name on the right side -- in M3Header.m3. If I make the left hand side just a leaf name, foo.m3, w/o ../AMD64_LINUX, that'll be nearly equivalent?i.e. it is assuming current directory is already AMD64_LINUX, or it is assuming src? See, funny thing is, the current paths resolve the same either way, but if I remove "../AMD64_LINUX", they don't. - Jay > Date: Wed, 29 Jun 2016 12:01:08 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > > > On 06/29/2016 04:44 AM, Jay K wrote: > > I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, > > the source file I am told about is the instantiated i3/m3 file. > > > > This isn't particularly useful for the programmer or convenient for the compiler, right? > > > > The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? > > I guess it is slightly easier in the compiler, but easy to do better? > > Yes. m3gdb does this, in many cases: > ------------------------------------------------------------------------------------------------------------ > (m3gdb) b VarArray.mg:136 > Breakpoint 2 at 0x4073c3: file ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg, line 136. > (m3gdb) c > Continuing. > > Breakpoint 2, New (InitElemValue=-9223372036854775808, InitialAlloc= > RECORD Lo = 9223372036854775807; Hi = -9223372036854775808; END, ExpansionFactor=1.10000002) > at ../AMD64_LINUX/IntIntVarArray.m3 => ../src/VarArray.mg:136 > 136 LExpansionFactorR := MAX ( ExpansionFactor , MinExpansionFactorR ) > (m3gdb) > > ------------------------------------------------------------------------------------------------------------ > > It has a bug though. Setting a breakpoint to a .mg file before any execution has > started appears to work, but the breakpoint won't trigger. > > I have not looked at cases like runtime errors without m3gdb. > > This does raise the question, however, that you might very well want to distinguish > different instantiations of the same generic when setting a breakpoint. > > This is a good example of where a debugger needs to have awareness of your language. > Modula-3 instantiations, being separate compilation units and having separate > generic and instantiation files is a model that, AFAIK, does not occur in other > languages. > > > > > I should look into make it so? > > > > My real agenda is I want to see: > > ../src/foo.mg > > > > instead of > > I386_DARWIN/foo.m3 > > > > I don't want the target variation, but other points seem true also, right? > > > > Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? > > > > I'll try to poke around more. > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Jun 30 02:34:13 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 29 Jun 2016 19:34:13 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: , Message-ID: <57746905.2080007@lcwb.coop> Hmm. I poked around a bit, and it looks like there is some cruft here, left over from something. The only call I can find on M3Header.Parse is M3Front.m3:37. This is in M3Front.ParseImports. I can find no calls on that at all. (There are other procedures by this name.) Also, it appears M3Header.ParseImports actually collects the exports, the imports, the generic actuals. As for the concocted (by M3Header) name of the form => , this looks to be used only to initialize the scanner's file number while scanning up through the formals of the generic, but that is not used. On 06/29/2016 11:34 AM, Jay K wrote: > There is similar code in Module.m3 and that is the code producing > the data "I don't like". > > M3Header isn't dead but I haven't seen it run yet. > > I changed them from "=>" to "1=>" and "2=>" and looked > where those occur in the output .s files under -keep. > > This is kinda something I wish were easier, like more strings > need to be longer for easier finding w/o ambiguity (the flip > side of my context arguments!) > > As things stand, I can't check that in. > I suppose maybe with a CG.comment but nevermind. > > - Jay > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: m3devel at elegosoft.com >> Subject: RE: [M3devel] source paths to generics? >> Date: Wed, 29 Jun 2016 15:22:14 +0000 >> >> mfront/src/misc/M3Header.m3 has this: >> >> >> (* build a synthetic file name & start reading *) >> filename := old_filename & " => " & filename; >> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >> >> >> and instantiated generics aren't what I thought. >> I thought the build system or compiler did a textual >> substitution of the generic parameters into the entire implementation, >> and saved that to the file system, >> and had the assert/debug paths point to it. >> >> So could step through what looks exactly like an .m3 file. >> >> But it doesn't. The instantiated file is a little stub, like: >> >> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >> (*generated by m3build*) >> >> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >> >> >> so I have one or two proposals. >> >> 1) old_filename above contains the target, it looks like: >> >> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >> or >> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >> >> These both occur. >> I'm not sure what the second is, seems like a mistake. >> >> I suggest the first string in both pairs be changed to be the leaf element, like: >> >> "TextAtomTbl.m3 => ../src/table/Table.mg" >> or >> "TextAtomTbl.m3 => ../src/table" >> >> and maybe the second string in both cases should be as it is in the second pair. >> >> 2) Possibly it should reall just be: >> ../src/table/Table.mg >> >> and developer can step through that, knowing that it is a somewhat abstracted >> form of what is running >> >> I'm willing to do this under like: >> >> CONST TemporaryForTargetConvergence = TRUE >> >> or >> VAR TemporaryForTargetConvergence: BOOLEAN; >> >> and a command line switch, but I suspect it isn't really useful to anyone asis, >> so might as well remove target unconditionally. >> >> >> ? >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>> Subject: [M3devel] source paths to generics? >>> >>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>> the source file I am told about is the instantiated i3/m3 file. >>> >>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>> >>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>> I guess it is slightly easier in the compiler, but easy to do better? >>> >>> I should look into make it so? >>> >>> My real agenda is I want to see: >>> ../src/foo.mg >>> >>> instead of >>> I386_DARWIN/foo.m3 >>> >>> I don't want the target variation, but other points seem true also, right? >>> >>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>> >>> I'll try to poke around more. >>> >>> - Jay >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 06:49:00 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:49:00 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <57746905.2080007@lcwb.coop> References: , , , , <57746905.2080007@lcwb.coop> Message-ID: I guess neither of us looked outside of m3front. The code runs. Not clear if the strings can get output. Maybe from asserts or Compiler.ThisFile in a generic? I"ll try some more. jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 ? ? ids := M3Front.ParseImports (source, s.m3env); ?- Jay ---------------------------------------- > Date: Wed, 29 Jun 2016 19:34:13 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > Hmm. I poked around a bit, and it looks like there is some cruft here, > left over from something. > > The only call I can find on M3Header.Parse is M3Front.m3:37. This is > in M3Front.ParseImports. I can find no calls on that at all. (There > are other procedures by this name.) > > Also, it appears M3Header.ParseImports actually collects the exports, the imports, > the generic actuals. > > As for the concocted (by M3Header) name of the form => , > this looks to be used only to initialize the scanner's file number while scanning > up through the formals of the generic, but that is not used. > > On 06/29/2016 11:34 AM, Jay K wrote: >> There is similar code in Module.m3 and that is the code producing >> the data "I don't like". >> >> M3Header isn't dead but I haven't seen it run yet. >> >> I changed them from "=>" to "1=>" and "2=>" and looked >> where those occur in the output .s files under -keep. >> >> This is kinda something I wish were easier, like more strings >> need to be longer for easier finding w/o ambiguity (the flip >> side of my context arguments!) >> >> As things stand, I can't check that in. >> I suppose maybe with a CG.comment but nevermind. >> >> - Jay >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: m3devel at elegosoft.com >>> Subject: RE: [M3devel] source paths to generics? >>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>> >>> mfront/src/misc/M3Header.m3 has this: >>> >>> >>> (* build a synthetic file name & start reading *) >>> filename := old_filename & " => " & filename; >>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>> >>> >>> and instantiated generics aren't what I thought. >>> I thought the build system or compiler did a textual >>> substitution of the generic parameters into the entire implementation, >>> and saved that to the file system, >>> and had the assert/debug paths point to it. >>> >>> So could step through what looks exactly like an .m3 file. >>> >>> But it doesn't. The instantiated file is a little stub, like: >>> >>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>> (*generated by m3build*) >>> >>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>> >>> >>> so I have one or two proposals. >>> >>> 1) old_filename above contains the target, it looks like: >>> >>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>> or >>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>> >>> These both occur. >>> I'm not sure what the second is, seems like a mistake. >>> >>> I suggest the first string in both pairs be changed to be the leaf element, like: >>> >>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>> or >>> "TextAtomTbl.m3 => ../src/table" >>> >>> and maybe the second string in both cases should be as it is in the second pair. >>> >>> 2) Possibly it should reall just be: >>> ../src/table/Table.mg >>> >>> and developer can step through that, knowing that it is a somewhat abstracted >>> form of what is running >>> >>> I'm willing to do this under like: >>> >>> CONST TemporaryForTargetConvergence = TRUE >>> >>> or >>> VAR TemporaryForTargetConvergence: BOOLEAN; >>> >>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>> so might as well remove target unconditionally. >>> >>> >>> ? >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>> Subject: [M3devel] source paths to generics? >>>> >>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>> the source file I am told about is the instantiated i3/m3 file. >>>> >>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>> >>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>> I guess it is slightly easier in the compiler, but easy to do better? >>>> >>>> I should look into make it so? >>>> >>>> My real agenda is I want to see: >>>> ../src/foo.mg >>>> >>>> instead of >>>> I386_DARWIN/foo.m3 >>>> >>>> I don't want the target variation, but other points seem true also, right? >>>> >>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>> >>>> I'll try to poke around more. >>>> >>>> - Jay >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 06:52:30 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:52:30 +0000 Subject: [M3devel] atomic.mg Message-ID: diff --git a/m3-libs/m3core/src/m3makefile b/m3-libs/m3core/src/m3makefile index 7efc97d..e371499 100644 --- a/m3-libs/m3core/src/m3makefile +++ b/m3-libs/m3core/src/m3makefile @@ -52,7 +52,7 @@ include_dir ("main") ?include_dir ("weakref") ?include_dir ("word") ?include_dir ("types") -% include_dir ("atomic")? DISABLE UNTIL I CHECK IN THE COMPILER FRONT-END FIXES -- Tony +include_dir ("atomic") ? ?% m3_option ("-times") ? "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** bad stack:? expected [ Int32 Int32 Addr ? ] got [ Int32 Addr? Addr ? ] "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** non-empty stack: [ Int32]**NIL** might be good to finish this up. hey, that is funny, now I see more of what those strings are for. The "2" in "2=>" is a local edit. ?- Jay From jay.krell at cornell.edu Thu Jun 30 06:58:58 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 04:58:58 +0000 Subject: [M3devel] M3Mid/M3Middle? Message-ID: I need a place to put stuff shared by backend and frontend. I believe the package for that is m3middle. But there is no miscelleanous module. M3Misc? M3Mid? M3Middle? The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. M3Middle ok? None of the existing modules in the package seem right. Thank you, ?- Jay From jay.krell at cornell.edu Thu Jun 30 07:04:05 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:04:05 +0000 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: References: Message-ID: Alternative that I don't really like is: CONST BOOLEAN TempFoo; in multiple places that need to be in sync but alternative for temporary purposes I don't mind is just to put it in Target. ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Thu, 30 Jun 2016 04:58:58 +0000 > Subject: [M3devel] M3Mid/M3Middle? > > I need a place to put stuff shared by backend and frontend. > I believe the package for that is m3middle. > > But there is no miscelleanous module. > > M3Misc? > M3Mid? > M3Middle? > > The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. > > M3Middle ok? > > None of the existing modules in the package seem right. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 07:24:03 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:24:03 +0000 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: , , <5773F2BE.3050902@lcwb.coop>, Message-ID: I think I get it now. I was/am missing some of the lines but I can imagine how it works. There are about 15 bits given to the file number and about 17 bits given to the line number. On a 32bit system. A file with more than 100,000 lines might have trouble and a package with more than 16,000 files might have trouble. I know 64,000 lines in generated .c files is not unheard of. I don't know what file counts are on the high end. It is almost a good place for a LONGINT, but TYPE SourceLocation = RECORD ? file, line: INTEGER or INT32 := 0; END? Ok to do this at some point? ?- Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Date: Wed, 29 Jun 2016 16:31:19 +0000 > Subject: Re: [M3devel] m3front scanner div wierdness? > > My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. > (in either case, the address space was 16 bit and ROM and periphal memory was in there, > so various bank switching employed to gain access; later similar with the 128k RAM machines...) > > > I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine > as you suggest). > > > I know that 32bits is overkill for a line number. > I also know 16 bits isn't overkill -- but more than 16 are used here so ok. > There is/was a warning in the Visual C++ compiler about truncating line numbers > or terminating debug information after 64k lines. Midl output would trigger it. > > I still don't follow completely. > It seems there is an aliasing situation, where lines very far apart can be deemed the same. > > I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. > Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... > In general you have to balance: > 1 System still compilable on 32bit. > 2 vs. 64bit system can do more > > For the first case, you want to limit integers to 32bits and for the second you do not. > > Also convenience and perf suggest *not* having to sprinkle div/mod all around, > though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... > >> FileNoAndLineNo > > Lately this is called "location" and things even have starts and ends, so the error messages can output > a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill > but clang is there and I think gcc went there. Yes the data is larger. > > Maybe shorter FileAndLine? > I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) > > - Jay > > > ---------------------------------------- >> Date: Wed, 29 Jun 2016 11:09:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] m3front scanner div wierdness? >> >> >> >> On 06/28/2016 10:54 PM, Jay K wrote: >>> Does anyone understand this stuff in m3front/Scanner.m3: >>> >>> Here vs. LocalHere? >>> SameFile? >>> >>> >>> I understand only this nuance: >>> offset MOD MaxLines >>> >>> MaxLines = 100000; >>> >>> >>> is to crudely handle that when asserts fail, >>> they pack the line number in with the assertion failure code, >>> potentially loosing bits. >>> >>> >>> I don't think this is a good design, they should just be separate INTEGERS, >>> but this is besides the point. >>> >> >> This is just pure speculation, but I am very confident of it. These >> offsets have a very high occurrence count. There is code all over m3front >> that copies Scanner.offset into various data structures. So the small space >> saving of one INTEGER instead of two would be multiplied by a big number. >> I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. >> Today, I have 8 Gig in the one I bought, and could easily afford more, if I >> thought I needed it. >> >> Two integers would be cleaner, but this design is not too bad *if* you know >> the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one >> field. As pure documentation, there really should be a distinct type name >> (say FileNoAndLineNo?) for all values that use this representation, even >> though it just equates to INTEGER. There are a lot of variables lying around >> all over the front end that use this invariant, but are just declared as >> INTEGER. That's maintainer-hostile. >> >> >>> >>> What doesn't makes sense to me is the machinations around file name. >>> >>> >>> Here: >>> file := files [offset DIV MaxLines]; >>> >>> vs. LocalHere: >>> file := local_files [fnum]; >>> >>> >>> LocalHere makes sense. Here does not. >>> >>> >>> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >>> BEGIN >>> RETURN (a DIV MaxLines) = (b DIV MaxLines); >>> END SameFile; >>> >>> >>> >>> Shouldn't this just be a = b? >>> >> >> As coded, this will return TRUE if a and b are different line numbers within >> the same file. The name "SameFile" at least suggests that is what is intended. >> A good example of a place where it would have been clearer if a & b were >> declared as the type name I proposed above. >> >>> >>> Well, anyway, this SameFile function isn't called. >>> >>> Here and LocalHere are used. >>> >>> >>> I'm looking here because I want to add a temporary measure >>> such that the file names are leaf-only. >>> >>> >>> In particular, because generic modules have target names in their paths >>> and I want to temporarly remove target names from output, so I can prove >>> that a few targets are identical. >>> >>> >>> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >>> This has to percolate quite a bit through the system -- the backends and the runtime. >>> >>> >>> And then this Here vs. LocalHere difference should fall away. >>> But still, what is it trying to do? >>> >>> >>> Thank you, >>> - Jay >>> >>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 07:28:00 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 05:28:00 +0000 Subject: [M3devel] atomic.mg In-Reply-To: <958EAC02-79C4-4051-9144-99AB17CAF0E3@anu.edu.au> References: , <958EAC02-79C4-4051-9144-99AB17CAF0E3@anu.edu.au> Message-ID: If you do get time, one of my top requests is cooperative suspend. This will have a few benefits: ? - remove a bunch of system-specific code? ? - improve portability ? ? - make NT386-on-amd64 much more trustworthy ?(SuspendThread + GetThreadContext are wonky there)? ? - make PPC_DARWIN on x86 DARWIN probably work where today it always fails (granted, obsolete system and never remotely worked) ?? ? - not break into debugger due to the signals ?? On the other hand, I realize the primitives we are after are "SuspendThread" and "GetThreadContext" and they might exist widely. I didn't realize what was being attempted before. I might also get time in the next few months for a bunch of stuff. :) ? - Jay ---------------------------------------- > Subject: Re: atomic.mg > From: antony.hosking at anu.edu.au > Date: Thu, 30 Jun 2016 15:11:03 +1000 > CC: m3devel at elegosoft.com > To: jay.krell at cornell.edu > > I may actually have some time for this in the next few months! > > Tony Hosking, Professor > Research School of Computer Science > The Australian National University > >> On 30 Jun 2016, at 2:52 PM, Jay K wrote: >> >> diff --git a/m3-libs/m3core/src/m3makefile b/m3-libs/m3core/src/m3makefile >> index 7efc97d..e371499 100644 >> --- a/m3-libs/m3core/src/m3makefile >> +++ b/m3-libs/m3core/src/m3makefile >> @@ -52,7 +52,7 @@ include_dir ("main") >> include_dir ("weakref") >> include_dir ("word") >> include_dir ("types") >> -% include_dir ("atomic") DISABLE UNTIL I CHECK IN THE COMPILER FRONT-END FIXES -- Tony >> +include_dir ("atomic") >> >> % m3_option ("-times") >> >> >> "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** bad stack: expected [ Int32 Int32 Addr ] got [ Int32 Addr Addr ] >> "../I386_DARWIN/AtomicWideChar.m3 2=> ../src/atomic/Atomic.mg", line 53: ********* M3CG_Check ERROR *********** non-empty stack: [ Int32]**NIL** >> >> >> might be good to finish this up. >> >> hey, that is funny, now I see more of what those strings are for. >> >> The "2" in "2=>" is a local edit. >> >> - Jay > From rodney_bates at lcwb.coop Thu Jun 30 17:53:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 10:53:34 -0500 Subject: [M3devel] source paths to generics? In-Reply-To: References: , , , , <57746905.2080007@lcwb.coop> Message-ID: <5775407E.6000401@lcwb.coop> On 06/29/2016 11:49 PM, Jay K wrote: > I guess neither of us looked outside of m3front. > The code runs. Not clear if the strings can get output. > Maybe from asserts or Compiler.ThisFile in a generic? > I"ll try some more. > I started looking in all of m3-sys, since the different compiler packages there are linked together into cm3. Later, I looked in the entire cm3 repo and didn't see anything. In any case, since it's part of the compiler, it really would be strange to call it from outside m3-sys. But if it is executing, we must have missed something. How did you find it executing? In a debugger? Can you do a backtrace? > jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 > ids := M3Front.ParseImports (source, s.m3env); > > > - Jay > > > ---------------------------------------- >> Date: Wed, 29 Jun 2016 19:34:13 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] source paths to generics? >> >> Hmm. I poked around a bit, and it looks like there is some cruft here, >> left over from something. >> >> The only call I can find on M3Header.Parse is M3Front.m3:37. This is >> in M3Front.ParseImports. I can find no calls on that at all. (There >> are other procedures by this name.) >> >> Also, it appears M3Header.ParseImports actually collects the exports, the imports, >> the generic actuals. >> >> As for the concocted (by M3Header) name of the form => , >> this looks to be used only to initialize the scanner's file number while scanning >> up through the formals of the generic, but that is not used. >> >> On 06/29/2016 11:34 AM, Jay K wrote: >>> There is similar code in Module.m3 and that is the code producing >>> the data "I don't like". >>> >>> M3Header isn't dead but I haven't seen it run yet. >>> >>> I changed them from "=>" to "1=>" and "2=>" and looked >>> where those occur in the output .s files under -keep. >>> >>> This is kinda something I wish were easier, like more strings >>> need to be longer for easier finding w/o ambiguity (the flip >>> side of my context arguments!) >>> >>> As things stand, I can't check that in. >>> I suppose maybe with a CG.comment but nevermind. >>> >>> - Jay >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: m3devel at elegosoft.com >>>> Subject: RE: [M3devel] source paths to generics? >>>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>>> >>>> mfront/src/misc/M3Header.m3 has this: >>>> >>>> >>>> (* build a synthetic file name & start reading *) >>>> filename := old_filename & " => " & filename; >>>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>>> >>>> >>>> and instantiated generics aren't what I thought. >>>> I thought the build system or compiler did a textual >>>> substitution of the generic parameters into the entire implementation, >>>> and saved that to the file system, >>>> and had the assert/debug paths point to it. >>>> >>>> So could step through what looks exactly like an .m3 file. >>>> >>>> But it doesn't. The instantiated file is a little stub, like: >>>> >>>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>>> (*generated by m3build*) >>>> >>>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>>> >>>> >>>> so I have one or two proposals. >>>> >>>> 1) old_filename above contains the target, it looks like: >>>> >>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>>> or >>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>>> >>>> These both occur. >>>> I'm not sure what the second is, seems like a mistake. >>>> >>>> I suggest the first string in both pairs be changed to be the leaf element, like: >>>> >>>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>>> or >>>> "TextAtomTbl.m3 => ../src/table" >>>> >>>> and maybe the second string in both cases should be as it is in the second pair. >>>> >>>> 2) Possibly it should reall just be: >>>> ../src/table/Table.mg >>>> >>>> and developer can step through that, knowing that it is a somewhat abstracted >>>> form of what is running >>>> >>>> I'm willing to do this under like: >>>> >>>> CONST TemporaryForTargetConvergence = TRUE >>>> >>>> or >>>> VAR TemporaryForTargetConvergence: BOOLEAN; >>>> >>>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>>> so might as well remove target unconditionally. >>>> >>>> >>>> ? >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: m3devel at elegosoft.com >>>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>>> Subject: [M3devel] source paths to generics? >>>>> >>>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>>> the source file I am told about is the instantiated i3/m3 file. >>>>> >>>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>>> >>>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>>> I guess it is slightly easier in the compiler, but easy to do better? >>>>> >>>>> I should look into make it so? >>>>> >>>>> My real agenda is I want to see: >>>>> ../src/foo.mg >>>>> >>>>> instead of >>>>> I386_DARWIN/foo.m3 >>>>> >>>>> I don't want the target variation, but other points seem true also, right? >>>>> >>>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>>> >>>>> I'll try to poke around more. >>>>> >>>>> - Jay >>>>> >>>>> >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Thu Jun 30 17:58:24 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 10:58:24 -0500 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: References: Message-ID: <577541A0.8050109@lcwb.coop> The compiler's being broken into several packages sometimes makes it hard to get info from where it is known to where it is needed. I did something kludgy about size of WIDECHAR. I don't remember details right now. M3Middle sounds OK. Is m3middle the lowest package in the compiler? On 06/29/2016 11:58 PM, Jay K wrote: > I need a place to put stuff shared by backend and frontend. > I believe the package for that is m3middle. > > But there is no miscelleanous module. > > M3Misc? > M3Mid? > M3Middle? > > The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. > > M3Middle ok? > > None of the existing modules in the package seem right. > > Thank you, > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 18:10:05 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:10:05 +0000 Subject: [M3devel] M3Mid/M3Middle? In-Reply-To: <577541A0.8050109@lcwb.coop> References: , <577541A0.8050109@lcwb.coop> Message-ID: My understand is not that it is the "lowest", but that it is shared between m3front and m3back. I share your frustration, but I also suspect the design is fairly sound. I'll send a larger proposal shortly. ?- Jay ---------------------------------------- > Date: Thu, 30 Jun 2016 10:58:24 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] M3Mid/M3Middle? > > The compiler's being broken into several packages sometimes makes > it hard to get info from where it is known to where it is needed. > I did something kludgy about size of WIDECHAR. I don't remember > details right now. M3Middle sounds OK. Is m3middle the lowest > package in the compiler? > > On 06/29/2016 11:58 PM, Jay K wrote: >> I need a place to put stuff shared by backend and frontend. >> I believe the package for that is m3middle. >> >> But there is no miscelleanous module. >> >> M3Misc? >> M3Mid? >> M3Middle? >> >> The initial use is only temporary -- a boolean to indicate to remove target from some strings that don't have a strict definition -- like those generic source paths. >> >> M3Middle ok? >> >> None of the existing modules in the package seem right. >> >> Thank you, >> - Jay >> >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jay.krell at cornell.edu Thu Jun 30 18:11:39 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:11:39 +0000 Subject: [M3devel] source paths to generics? In-Reply-To: <5775407E.6000401@lcwb.coop> References: , , , , , , , <57746905.2080007@lcwb.coop> ,<5775407E.6000401@lcwb.coop> Message-ID: I put in a bunch of calls to RTIO.PutText("I'm here 1"); RTIO.Flush(), RTIO.PutText("I'm here 2"); RTIO.Flush(). When I saw those I knew I missed it somehow and looked again. I still don't see the strings in the output though. ?- Jay ---------------------------------------- > Date: Thu, 30 Jun 2016 10:53:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] source paths to generics? > > > > On 06/29/2016 11:49 PM, Jay K wrote: >> I guess neither of us looked outside of m3front. >> The code runs. Not clear if the strings can get output. >> Maybe from asserts or Compiler.ThisFile in a generic? >> I"ll try some more. >> > > I started looking in all of m3-sys, since the different compiler > packages there are linked together into cm3. Later, I > looked in the entire cm3 repo and didn't see anything. > In any case, since it's part of the compiler, it really would be > strange to call it from outside m3-sys. > > But if it is executing, we must have missed something. How did you > find it executing? In a debugger? Can you do a backtrace? > >> jair:m3core jay$ grep -i parseimp ../../m3-sys/cm3/src/Builder.m3 >> ids := M3Front.ParseImports (source, s.m3env); >> >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Wed, 29 Jun 2016 19:34:13 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] source paths to generics? >>> >>> Hmm. I poked around a bit, and it looks like there is some cruft here, >>> left over from something. >>> >>> The only call I can find on M3Header.Parse is M3Front.m3:37. This is >>> in M3Front.ParseImports. I can find no calls on that at all. (There >>> are other procedures by this name.) >>> >>> Also, it appears M3Header.ParseImports actually collects the exports, the imports, >>> the generic actuals. >>> >>> As for the concocted (by M3Header) name of the form => , >>> this looks to be used only to initialize the scanner's file number while scanning >>> up through the formals of the generic, but that is not used. >>> >>> On 06/29/2016 11:34 AM, Jay K wrote: >>>> There is similar code in Module.m3 and that is the code producing >>>> the data "I don't like". >>>> >>>> M3Header isn't dead but I haven't seen it run yet. >>>> >>>> I changed them from "=>" to "1=>" and "2=>" and looked >>>> where those occur in the output .s files under -keep. >>>> >>>> This is kinda something I wish were easier, like more strings >>>> need to be longer for easier finding w/o ambiguity (the flip >>>> side of my context arguments!) >>>> >>>> As things stand, I can't check that in. >>>> I suppose maybe with a CG.comment but nevermind. >>>> >>>> - Jay >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> From: jay.krell at cornell.edu >>>>> To: m3devel at elegosoft.com >>>>> Subject: RE: [M3devel] source paths to generics? >>>>> Date: Wed, 29 Jun 2016 15:22:14 +0000 >>>>> >>>>> mfront/src/misc/M3Header.m3 has this: >>>>> >>>>> >>>>> (* build a synthetic file name & start reading *) >>>>> filename := old_filename & " => " & filename; >>>>> Scanner.Push (filename, s.generic, is_main := Scanner.in_main); >>>>> >>>>> >>>>> and instantiated generics aren't what I thought. >>>>> I thought the build system or compiler did a textual >>>>> substitution of the generic parameters into the entire implementation, >>>>> and saved that to the file system, >>>>> and had the assert/debug paths point to it. >>>>> >>>>> So could step through what looks exactly like an .m3 file. >>>>> >>>>> But it doesn't. The instantiated file is a little stub, like: >>>>> >>>>> jair:libm3 jay$ more I386_DARWIN/TextAtomTbl.m3 >>>>> (*generated by m3build*) >>>>> >>>>> MODULE TextAtomTbl = Table (Text, Atom) END TextAtomTbl. >>>>> >>>>> >>>>> so I have one or two proposals. >>>>> >>>>> 1) old_filename above contains the target, it looks like: >>>>> >>>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table/Table.mg" >>>>> or >>>>> "../I386_DARWIN/TextAtomTbl.m3 => ../src/table" >>>>> >>>>> These both occur. >>>>> I'm not sure what the second is, seems like a mistake. >>>>> >>>>> I suggest the first string in both pairs be changed to be the leaf element, like: >>>>> >>>>> "TextAtomTbl.m3 => ../src/table/Table.mg" >>>>> or >>>>> "TextAtomTbl.m3 => ../src/table" >>>>> >>>>> and maybe the second string in both cases should be as it is in the second pair. >>>>> >>>>> 2) Possibly it should reall just be: >>>>> ../src/table/Table.mg >>>>> >>>>> and developer can step through that, knowing that it is a somewhat abstracted >>>>> form of what is running >>>>> >>>>> I'm willing to do this under like: >>>>> >>>>> CONST TemporaryForTargetConvergence = TRUE >>>>> >>>>> or >>>>> VAR TemporaryForTargetConvergence: BOOLEAN; >>>>> >>>>> and a command line switch, but I suspect it isn't really useful to anyone asis, >>>>> so might as well remove target unconditionally. >>>>> >>>>> >>>>> ? >>>>> >>>>> - Jay >>>>> >>>>> >>>>> >>>>> ---------------------------------------- >>>>>> From: jay.krell at cornell.edu >>>>>> To: m3devel at elegosoft.com >>>>>> Date: Wed, 29 Jun 2016 09:44:51 +0000 >>>>>> Subject: [M3devel] source paths to generics? >>>>>> >>>>>> I haven't fully verified this, but it appears for example if I debug a generic, or fail an assert in a generic, >>>>>> the source file I am told about is the instantiated i3/m3 file. >>>>>> >>>>>> This isn't particularly useful for the programmer or convenient for the compiler, right? >>>>>> >>>>>> The programmer would rather see the .ig/.mg files, and this is easy to provide in the compiler? >>>>>> I guess it is slightly easier in the compiler, but easy to do better? >>>>>> >>>>>> I should look into make it so? >>>>>> >>>>>> My real agenda is I want to see: >>>>>> ../src/foo.mg >>>>>> >>>>>> instead of >>>>>> I386_DARWIN/foo.m3 >>>>>> >>>>>> I don't want the target variation, but other points seem true also, right? >>>>>> >>>>>> Right? The line numbers match between them, and the generic syntax is so close to "normal" that a programmer not used to it won't be confused, right? >>>>>> >>>>>> I'll try to poke around more. >>>>>> >>>>>> - Jay >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> M3devel mailing list >>>>>> M3devel at elegosoft.com >>>>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Thu Jun 30 18:12:11 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 Jun 2016 11:12:11 -0500 Subject: [M3devel] m3front scanner div wierdness? In-Reply-To: References: , , <5773F2BE.3050902@lcwb.coop>, Message-ID: <577544DB.7050900@lcwb.coop> On 06/30/2016 12:24 AM, Jay K wrote: > I think I get it now. I was/am missing some of the lines but I can imagine how it works. > > There are about 15 bits given to the file number and about 17 bits given to the line number. > On a 32bit system. > > A file with more than 100,000 lines might have trouble and a package with more than 16,000 files might have trouble. > I know 64,000 lines in generated .c files is not unheard of. > I don't know what file counts are on the high end. > I think 100,000 lines could be a bit marginal for mechanically generated code, but the file count space is probably over generous. I presume they made it a power of 10 so a human could do the DIV or MOD visually on the decimal value. Increasing to a million would still give 2147 files. > It is almost a good place for a LONGINT, but > TYPE SourceLocation = RECORD > file, line: INTEGER or INT32 := 0; > END? > > Ok to do this at some point? It will be pervasive and truly algorithmic. An alternative would be to keep it an INTEGER, but use a distinct type name on all variables/fields that use the MOD/DIV invariant. Pervasive too, but no risk of runtime breakage. If you put just the one INTEGER inside a record with an unlikely name, that would preserve the space savings but still get type checking and still get the compiler to find all the places that need to change. > > - Jay > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Date: Wed, 29 Jun 2016 16:31:19 +0000 >> Subject: Re: [M3devel] m3front scanner div wierdness? >> >> My first computer, at home, had 80k, and it was oddly high end, most people had 48k or 64k. >> (in either case, the address space was 16 bit and ROM and periphal memory was in there, >> so various bank switching employed to gain access; later similar with the 128k RAM machines...) >> >> >> I won't just change one place and break all the others, but maybe we should try to split it everywhere as you suggest (and recombine >> as you suggest). >> >> >> I know that 32bits is overkill for a line number. >> I also know 16 bits isn't overkill -- but more than 16 are used here so ok. >> There is/was a warning in the Visual C++ compiler about truncating line numbers >> or terminating debug information after 64k lines. Midl output would trigger it. >> >> I still don't follow completely. >> It seems there is an aliasing situation, where lines very far apart can be deemed the same. >> >> I'll look closer though, as e.g. 20 bits would seem enough for a line number and 20 bits for a file number. >> Or maybe the answer is 32 bits in general, and the 64bit machines can move the pair together... >> In general you have to balance: >> 1 System still compilable on 32bit. >> 2 vs. 64bit system can do more >> >> For the first case, you want to limit integers to 32bits and for the second you do not. >> >> Also convenience and perf suggest *not* having to sprinkle div/mod all around, >> though granted, div/mod by a constant is emininently optimizable, at least 32 bit operations... >> >>> FileNoAndLineNo >> >> Lately this is called "location" and things even have starts and ends, so the error messages can output >> a line and then point out the parts of the line. I'm not sure if this is obviously good and nice or overkill >> but clang is there and I think gcc went there. Yes the data is larger. >> >> Maybe shorter FileAndLine? >> I realize it is ambiguous, they could both be strings, or line could be a file offset (a useful quantity!) >> >> - Jay >> >> >> ---------------------------------------- >>> Date: Wed, 29 Jun 2016 11:09:34 -0500 >>> From: rodney_bates at lcwb.coop >>> To: m3devel at elegosoft.com >>> Subject: Re: [M3devel] m3front scanner div wierdness? >>> >>> >>> >>> On 06/28/2016 10:54 PM, Jay K wrote: >>>> Does anyone understand this stuff in m3front/Scanner.m3: >>>> >>>> Here vs. LocalHere? >>>> SameFile? >>>> >>>> >>>> I understand only this nuance: >>>> offset MOD MaxLines >>>> >>>> MaxLines = 100000; >>>> >>>> >>>> is to crudely handle that when asserts fail, >>>> they pack the line number in with the assertion failure code, >>>> potentially loosing bits. >>>> >>>> >>>> I don't think this is a good design, they should just be separate INTEGERS, >>>> but this is besides the point. >>>> >>> >>> This is just pure speculation, but I am very confident of it. These >>> offsets have a very high occurrence count. There is code all over m3front >>> that copies Scanner.offset into various data structures. So the small space >>> saving of one INTEGER instead of two would be multiplied by a big number. >>> I remember working in Modula-3 on a company-paid computer with 16 Meg of ram. >>> Today, I have 8 Gig in the one I bought, and could easily afford more, if I >>> thought I needed it. >>> >>> Two integers would be cleaner, but this design is not too bad *if* you know >>> the MOD/DIV invariant. It is commented at Scanner.m3:54, but only for one >>> field. As pure documentation, there really should be a distinct type name >>> (say FileNoAndLineNo?) for all values that use this representation, even >>> though it just equates to INTEGER. There are a lot of variables lying around >>> all over the front end that use this invariant, but are just declared as >>> INTEGER. That's maintainer-hostile. >>> >>> >>>> >>>> What doesn't makes sense to me is the machinations around file name. >>>> >>>> >>>> Here: >>>> file := files [offset DIV MaxLines]; >>>> >>>> vs. LocalHere: >>>> file := local_files [fnum]; >>>> >>>> >>>> LocalHere makes sense. Here does not. >>>> >>>> >>>> PROCEDURE SameFile (a, b: INTEGER): BOOLEAN = >>>> BEGIN >>>> RETURN (a DIV MaxLines) = (b DIV MaxLines); >>>> END SameFile; >>>> >>>> >>>> >>>> Shouldn't this just be a = b? >>>> >>> >>> As coded, this will return TRUE if a and b are different line numbers within >>> the same file. The name "SameFile" at least suggests that is what is intended. >>> A good example of a place where it would have been clearer if a & b were >>> declared as the type name I proposed above. >>> >>>> >>>> Well, anyway, this SameFile function isn't called. >>>> >>>> Here and LocalHere are used. >>>> >>>> >>>> I'm looking here because I want to add a temporary measure >>>> such that the file names are leaf-only. >>>> >>>> >>>> In particular, because generic modules have target names in their paths >>>> and I want to temporarly remove target names from output, so I can prove >>>> that a few targets are identical. >>>> >>>> >>>> I guess, really, I propose the interface to assertion failures be expanded to take the line number separate from the failure code. >>>> This has to percolate quite a bit through the system -- the backends and the runtime. >>>> >>>> >>>> And then this Here vs. LocalHere difference should fall away. >>>> But still, what is it trying to do? >>>> >>>> >>>> Thank you, >>>> - Jay >>>> >>>> >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >>>> >>> >>> -- >>> Rodney Bates >>> rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Jun 30 18:17:22 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 30 Jun 2016 16:17:22 +0000 Subject: [M3devel] removing occurences of target name output? Message-ID: Ok here is are some choices/proposal. ?- Do nothing -- or at least commit nothing. ?I have to do something -- my goal is to show that various ?targets are the same, with the C backend and possibly cm3cg. ? ?- add switch cm3 -trim-paths ?confusing name? ? ?add swich cm3 -omit-target confusing name? ? ?Whatever is this switch, add it to cm3g also. The same name. ? ?Add switch to scripts/python/pylib.py -cm3:xxx ? The xxx part is passed to cm3. ? ?e.g. do-pk.py -cm3:-omit-target ? ? ?The behavior of the switch: ? in cm3, it is passed to the backend somehow.? ? ? ?Either as a flag to the quake function, or ? ? ?by setting a quake global; somehow shield ? ? ?the LLVM backend.? ? ? ? ? Bring back the function m3front/Host.Tail. ? This historically finds the last forward slash or ? backward slash or space in a string, and returns everything ? after it. ? Modify it to check this new flag, and if the new flag isn't set, ? return the path unchanged. ?? ?? ? In m3cc, dbxout.c and dwarf2out.c, there is one place ? each that outputs the current working directory. ? If this switch is set, don't do that. ?? ?? ? In m3front/M3Header.m3 and Scanner.m3 where it makes up those ? strings with "=>" in them, pass the left hand side through ? Host.Tail. ?? ? In m3front Scanner.Here, if the flag is set, return ? the value of LocalHere instead. I think. I think that is ? required to remove more occurences. I can double check. ?? ?? ? In m3back/M3C.c there is one place that outputs the ? target in a comment. Either just remove that unconditionally, or ? remove it subject to this switch. ?? ?? ? That last point, and the actual goal, is why "trim-path" ? isn't an accurate name. It does so happen that "paths" ? are usually where target occurs in the output semi-unnecessarily. ?? ?? ? It could very well be that we do want this behavior in cm3cg long term, ? such that whoever makes a distribution, at whatever path, gets the same result. ?? ?I do not want to go the route of omitting debug info entirely, which also fixes this. ?? ? It would be nice if something could be specified symbolically. ? Such as wherever the expanded form of $CM3_TARGET occurs in a path ? in the output, change it back to $CM3_TARGET. Like how the ? .m3ship files are optionally generated and like I've seen ? in other compilers -- there is a goal in general to have ? some sort of ull path, but not require everyone to build ? with the same paths, and still get identical outputs. ?This somewhat requires debugger cooperation, depending on their source search algorithm. ? ?? ? When cm3 parses the command line, store this boolean in... Target.OmitPathss? ? Target.TrimTargetString? M3Middle.LessTargetDependentOutput?? ? M3Middle.TrimTarget? ?? ?? ? Obviously this is a minor tactic -- sizeof(integer) will for now ? continue to be endemic in the IR and the backend output. ?? ? The actual goal here is not to make one C output, but a small number. ? I intend soon to introduce the following targets, something like: ? ? POSIX32LE ? ? ? POSIX32BE ? ? ? POSIX64LE ? ? ? POSIX64BE ? ? ? WIN32LE? ? ? WIN64LE? ?C or C++ backend is implied, by lack of processor.? ? ? ? ?Or maybe: ? ? C32EL_POSIX ? ? C64EL_POSIX ? ? C32BE_POSIX ? ? C64BE_POSIX ? ? C32EL_WIN32 ? ? C64EL_WIN32 These 6 targets shall be the basis of a "universal" download/bootstrap, easier for people to get started with. ? ?Eventually the output will use C++ exception handling so I'm leary ?of backing the name "C" in too much. ? ?But I'm getting ahead of myself. ? ?This "trim target" is an important validation step for that. ? ?Ok in principle? ?Ok in detail? ?Or just do nothing? ? ?- Jay From jayk123 at hotmail.com Wed Jun 1 11:28:08 2016 From: jayk123 at hotmail.com (Jay K) Date: Wed, 1 Jun 2016 09:28:08 +0000 Subject: [M3devel] [modula3/cm3] New release for Mac OS X, please? (#13) [Originally From]: notifications@github.com In-Reply-To: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> References: <76684c92a94845068c63876abe971df8@sf-e2013-02.exchange.cornell.edu> Message-ID: I kinda want to encourage ?a different approach, at least for Unix systems where binary compatibility is currently so low. So here is what I did/am doing: ?I have made assembly and C bootstraps for I386_LINUX and AMD64_LINUX.? ?I'm still testing them (waiting for boot2 to finish). ? ? ?Here is roughly how:? ? For assembly (gcc-bassed ) bootstrap:? ?cd ${CM3_ROOT}/m3-sys/m3cc? ? ?sh -c ./src/buildmany.sh? ? That actually does way more than necessary,? ? and might fail in the end. I didn't wait for it.? ? In particular, it builds a gcc-based backend for every target.? ?If you look -- buildmany.sh is trivial.? ?m3cc/src/m3makefile has always had the support. buildmany.sh is a tiny wrapper.? ? Actually the first step should be redundant with boot1.py anyway. ? ? cd ${CM3_ROOT}/scripts/python? ?# I don't think these are case sensitive even.? ?./do-cm3-all.py I386_LINUX realclean? ?./do-cm3-all.py I386_LINUX c realclean? ?./do-cm3-all.py AMD64_LINUX realclean? ?./do-cm3-all.py AMD64_LINUX c realclean? ?./boot1.py I386_LINUX c? ?./boot1.py I386_LINUX ? ?./boot1.py AMD64_LINUX c? ?./boot1.py AMD64_LINUX ? If you look, boot1.py doesn't seem obviously simple, but?it really is just a wrapper around roughly: ?cd ? ?cm3? ?mkdir? ?cp ? ?tar? ? ? ?"All else being equal", i.e. lacking broad adoption of the C or LLVM backends, ?we should make releases "this way" "and then some". ? ? ?"and then some": ? ?all supported/tested targets (Darwin, Solaris, FreeBSD, maybe NT). Enhancing it to include the entire system, installing a proper install, and supporting shared libraries. Shared libraries: either autotools or cmake or replicating the config files ? i.e. into pylib.py, or moving pylib.py logic into cm3, esp. cm3 -boot ?? Key advantages of this approach: ?1) It is cross building, we can do it all on one machine (or scale it out, but ? ? it doesn't take too long)? ?2) More importantly, the results are less machine-specific. There are ? ? no paths to dynamic linked libraries or versions thereof. This is roughly how 3.6 was released -- along with a matching source tree for the entire system and good directions to build the multiple pieces. ?The directions today are like:? ? get the right bootstrap archive ? get the source tree ? extract bootstrap ? build it (make) (again, if you look, simple stuff)? ? install it ? ? mkdir /somewhere/bin? ? ? mv cm3 /somewhere/bin ? ? ? export PATH=/somewhere/bin:$PATH ? ? ? ./boot2.py ? ? ? or ? ? ?./boot2.py c to use the C backend ? ? ? ? ? ? ? ?(mklib is only for NT targets)? ?? ?? I'd really soon like there to be fewer C variations though -- just unixc32le, unixc32be, unixc64le, posix64be, ntc32le, ntc64le. And this is almost overkill -- 64le is the vast majority. ? - Jay ---------------------------------------- Date: Tue, 31 May 2016 08:28:32 -0700 From: notifications at github.com To: cm3 at noreply.github.com CC: jay.krell at cornell.edu; comment at noreply.github.com Subject: Re: [modula3/cm3] New release for Mac OS X, please? (#13) Yes, I saw the min and all tar.gz files yesterday. I extracted the "all" tar file and the compiler worked fine. (The tar.gz files are gone from my /var directory now.) I'd be happy to help with a release. What should I do? From jay.krell at cornell.edu Wed Jun 1 11:30:16 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 09:30:16 +0000 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? Does/can it retain its incrementality? Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? - Jay ---------------------------------------- > From: jay.krell at cornell.edu > To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es > Date: Tue, 31 May 2016 17:18:46 +0000 > Subject: Re: [M3devel] replace build system?? > > We should probably learn how to get the number of processors and optionally system load and make this more automatic. > Posix should standardize more of this. :( > > - Jay > > > > > > > ---------------------------------------- >> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >> To: dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 10:16:05 -0700 >> From: mika at async.caltech.edu >> Subject: Re: [M3devel] replace build system?? >> >> I mentioned this before a few times on this mailing list.... CM3 is already fairly >> parallel if you turn on the right options. Every back-end invocation can be done >> in parallel. >> >> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >> >> "Daniel Alejandro Benavides D." writes: >> ... >>>A make-based build solution would have other advantages as well like=20 >>>f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>four times faster. >> ... >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Wed Jun 1 13:17:43 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 1 Jun 2016 11:17:43 +0000 Subject: [M3devel] IR init_int with out of range values? Message-ID: Is this reasonable? m3-libs/m3core/src/float/IEEE-be/LongRealRep.i3 INTERFACE LongRealRep; (* This interface describes the layout of IEEE double precision reals ? ?on big endian machines *) TYPE ? T = RECORD? ? ? sign ? ? ? ? : BITS ?1 FOR [0..1] ? ? ? ? ? ? ? ? ? ? ? ? ?:= 0; ? ? exponent ? ? : BITS 11 FOR [0..16_7FF] ? ? ? ? ? ? ? ? ? ? := 0; ? ? significand0 : BITS 20 FOR [0..16_FFFFF] ? ? ? ? ? ? ? ? ? := 0; ? ? significand1 : BITS 32 FOR [-16_7fffffff-1 .. 16_7fffffff] := 0; ? END; CONST ? NegInf = T { sign := 1, exponent := 16_7FF }; ? PosInf = T { sign := 0, exponent := 16_7FF }; ? Nan ? ?= T { sign := 0, exponent := 16_7FF, significand0 := 1 }; END LongRealRep. begin_init v.1 # constant NegInf # constant PosInf init_int 0 18442240474082181120 Int.64 # constant Nan init_int 8 9218868437227405312 Int.64 My C backend doesn't like 18442240474082181120 Int.64, since 18442240474082181120 does not fit in a signed 64bit integer. Should the frontend be obligated to make it Word.64? I guess I can make it work. jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ edit 1.c jair:python jay$ cat 1.c #include int main() { long long a = 18442240474082181120ll; printf("%llu %llx %lld %llx\n", a, a, a, a); return 0; } jair:python jay$ cc 1.c 1.c:5:15: warning: integer literal is too large to be represented in a signed integer type, interpreting as unsigned ? ? ? [-Wimplicitly-unsigned-literal] long long a = 18442240474082181120ll; ? ? ? ? ? ? ? ^ 1 warning generated. jair:python jay$ ./a.out 18442240474082181120 fff0000000000000 -4503599627370496 fff0000000000000 Thank you, ?- Jay From rodney_bates at lcwb.coop Thu Jun 2 00:08:34 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 01 Jun 2016 17:08:34 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , <20160531171605.13AA61A2068@async.async.caltech.edu>, Message-ID: <574F5CE2.8080506@lcwb.coop> On 06/01/2016 04:30 AM, Jay K wrote: > We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. It doesn't, not very well. A deep principle, if you can call it that, of C is that each compilation has no knowledge of any other compilation. Likewise, when linking them together, it's all just binary stuff, addresses, constants, etc. To get a semblance of type-safe compilation, all coders have to correctly follow some variant of a header file/#include idiom and create a matching makefile that correctly reflects the actual interdependencies. There is no checking that these are done correctly, and from direct experience, I can say they are routinely done wrong. All the modular languages have direct linguistic support for type-safe separate compilation. Implementing this correctly while using the C compilation model lies somewhere between very kludgy and impossible. > > Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? It couldn't possibly. Typical front end compilation actual work is similar to or less than executable program startup/shutdown overhead. This effect might be partially lost in the fact that m3cc is a separate executable and it has to start up for every .[im]3 file. But it would still about double the number of program executions and startup overheads. > Does/can it retain its incrementality? I don't think this is possible all. No matter what is in the makefile, Make only understands dependencies among whole source files and must rebuild if a depended-on file has been touched, even no changes to content. Our current build system works on declaration granularity. (I presume this is what you mean by incrementality." I am not aware of any language-independent build infrastructure that keeps track of dependencies on other that source file granularity. > > Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler frontend, and language-specific link-time functions in one executable. I see no benefit and almost certainly losses, in putting these in different executables. > - Jay > > > > > ---------------------------------------- >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >> Date: Tue, 31 May 2016 17:18:46 +0000 >> Subject: Re: [M3devel] replace build system?? >> >> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >> Posix should standardize more of this. :( >> >> - Jay >> >> >> >> >> >> >> ---------------------------------------- >>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>> To: dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 10:16:05 -0700 >>> From: mika at async.caltech.edu >>> Subject: Re: [M3devel] replace build system?? >>> >>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>> parallel if you turn on the right options. Every back-end invocation can be done >>> in parallel. >>> >>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>> >>> "Daniel Alejandro Benavides D." writes: >>> ... >>>> A make-based build solution would have other advantages as well like=20 >>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>> four times faster. >>> ... >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From microcode at zoho.com Thu Jun 2 08:08:47 2016 From: microcode at zoho.com (microcode at zoho.com) Date: Thu, 2 Jun 2016 06:08:47 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com> <20160531171605.13AA61A2068@async.async.caltech.edu> <574F5CE2.8080506@lcwb.coop> Message-ID: <20160602060847.GA3051@zoho.com> On Wed, Jun 01, 2016 at 05:08:34PM -0500, Rodney M. Bates wrote: > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. Agreed with everything you wrote not withstanding my total lack of M3 knowledge. Ada is another language (possibly the first?) that specifies a lot of how it needs to be built in the language spec. It deals with the issues you mentioned (and more I think) like checking parameters and types across the system, interface compliance, incremental compilation etc. There is an open source version available so it should be possible to at least get a look how they are accomplishing those things and see if it could be scrounged, adapted etc. for M3. I would think they're very Ada-centric as these kinds of things seem to always turn out to be, but who knows. The Ada compiler is part of gcc, it's called gnat or gcc-ada. The specific top-level piece that is responsible for figuring out all the dependencies is called gnatmake. From jay.krell at cornell.edu Fri Jun 3 09:22:45 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 3 Jun 2016 07:22:45 +0000 Subject: [M3devel] replace build system?? In-Reply-To: <574F5CE2.8080506@lcwb.coop> References: , ,,, ,,<1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: I understand the C model very well, but I'm not certain of all the ramifications for Modula-3. I understand C++ has limited checking via name mangling. And that other systems carry names/signatures forward to runtime and check them there. Even dynamically linked C++. And Modula-3. However function names aren't everything -- i.e. they don't include structural hashes of types. It is name-based, not structure-based. The most common mistake in the loose C systems is not doing things wrong per se, but accidentally mixing old and new versions, and the result is undefined/crash. We have the same problem with our "extern" mechanism -- how many times I debugged stat corrupting stack until I put in the extra layer. C++ systems are getting better and check for "ODR violations" at link time. But checking at link time kinda contradicts my question. Goal being to fit into "generic" build systems but C++ only starts winning when the linker knows about it -- when the system is C++ specific. If C++ requires C++ linker/driver, we shouldn't feel too bad requiring a Modula-3 linker/driver. ?> Typical front end compilation actual work is similar to? ?> or less than executable program startup/shutdown overhead.? ?Ideally this is not the case. Ideally process startup is just referencing ?pages already in file system cache. Demand paging, unified file system cache ?and memory manager and all that (all systems NT, Linux, etc. are the same ?these days). But I realize in reality there is alway some amount of other CPU ?to burn at each process start. The ideal is not quite reached. ?> m3cc is a separate executable..has to start up for every .[im]3 file. B ? ?I wonder if we can fix this btw. ?If we can start it just once per library. ?Process create really is fast on typical Posix systems, so people maybe ?don't notice. ? ?There are a few methods: ? ?- alway keep the m3cg ir files ? ?- if any out of date, read them all into one large IR and write out one large assembly file ? ?- I want do that anyway, to achieve a rich/poor man's LTCG/LTO. ? ? ?poor: No need to worry about linker/driver support.? ? ? ?rich: We can afford the memory these days. Though at some point it wouldn't scale.? ? - I want do similar for C backend, as it seems noticable either the compiler startup ? ? or the few #includes I sometimes use (though C++ modules promise some relief..) ? ?? ? ? Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them? ? separately as usual, but within one m3cg invocation.? ?? ?> Does/can it retain its incrementality? ? ?I think the best but not great we could do is make would decide pessmisically ?to run the compiler, but the compiler could figure out that the old/existing ?output is still correct. The world to decide up-to-date is significant but ?perhaps somewhat less than the work to just do all the work. ? ?> ?see no benefit and almost certainly losses, in putting these in different executables. ?? ? Less code to maintain. Easier to redistribute. ? I like the model of distribution of C source. ? ? ?wget source && tar xf && configure && make && make install ? I want something like that. ? I am uncertain of "redistribution" and "active development" must/should be different. ? i.e. We do have something like this, and the two ways of working cooexist. ?? ? ? ?- Jay ---------------------------------------- > Date: Wed, 1 Jun 2016 17:08:34 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] replace build system?? > > > > On 06/01/2016 04:30 AM, Jay K wrote: >> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. > > It doesn't, not very well. A deep principle, if you can call it that, of C is that > each compilation has no knowledge of any other compilation. Likewise, when linking > them together, it's all just binary stuff, addresses, constants, etc. To get a > semblance of type-safe compilation, all coders have to correctly follow some variant > of a header file/#include idiom and create a matching makefile that correctly reflects > the actual interdependencies. There is no checking that these are done correctly, and > from direct experience, I can say they are routinely done wrong. > > All the modular languages have direct linguistic support for type-safe separate > compilation. Implementing this correctly while using the C compilation model lies > somewhere between very kludgy and impossible. > >> >> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? > > It couldn't possibly. Typical front end compilation actual work is similar to > or less than executable program startup/shutdown overhead. This effect > might be partially lost in the fact that m3cc is a separate executable and it > has to start up for every .[im]3 file. But it would still about double the > number of program executions and startup overheads. > >> Does/can it retain its incrementality? > > I don't think this is possible all. No matter what is in the makefile, Make > only understands dependencies among whole source files and must rebuild if > a depended-on file has been touched, even no changes to content. Our current > build system works on declaration granularity. (I presume this is what you > mean by incrementality." I am not aware of any language-independent build > infrastructure that keeps track of dependencies on other that source file > granularity. > >> >> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >> > > In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler > frontend, and language-specific link-time functions in one executable. I see no > benefit and almost certainly losses, in putting these in different executables. > >> - Jay >> >> >> >> >> ---------------------------------------- >>> From: jay.krell at cornell.edu >>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>> Date: Tue, 31 May 2016 17:18:46 +0000 >>> Subject: Re: [M3devel] replace build system?? >>> >>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>> Posix should standardize more of this. :( >>> >>> - Jay >>> >>> >>> >>> >>> >>> >>> ---------------------------------------- >>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>> To: dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>> From: mika at async.caltech.edu >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>> parallel if you turn on the right options. Every back-end invocation can be done >>>> in parallel. >>>> >>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>> >>>> "Daniel Alejandro Benavides D." writes: >>>> ... >>>>> A make-based build solution would have other advantages as well like=20 >>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>> four times faster. >>>> ... >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >> > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Fri Jun 3 16:44:12 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:44:12 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575197BC.1030100@lcwb.coop> BTW, 'though the current build system is considerably sophisticated, it does have some bugs, which we should fix. I have the exact setup & symptom for one of them written down somewhere. I think it's when adding a procedure signature to an interface when a procedure body for it was already present in a compiled exporting module, things don't get recompiled right. Also, I occasionally get recompile failures that I don't think are valid, and when I clean the package and recompile that, all is well. I haven't ever tracked down how to reproduce this or noted its exact symptom. We also don't keep track of compile options that affect generated code, so as to recompile when only the options change. E.g., optimization level, debug info presence and format. I did do this for the size of WIDECHAR, since mismatches here would be disastrous. On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invocation can be done >>>>> in parallel. >>>>> >>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go... >>>>> >>>>> "Daniel Alejandro Benavides D." writes: >>>>> ... >>>>>> A make-based build solution would have other advantages as well like=20 >>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20 >>>>>> four times faster. >>>>> ... >>>>> _______________________________________________ >>>>> M3devel mailing list >>>>> M3devel at elegosoft.com >>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>>> >>>> _______________________________________________ >>>> M3devel mailing list >>>> M3devel at elegosoft.com >>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel >>> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Jun 3 16:53:16 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 03 Jun 2016 09:53:16 -0500 Subject: [M3devel] replace build system?? In-Reply-To: References: , , , , , , <1784674298.4028093.1464714373116.JavaMail.yahoo@mail.yahoo.com>, , , <20160531171605.13AA61A2068@async.async.caltech.edu>, , , , <574F5CE2.8080506@lcwb.coop> Message-ID: <575199DC.4060305@lcwb.coop> On 06/03/2016 02:22 AM, Jay K wrote: > I understand the C model very well, but I'm not certain of all > the ramifications for Modula-3. > > > I understand C++ has limited checking via name mangling. > > > And that other systems carry names/signatures forward to runtime > and check them there. Even dynamically linked C++. And Modula-3. > > > However function names aren't everything -- i.e. they don't include > structural hashes of types. It is name-based, not structure-based. > > > The most common mistake in the loose C systems is not doing things > wrong per se, but accidentally mixing old and new versions, > and the result is undefined/crash. > > > We have the same problem with our "extern" mechanism -- how many times > I debugged stat corrupting stack until I put in the extra layer. > > > C++ systems are getting better and check for "ODR violations" at link time. > But checking at link time kinda contradicts my question. > Goal being to fit into "generic" build systems but C++ only starts > winning when the linker knows about it -- when the system is C++ specific. > If C++ requires C++ linker/driver, we shouldn't feel too bad > requiring a Modula-3 linker/driver. > > > > Typical front end compilation actual work is similar to > > or less than executable program startup/shutdown overhead. > > > Ideally this is not the case. Ideally process startup is just referencing > pages already in file system cache. Demand paging, unified file system cache > and memory manager and all that (all systems NT, Linux, etc. are the same > these days). But I realize in reality there is alway some amount of other CPU > to burn at each process start. The ideal is not quite reached. > > > m3cc is a separate executable..has to start up for every .[im]3 file. B > > I wonder if we can fix this btw. > If we can start it just once per library. > Process create really is fast on typical Posix systems, so people maybe > don't notice. > > There are a few methods: > - alway keep the m3cg ir files > - if any out of date, read them all into one large IR and write out one large assembly file > - I want do that anyway, to achieve a rich/poor man's LTCG/LTO. > poor: No need to worry about linker/driver support. > rich: We can afford the memory these days. Though at some point it wouldn't scale. > - I want do similar for C backend, as it seems noticable either the compiler startup > or the few #includes I sometimes use (though C++ modules promise some relief..) > > > Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them > separately as usual, but within one m3cg invocation. > I like this idea, but we don't want to force serialization where we now do parallel runs of m3cg. OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands seem to scroll by slower than the unit compiles when running cm3, despite the fact that the latter include a main program invocation of m3cc. Surely it's not merely smaller units? And we are also running separately too. > > > Does/can it retain its incrementality? > > I think the best but not great we could do is make would decide pessmisically > to run the compiler, but the compiler could figure out that the old/existing > output is still correct. The world to decide up-to-date is significant but > perhaps somewhat less than the work to just do all the work. > > > see no benefit and almost certainly losses, in putting these in different executables. > > Less code to maintain. Easier to redistribute. > I like the model of distribution of C source. > wget source && tar xf && configure && make && make install > We could just wrap what we have in makefile that, almost trivially invokes cm3 with a few odd options. Or is that cheating? > I want something like that. > I am uncertain of "redistribution" and "active development" must/should be different. > i.e. We do have something like this, and the two ways of working cooexist. > > > - Jay > > > > > > ---------------------------------------- >> Date: Wed, 1 Jun 2016 17:08:34 -0500 >> From: rodney_bates at lcwb.coop >> To: m3devel at elegosoft.com >> Subject: Re: [M3devel] replace build system?? >> >> >> >> On 06/01/2016 04:30 AM, Jay K wrote: >>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure. >> >> It doesn't, not very well. A deep principle, if you can call it that, of C is that >> each compilation has no knowledge of any other compilation. Likewise, when linking >> them together, it's all just binary stuff, addresses, constants, etc. To get a >> semblance of type-safe compilation, all coders have to correctly follow some variant >> of a header file/#include idiom and create a matching makefile that correctly reflects >> the actual interdependencies. There is no checking that these are done correctly, and >> from direct experience, I can say they are routinely done wrong. >> >> All the modular languages have direct linguistic support for type-safe separate >> compilation. Implementing this correctly while using the C compilation model lies >> somewhere between very kludgy and impossible. >> >>> >>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file? >> >> It couldn't possibly. Typical front end compilation actual work is similar to >> or less than executable program startup/shutdown overhead. This effect >> might be partially lost in the fact that m3cc is a separate executable and it >> has to start up for every .[im]3 file. But it would still about double the >> number of program executions and startup overheads. >> >>> Does/can it retain its incrementality? >> >> I don't think this is possible all. No matter what is in the makefile, Make >> only understands dependencies among whole source files and must rebuild if >> a depended-on file has been touched, even no changes to content. Our current >> build system works on declaration granularity. (I presume this is what you >> mean by incrementality." I am not aware of any language-independent build >> infrastructure that keeps track of dependencies on other that source file >> granularity. >> >>> >>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level? >>> >> >> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler >> frontend, and language-specific link-time functions in one executable. I see no >> benefit and almost certainly losses, in putting these in different executables. >> >>> - Jay >>> >>> >>> >>> >>> ---------------------------------------- >>>> From: jay.krell at cornell.edu >>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es >>>> Date: Tue, 31 May 2016 17:18:46 +0000 >>>> Subject: Re: [M3devel] replace build system?? >>>> >>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic. >>>> Posix should standardize more of this. :( >>>> >>>> - Jay >>>> >>>> >>>> >>>> >>>> >>>> >>>> ---------------------------------------- >>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com >>>>> To: dabenavidesd at yahoo.es >>>>> Date: Tue, 31 May 2016 10:16:05 -0700 >>>>> From: mika at async.caltech.edu >>>>> Subject: Re: [M3devel] replace build system?? >>>>> >>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly >>>>> parallel if you turn on the right options. Every back-end invoca