From jay.krell at cornell.edu Tue Sep 1 01:44:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 23:44:23 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: <55E34A77.1050001@lcwb.coop> References: , <55E34A77.1050001@lcwb.coop> Message-ID: Agreed -- I forgot to acknowledge/repeat that this is relatedto the first instruction in a function, not arbitrary instructions. On the matter of 4 or 8 bytes though -- the thing is -- many but not all64bit architectures are the exact same instruction set as 32 bits,just a different selection of instructions. Same size, same encodings, same alignment. I believe this is true for alpha, mipa, ppc, sparc, hppa. It is almost true for x86, though that kinda doesn't count,since there is no fixed size or alignment requirement. Fyi: x86/amd64 instructions are between 1 and 15 bytes in size,inclusive, and all sizes in between. All the "lower" sizesincluding 1 are common. Upper sizes are rare, maybe starting around 7 bytes.The encoding scheme allows for longer instructions, butthere is a deliberate documented limit. Not sure about arm -- arm64 at least gets rid of thumb.So arm instructions either 4-aligned or at odd addresses,and arm64 is like all the rest -- fixed size 4 byte aligned. So 4 byte -1 e.g. on ppc32 has the same meaning as 8 byte -1 on ppc64,just that it is two in a row. It looks like we have agreement -- we can set this to a constant. The marker value is -1 I checked. I still find the function If_closure a little unclear/subtle.I read through my comments in Target.m3 again. It is false if: - the system ever has alignment faults - AND functions/instructions might be less aligned than closures Some of the commented I added in the code are incorrect. The comments say that a misaligned function pointer is readone byte at a time. I believe the actual behavior is thata misaligned function pointer is deemed not a closure,and reading the marker is skipped. The code is all still there, but branched around.When aligned_procedures == true, less code is generated,no alignment check is generated. Most 32bit platforms set Aligned_procedures = trueand most 64bit platforms set it to false. The exceptions: amd64 is also true. arm32 is false, due to Thumb mode. Because most platforms, 32bit and 64bit, have fixed size4 byte aligned instructions. So 32bit platforms haveinstructions (and functions) thereforealigned the same as closures. 64bit platforms generallyhave closures more aligned than instructions. I think if we ignored arm32, and penalized only amd64,which granted is the most common platform,we could set aligned_procedures = false for all 64bit,true for all 32bit. And having another variablethat coincided with word size is ok. But arm32's thumb instructions mean a function pointermight be unaligned. I'll make it always false. > Method body procedures are required to be top-level Kind of a language limitation compared to "more dynamic" languages. Definitely not trivial to do otherwise -- heap allocated garbage collected stack frames and such.. Thank you, - Jay > Date: Sun, 30 Aug 2015 13:24:55 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Aligned_procedures and closure markers? > > > > On 08/30/2015 02:45 AM, Jay K wrote: > > The agenda remains seeing if Target variables can be made constants. > > > > The discussion in this case is more complicated and some facts are unclear. > > > > > > Background: > > > > > > Nested functions are a problem. > > In particular, if you can take their address. > > Taking the address of a nested function presents a problem. > > You are presented with two or three solutions. > > > > > > - runtime code gen > > - either on the stack > > > > - or somewhere more expensive, possibly with garbage collection > > > > - possibly "templated" instead of "arbitrary"; the meaning > > of this is a lot to explain -- related to libffi and mremap, which > > isn't entirely portable, but is e.g. portable to Windows > > > > > > - or instead of runtime codegen, altering how function pointers are > > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > > The sequence is roughly: > > Check if it is a "regular" function pointer or a "closure". > > If it is a "regular" function pointer, just call it. > > If it is a "closure", it contains a function pointer and a "static link". > > Call the function pointer, passing the static link. > > > > > > To tell if it is "regular" function pointer or a "closure", roughly > > what is done is the data at the function pointer is read and compared > > against a marker value. If it equals the marker value, it is a closure. > > > > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > > The value of the marker checked for is 0 or -1, I'd have to check. > > The alignment of the pointer might be a factor. In particular, on most > > architectures, all instructions have a certain alignment. If the pointer has > > less alignment, it can't be an instruction. Or maybe on those architectures, > > the bytes are read one at a time to avoid alignment faults. > > > > > > In particular, as far as I know, the following: > > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > > 4 bytes and 4 byte aligned, so functions are at least also as much > > > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > > pointer is actually odd as well, and the low bit is removed to really find the instructions > > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > > > > I could use confirmation on much of this. > > > > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > > value that works on all. > > > > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > > then checking for two in a row reduces the odds of a false positive. > > > > > > However, given that the closure is a marker and two pointers, it isn't like you are going > > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > > > > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then > it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part > of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it > by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the > space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a > valid instruction or sequence thereof. > > So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized > alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity > here. > > Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. > It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for > certain (especially if we want to be able to call procedures produced by other compilers) but much less > likely to result in a problem. > > Just a wild guess, but I would not be surprised if ELF and other object formats would require the > machine code of a function/procedure to begin on a native word boundary, even if the hardware > instruction set does not. Where so, this would obviate checking the alignment before checking > for a closure, though probably target-dependently. > > > > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > > > > However, I want less target-variation not more. > > > > > > Here are some my lingering questions: > > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > > - Is a 64bit marker value actually sufficient on IA64? > > The way to help here, I think, is to ensure that a 64bit marker, > > not a 128bit marker, contains the "template", and an invalid "template". > > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > > > > i believe all of these function pointers are rare. > > I hope/believe the object method calls do not check for closures -- though actually > > that is related to a useful language construct, that I doubt we have. > > > > Method body procedures are required to be top-level, ensured statically, so there is no > need for method call code to consider the possibility of the pointer in the object type > to be a closure. > > > > > The simplest solution is likely: > > - ignore IA64, or research it further > > - keep marker size at integer > > Pointer would be target-independent in getting the following two pointers aligned. > > > - for the C backend, assume no alignment of function pointers -- give up > > any of the optimization, esp. x86/amd64. > > I think this optimization both applies to a low-frequency situation and has a very > small benefit, so I would not worry about giving up on it. > > > > > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > > > > Thoughts? > > > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 07:52:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 05:52:16 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816134306.GB5008@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com> Message-ID: old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd "stress test" my output.Digital Mars might be another.And yes the Intel compiler -- which is a proxy for a set of compilers -- the EDG frontend.(There are approx four C++ frontends in practical use these days: Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front end.) - Jay > Date: Sun, 16 Aug 2015 13:43:06 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > > Autoconf output has little/none dependency.Sun's SPARC optimization are > > mostly irrelevant as we have little C code,unless you use the C backend. > > Ok, well, it sounds like it will be increasingly important as you go to your > C backend though. > > > > somewhat of a test of C portability > > This I appreciate. We have some C code and the C backend.More compilers > > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > > now clang -- having found and worked around a bug in its assembler.Some > > time soon I'll expand to Metrowerks and Digital Mars.. > > I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have > the latest versions. C/C++ are not my thing but if I can help by just > building stuff to see what messages we get or if it breaks etc. let me know > and I will try to participate. > > I didn't know Metrowerks was still around. I thought they got swallowed up > by Motorola and then removed from all the non-embedded space. > > > > - Jay > > > > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > > From: microcode at zoho.com > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] Build Server - Plan > > > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > > The output of autoconf/automake should have lightweight dependencies.They > > > > might stress make, might require GNU make.They might stress the sh, but I > > > > think there are adequate shells out there. > > > > > > That is typically one issue with autoconf, requiring gnu tools in the > > > path. On Solaris this can be annoying since most Solaris people don't use > > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > > are up to date or even current by any stretch of the imagination. > > > > > > > They are meant to be easy for people building stuff to use.They aren't > > > > meant to be easy for people developing stuff to use. > > > > > > Not sure what you meant here... > > > > > > > Look at this way..while people complain and there are widelyused > > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > > Aix, Irix, etc. > > > > > > They often don't "work" for Solaris as-installed but they can most often be > > > made to work. Increasingly, as automake and its prereqs get version bumps > > > there are problems building apps on Solaris because Solaris installs with a > > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > > ran into a problem with the last year where Solaris awk was not good enough > > > to install(!) an app that compiled on Solaris as part of autotools so I had > > > to download a newish copy of gawk and install it. So really, it is much > > > better not to go there if you can avoid it. > > > > > > I am not suggesting an alternative to autotools but just pointing out it is > > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > > Linux. I also run BSD on several platforms and because of the same issues as > > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > > shells, etc. not being good enough for autotools) it is sometimes > > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > > no good/easy solution to this, just to point out it is not the slam-dunk as > > > might be thought. > > > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > > buildingthat with autoconf/automake. > > > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > > is in a non-standard location and is often not used. Ideally, apps to be run > > > on Solaris should be able to be built with native (non-gnu) tools. The > > > Studio compilers are very good and have optimizations for SPARC that should > > > be better than what gcc can provide and it is also somewhat of a test of C > > > portability since Studio doesn't necessarily provide all non-standard gcc > > > extensions. > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 08:22:24 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 06:22:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> <20150816134306.GB5008@zoho.com> Message-ID: <20150901062224.GA3050@zoho.com> On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > "stress test" my output.Digital Mars might be another.And yes the Intel > compiler -- which is a proxy for a set of compilers -- the EDG > frontend. Hi, You lost me here. What is EDG? As far as I know, the Intel compiler is a proprietary (and complete, self-contained) product of Intel. I think I read on Intel's forums in a post by Steve Lionel that earlier versions were based on code that made its way from DEC to COMPAQ but was later reworked so that it is almost all new. An interesting tangent: I ran some tests earlier and can't remember for sure but it seems to me there is some Open64 code in Intel's compilers. Whether Intel wrote it and it made its way to Open64, or whether it was part of Open64 and Intel took some of it, I do not know. > (There are approx four C++ frontends in practical use these days: > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > end.) From jay.krell at cornell.edu Tue Sep 1 08:23:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:23:42 +0000 Subject: [M3devel] m3front -unfold_nested_procs Message-ID: It'd be nice, if someone has the time, to let -unfold_nested_procs work with cm3cg.I tried just silently ignoring it, but then it crashes. -unfold_nested_procs is required by the C backend.I and I think the NTx86 backend. And required to be omitted by the gcc/cm3cg backend. Ideally, even if they aren't ABI compatible (due to how static link is passed,unfortunate, I wish I could fix it), they could at least operate on identical M3 IR.To ease moving between them. And granted, it'd be nice if the C backend didn't care. I think I can do that,by moving the functions around afterward.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 08:27:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:27:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150901062224.GA3050@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com>, , <20150901062224.GA3050@zoho.com> Message-ID: edg.comWhile almost nobody writes their own C++ compiler, even fewer people write their own C++ frontend.Edison Design Group.A small company whose main/only product is a C++ front end that they license in sourceform to people who are interested in, say, writing a backend only. - Jay > Date: Tue, 1 Sep 2015 06:22:24 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > > "stress test" my output.Digital Mars might be another.And yes the Intel > > compiler -- which is a proxy for a set of compilers -- the EDG > > frontend. > > Hi, > > You lost me here. What is EDG? > > As far as I know, the Intel compiler is a proprietary (and complete, > self-contained) product of Intel. I think I read on Intel's forums in a post > by Steve Lionel that earlier versions were based on code that made its way > from DEC to COMPAQ but was later reworked so that it is almost all new. > > An interesting tangent: I ran some tests earlier and can't remember for sure > but it seems to me there is some Open64 code in Intel's compilers. Whether > Intel wrote it and it made its way to Open64, or whether it was part of > Open64 and Intel took some of it, I do not know. > > > (There are approx four C++ frontends in practical use these days: > > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > > end.) > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 09:18:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 07:18:02 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] 4537cb: remove unused locals In-Reply-To: References: <55e53db0990a0_3be33fd3fe0af2a068547@hookshot-fe2-cp1-prd.iad.github.net.mail>, <7C1D0FBD-5C9A-4883-A47C-57E613FA171E@purdue.edu>, <4E019C37-773B-42AE-AC49-F53C52249195@purdue.edu>, <2E8C2B23-F814-46FF-A59A-C92F486EC273@purdue.edu>, Message-ID: I use git pull to update my local repository to be "like" github.I use git commit to commit my local edits to my local repository.It use git push to push my local repository changes to the master.I don't know what rebasing is or how/why to use it. - Jay Subject: Re: [M3commit] [modula3/cm3] 4537cb: remove unused locals From: hosking at purdue.edu Date: Tue, 1 Sep 2015 17:10:51 +1000 CC: jay.krell at cornell.edu; m3commit at elegosoft.com; m3devel at elegosoft.com To: hosking at purdue.edu It appears to be this merge action which is causing the problem.I don?t have time to explore right now. If anyone else can it would be most appreciated. On 1 Sep 2015, at 5:08 pm, Antony Hosking wrote:Whatever you are doing I now don?t know how to recover from.It appears that somehow you are rebasing the history on the master branch, but I don?t quite know how. On 1 Sep 2015, at 5:00 pm, Antony Hosking wrote:In your work flow are you using "git push" to push your local commits? Or are you doing something else? On 1 Sep 2015, at 4:58 pm, Antony Hosking wrote:Jay, it appears that your commits are still messing up the history. As of your most recent commit we have lost history again. What exactly are you doing when you use git against the repo? For instance, if you take a look at RTCollector we see a history with the oldest commit labelled "initial diff from ../config/PPC_LINUX" from Jan 4, 2008. Whereas it should have the oldest commit labeled as follows from when Olaf first imported from cvs to svn. commit cfac2c79141d61c291a40177f0d5490f774c032fAuthor: Olaf Wagner Date: Wed Jan 24 12:24:49 2001 +0000 This commit was generated by cvs2svn to compensate for changes in r20, which included commits to RCS files with non-trunk default branches. On 1 Sep 2015, at 3:54 pm, jaykrell wrote: Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 4537cb804fbf864a076235495db77eebf9c5dab6 https://github.com/modula3/cm3/commit/4537cb804fbf864a076235495db77eebf9c5dab6 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- remove unused locals Commit: 477c3d058a95ba8a06c90ac5d36dceefd3e7f39a https://github.com/modula3/cm3/commit/477c3d058a95ba8a06c90ac5d36dceefd3e7f39a Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- Fix regression from "7/31 Initial integration of llvm back end into cm3's build system." when bootstrapping using cm3cg. .c files pass through for bootstrap, along with any .h or checked in .s/.o etc. bootstrapping is using cm3 -boot. Commit: 459512da7848b534636c1fe4e1095eee76ad2d89 https://github.com/modula3/cm3/commit/459512da7848b534636c1fe4e1095eee76ad2d89 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: A m3-sys/llvm3.6.1/src/LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 A m3-sys/llvm3.6.1/src/Main.m3 A m3-sys/llvm3.6.1/src/m3makefile Log Message: ----------- Merge branch 'master' of github.com:modula3/cm3 pull from master Compare: https://github.com/modula3/cm3/compare/b068a306fe87...459512da7848_______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 18:28:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? Message-ID: I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 19:00:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 17:00:07 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: Sorry, this merits more discussion. - There are at least three ways compilers chose between C and C++. invocation name: gcc vs. g++, lowercase cc vs. uppercase CC I think clang vs. clang++. Visual C++ only has one invocation: cl Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. Command line switches: gcc -x c++ clang -x C++ cl -Tc means next source file is C -TC means all are C -Tp means next is C++ -TP means all are C++ We'd have to read up on others. One of the combinations implemented by gcc is deprecated by clang. I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). So I switched to use -x c++. "cpp" is ambiguous between C preprocessor and C++. "cxx" is often therefore used to indicate C++. x is a rotated +. We could have config files specify SYSTEM_CC and SYSTEM_CXX. And we could have functions compile_c and compile_cxx. We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe others, error if there are multiple, and chose compile_c vs. compile_cxx. Or compile_c could chose based on the extension. In any case, makes changes to the C backend path, and in the llvm binding/backend directories should be easy. We don't *have* to change the meanings everywhere to effect them. I do not actually intend to rewrite all of our little bits of C using C++ specific features. But the LLVM stuff uses C++ and the C backend imho ought to. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 19:08:58 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 17:08:58 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <20150901170858.GA6140@zoho.com> Since you or somebody else mentioned autotools, I think the way this is normally handled is for the script to use cc or CC and it expects you to set those values in your shell to whatever compiler you want to use. On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > Sorry, this merits more discussion. > > - There are at least three ways compilers chose between C and C++. > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > I think clang vs. clang++. > Visual C++ only has one invocation: cl > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > Command line switches: > gcc -x c++ > clang -x C++ > cl -Tc means next source file is C > -TC means all are C > -Tp means next is C++ > -TP means all are C++ > We'd have to read up on others. > > One of the combinations implemented by gcc is deprecated by clang. > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > So I switched to use -x c++. > > "cpp" is ambiguous between C preprocessor and C++. > "cxx" is often therefore used to indicate C++. x is a rotated +. > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > And we could have functions compile_c and compile_cxx. > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > others, error if there are multiple, and chose compile_c vs. compile_cxx. > Or compile_c could chose based on the extension. > > In any case, makes changes to the C backend path, and in the llvm binding/backend > directories should be easy. We don't *have* to change the meanings everywhere > to effect them. I do not actually intend to rewrite all of our little bits > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > imho ought to. > > - Jay > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Tue, 1 Sep 2015 16:28:10 +0000 > Subject: [M3devel] gcc vs. g++ in config files? > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- From jay.krell at cornell.edu Tue Sep 1 20:02:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 18:02:27 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: <20150901170858.GA6140@zoho.com> References: , , <20150901170858.GA6140@zoho.com> Message-ID: For now we aren't using autotools. I remain on the fence. I want some of what they do, but they really are fairly ugly and slow and overkill.I kinda want some other small portable scripting language.The way they find the C compiler is pretty simple really.Maybe python. Embedding it and adding functions? I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? libtool supports more systems than our config files, but is alsokinda ugly/overkill/slow and somewhat philosophically poor.For example, the fact that they default to compiling everything twice,pic and non-pic, bothers me. Surely it is not worth a) the double timeto build b) the slight loss of security in executables.All code should just be pic.They do make it an option, but when people foist that default decisionon the masses, it makes me wonder.Apple has no such modes.Windows has no such modes (though Windows doesn't do quitewhat I want either). In particular, I want someone else to - find C and C++ compiler -- but what autotools does here is pretty simple - pick out int8, int16, int32, int64 for me, with greater portability to older and newer systems -- but there is inttypes.h on most systems anyway - figure out how to link -- for static there is just one portable way (I adjusted the config files recently), but for shared, libtool or cmake do it better?easier? - generate portable makefiles for bootstrapping (or for regular building?) Or just GNU makefiles? That is one some guidance I've seen -- give up on portable makefiles and just use GNU make. cmake?? I don't want to lose the fine grained incrementality the integrated builder has,but it is tempting to make the system just generate makefiles. I wonder which parts of the system are worth maintaining, vs. giving in to the largerecosystem of stuff being maintained for us. cm3 is ahead of its time for 20 years ago but these days seems largely redundant.Perhaps we can/should whittle it down to a more unique and worthwhile core?I don't know. - Jay > Date: Tue, 1 Sep 2015 17:08:58 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] gcc vs. g++ in config files? > > Since you or somebody else mentioned autotools, I think the way this is > normally handled is for the script to use cc or CC and it expects you to > set those values in your shell to whatever compiler you want to use. > > On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > > Sorry, this merits more discussion. > > > > - There are at least three ways compilers chose between C and C++. > > > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > > I think clang vs. clang++. > > Visual C++ only has one invocation: cl > > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > > Command line switches: > > gcc -x c++ > > clang -x C++ > > cl -Tc means next source file is C > > -TC means all are C > > -Tp means next is C++ > > -TP means all are C++ > > We'd have to read up on others. > > > > One of the combinations implemented by gcc is deprecated by clang. > > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > > So I switched to use -x c++. > > > > "cpp" is ambiguous between C preprocessor and C++. > > "cxx" is often therefore used to indicate C++. x is a rotated +. > > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > > And we could have functions compile_c and compile_cxx. > > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > > others, error if there are multiple, and chose compile_c vs. compile_cxx. > > Or compile_c could chose based on the extension. > > > > In any case, makes changes to the C backend path, and in the llvm binding/backend > > directories should be easy. We don't *have* to change the meanings everywhere > > to effect them. I do not actually intend to rewrite all of our little bits > > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > > imho ought to. > > > > - Jay > > > > > > > > > > From: jay.krell at cornell.edu > > To: m3devel at elegosoft.com > > Date: Tue, 1 Sep 2015 16:28:10 +0000 > > Subject: [M3devel] gcc vs. g++ in config files? > > > > > > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Sep 1 20:32:00 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 1 Sep 2015 14:32:00 -0400 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: <20150901170858.GA6140@zoho.com> Message-ID: <20150901183200.GA10167@topoi.pooq.com> On Tue, Sep 01, 2015 at 06:02:27PM +0000, Jay K wrote: ... ... > I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? It might be scons you're looking for. It's a build system that's made out of python. It is not a plug-compatible make replacement. It is completely different. -- hendrik From rodney_bates at lcwb.coop Wed Sep 2 17:32:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 02 Sep 2015 10:32:03 -0500 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <55E71673.4010508@lcwb.coop> On 09/01/2015 11:28 AM, Jay K wrote: > I suggest we replace "gcc" with "g++". > In all the config files (and in the bootstrap archives). > And similar for Sun CC, and Visual C++ use -TP. > > > rationale: > It will likely cleanup our LLVM stuff, which currently > uses a custom Makefile and drops derived files into the src directory. There are a couple of problems here. I initially tried putting compiled C/C++ code in the build directory. Since it is target-dependent, it obviously does not belong in src. But a bug in the builder just wipes out the file with a circular symbolic link: % import_obj needs work!! % import_obj() puts a symlink in ../BUILD_DIR/.o, pointing to , % located relative to the src directory. Later, it links the symlink into the package. % If the file is target-dependent, (as these and most .o files are), we want to put them % in ../BUILD_DIR, the target build directory, and only target-dependent place we have. % But import_obj("../BUILD_DIR/") just replaces it with a circular symlink. % When this is fixed, the Makefile will have to be changed to put it in the right place. The bug shouldn't be hard to fix, it just hasn't gotten to the top of my list. Then the .o files can be moved to the build directory. But that's still using make to build the .o files. The other problem is the cm3 build system. It's really quite sophisticated about dependencies within Modula3 code, but things like 'c_source' don't, as far as I can see, do any dependency analysis at all. They work fine for the odd .c and .h files here and there in the cm3 tree. But the bindings to llvm (the parts we must write in C & C++) bring in large #include closures from the llvm source tree and then link in several compiled llvm libraries. I don't think cm3's build system is at all up to this. So I used make for that. Actually, as far as I can find, llvm has not provided any dependency information about what libraries will be needed, in what order (which, BTW, undergoes reversal of certain segments). So far, it has been a very dicey and frustrating game of wild guesses and experiments. I am sure the lists I have come up with must have some undetected unnecessary things. I haven't even figured out a way to find out what library a link name declaration is in. > It will ease/allow using C++ exception handling in the C backend. > I've been using g++ for months/years on Darwin. > counter-rationale: > Not all C code is valid C++, and sometimes the meanings are different. > I converted the coverage code only recently. > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Sep 3 01:56:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 2 Sep 2015 23:56:06 +0000 Subject: [M3devel] build_standalone errors Message-ID: import_lib("dl", "/usr/lib")import_lib("ncurses", "/usr/lib") + build_standalone() => on Darwin, where is only /usr/lib/libdl.dylib and /usr/lib/libncurses.dylb. clang: error: no such file or directory: '/usr/lib/libncurses.a'clang: error: no such file or directory: '/usr/lib/libdl.a' build_standalone should probably be much smarterand at least probe file existance; if found specify it;if not found use -lfoo. The idea being to be standalone with respect to available static libs,not all libs. More so, really, stand-aloneness should be specifiable on a libraryby library, or directory basis. This is while building m3-sys/llvm, making it standalone, whichI expect *might* be desired. I'll go ahead without standalone. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 4 23:48:42 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 04 Sep 2015 16:48:42 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail> Message-ID: <55EA11BA.7010704@lcwb.coop> Hmm. As near as I remember, I assumed the front end was unnesting calls just because it did it on one example (using m3cc). Looking at the code, I see command line options -nested_calls and -no_nested_calls (the default). However, the only use of this I can find affects whether an independent check by a M3CG_Check.T instance verifies that there is no nesting. So it looks like it's actually unconditional after all. I'll leave the comments as they are unless this changes. I would expect a back end not to have to cope with them, unless it was generating final code for a bytecode or stack-like machine that could directly handle them nested. On 09/03/2015 05:38 PM, Jay wrote: > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > - Jay > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > >> Branch: refs/heads/master >> Home: https://github.com/modula3/cm3 >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/m3middle/src/M3CG_Ops.i3 >> >> Log Message: >> ----------- >> Comment no nested calls occur in cm3 IR. >> >> Comment the fact that nested calls in source code are unnested in >> cm3 intermediate representation. This is an essential invariant >> to both producers and consumers of this IR. >> >> Changes to be committed: >> >> modified: src/M3CG_Ops.i3 >> >> >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Revert an incorrect fix due to misdiagnosis. >> >> The actual problem here was not a lack of a set_source_file, >> it was reading binary cm3 IR with the ascii reader. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 >> >> Lots of changes happened to this file in package llvm, while the version >> in package llvm3.6.1 has been undergoing integration with the greatly >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 >> found in package llvm3.6.1. Also some random manual review and fixes. >> >> Compiles and runs to completion, with superficially believable output, >> on one small test case involving separate temporaries in and out of >> a FINALLY procedure. >> >> Works with cm3 command, when its executable m3llvm has been shipped. >> >> The test compile fails to link, not finding alloca. m3llvm needs to >> detect calls on alloca and translate internally into llvm alloca >> instructions, rather than letting them through to be linked in. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Sep 5 03:51:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 5 Sep 2015 01:51:16 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: <55EA11BA.7010704@lcwb.coop> References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail>, , <55EA11BA.7010704@lcwb.coop> Message-ID: -unfold_nested_procs is what I'm referring to. Sorry, I comprehend better now -- "procs" vs. "calls". nested call: foo(bar()) unnested call: temp = bar() foo(temp) nested proc: void foo(){ void bar() { }} unnested proc: void foo(){ } void bar(){} And then everything I said. :)I want fewer variables and more constants. :)I want one (or fewer) persisted form of the IR and an ability for a backend to easily transform.I need to push the M3CG_MultiPass stuff around more I think.Then again, the IR doesn't tend to live long and there isn't likely a cause for it to, so variation is ok.The only time it lives long is when iterating tightly on a backend.It could be useful for bootstrapping perhaps, but I think we have a better option already -- C/C++. - Jay > Date: Fri, 4 Sep 2015 16:48:42 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. > > Hmm. As near as I remember, I assumed the front end was unnesting calls just because it > did it on one example (using m3cc). Looking at the code, I see command line options > -nested_calls and -no_nested_calls (the default). However, the only use of this I can > find affects whether an independent check by a M3CG_Check.T instance verifies that > there is no nesting. So it looks like it's actually unconditional after all. I'll > leave the comments as they are unless this changes. > > I would expect a back end not to have to cope with them, unless it was generating > final code for a bytecode or stack-like machine that could directly handle them nested. > > On 09/03/2015 05:38 PM, Jay wrote: > > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > > > - Jay > > > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > > > >> Branch: refs/heads/master > >> Home: https://github.com/modula3/cm3 > >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 > >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/m3middle/src/M3CG_Ops.i3 > >> > >> Log Message: > >> ----------- > >> Comment no nested calls occur in cm3 IR. > >> > >> Comment the fact that nested calls in source code are unnested in > >> cm3 intermediate representation. This is an essential invariant > >> to both producers and consumers of this IR. > >> > >> Changes to be committed: > >> > >> modified: src/M3CG_Ops.i3 > >> > >> > >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 > >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Revert an incorrect fix due to misdiagnosis. > >> > >> The actual problem here was not a lack of a set_source_file, > >> it was reading binary cm3 IR with the ascii reader. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 > >> > >> Lots of changes happened to this file in package llvm, while the version > >> in package llvm3.6.1 has been undergoing integration with the greatly > >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 > >> found in package llvm3.6.1. Also some random manual review and fixes. > >> > >> Compiles and runs to completion, with superficially believable output, > >> on one small test case involving separate temporaries in and out of > >> a FINALLY procedure. > >> > >> Works with cm3 command, when its executable m3llvm has been shipped. > >> > >> The test compile fails to link, not finding alloca. m3llvm needs to > >> detect calls on alloca and translate internally into llvm alloca > >> instructions, rather than letting them through to be linked in. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 > >> _______________________________________________ > >> M3commit mailing list > >> M3commit at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Sep 11 03:53:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 11 Sep 2015 01:53:36 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone Message-ID: Here is what I get: some "crash", some "fail", some get the file format mixed up, some work!6 modes. 4 crash, 1 appears to work, 1 appears close jair:m3core jay$ for a in \IntLlvmObj \IntLlvmAsm \ExtLlvmObj \ExtLlvmAsm \StAloneLlvmObj \StAloneLlvmAsm; \ do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done IntLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3*** IntLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** ExtLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 ExtLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL StAloneLlvmObjm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io => success? except for:error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) StAloneLlvmAsmclose, maybe just a config problemm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.isfg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.ioRT0.is:5:2: error: unknown directive I assume we should ignore Int and Ext and focus on StAlone for now. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 11 22:13:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 11 Sep 2015 15:13:43 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: Message-ID: <55F335F7.9010108@lcwb.coop> Yes, I put in all the modes I thought of while working in there, as it's a lot easier to do it when you are familiar with the code than refamiliarize later. There were already some with comments like "No such back end". Only the StAloneLlvm ones have an actual back end. The others are intended for modes that link in at least some of llvm into cm3. As discussed, we may not want to do this. The only mode I have tried is StAloneLlvmAsm. This is running to compile completion on small examples and passing a good few of the compiler tests. Right now, I am working on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. On 09/10/2015 08:53 PM, Jay K wrote: > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > jair:m3core jay$ for a in \ > IntLlvmObj \ > IntLlvmAsm \ > ExtLlvmObj \ > ExtLlvmAsm \ > StAloneLlvmObj \ > StAloneLlvmAsm; \ > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > IntLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > *** > > IntLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** > > ExtLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > ExtLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > > StAloneLlvmObj > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > => success? > > except for: > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > StAloneLlvmAsm > close, maybe just a config problem > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > RT0.is:5:2: error: unknown directive > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 13 01:26:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:26:35 +0000 Subject: [M3devel] builder behavior upon error -- subsequent files not fully compiled Message-ID: fyi, I observe the following: When compiling multiple files, and one fails, the failure inappropriately affects later files, in that, one or more of: - all outputs are deleted - or -keep isn't honored - or the "build pipeline" is truncated, i.e. we might check the validity of .m3/.i3 files but we don't progress all the way to generate assembly/object files It used to be worse, we just discussed, I had regressed it years ago by accident, in that compilation stopped and no further errors are reported. But now it can largely succeed, but not have the intended outputs, so you fix the one error, and then recompile everything. There is just one boolean "compile_failed" that is set by the first failure, and left set, and the code doesn't know if it was from an earlier file or the current file. It either needs to be a counter, and captured before certain steps and checked for > after them, or there should be two booleans, one that is set/cleared per file, one that is set whenever the first becomes set and is never cleared, or functions should return a boolean, for the current file, and either also set the "global", or set the global near the top of the call tree. It isn't global, but it is nearly so. Making this code multithreaded will be an unfortunately large task.. I'll probably fix this soon. This is just a heads up. I'm not addressing the single-threadedness, just the other. It shouldn't be a big change. This is related to: What I'm actually seeing with one of the LLVM passes is the first invocation of some tool crashes, so the rest truncate their build pipeline and claim m3front failed, which isn't true, either it didn't run or didn't fail. The failure is from the first file. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 13 01:50:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:50:08 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F335F7.9010108@lcwb.coop> References: , <55F335F7.9010108@lcwb.coop> Message-ID: Those comments were from me, since I went through and 1) added the C mode 2) tried to clean up what I percieve as rather messy, and still messy. You helped my vocabulary here -- it should be "cartesian factored". Rather than every "product" be its own case. I moved it toward that, but it still confusing. Basically you want a set of available steps, and then just sequence them, possibly omitting some of them. I can see how you might have roughly 10 choices, depending on if you are outputing bitcode or LLVM assembly or building LLVM data structures, and linking to LLVM or not, and LLVM is producing assembly or object files. I would suggest that LLVM produing assembly is not a useful choice. Remove that from the matrix. Have it produce object files if you can. For debugging, you can fiddle with the switches or dump the files afterward. If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. You can use the "external object" mode plus config file stuff. I started that way for C and didn't change the builder initially. But it was slower. I don't remember why i needed builder support, vs. the existing "internal object" mode, maybe to more easily reuse the existing support for compiling C. My mode is kind of "internal plus run some quake code". I guess I could setup linkage so the C backend could call the quake code w/o the builder knowing about it. Maybe I should try that, so we can get back to the original set of modes. Basically "external" means "produce cm3cg IR files". "Internal" means don't produce them. And then "assembly" means "run assembler after backend" and "object" means don't run assembler because we already have objects. Whether or not the m3cg IR files are being converted to C and then objects or gcc internals and then objects, or LLVM internals and then objects, the builder doesn't need to know or act differently. It just runs the quake function m3_backend or such, and it decides, via a separate factor from mode to run llvm or cm3cg on the IR file. Basically, let's say we have n different external backends producing object files. That doesn't imply any new modes. But some other variable that affects the quake code that runs cm3cg. Thank you for the confirmation as to which mode you are using. - Jay > Date: Fri, 11 Sep 2015 15:13:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > to do it when you are familiar with the code than refamiliarize later. There were > already some with comments like "No such back end". Only the StAloneLlvm ones > have an actual back end. The others are intended for modes that link in at least > some of llvm into cm3. As discussed, we may not want to do this. > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > on small examples and passing a good few of the compiler tests. Right now, I am working > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > On 09/10/2015 08:53 PM, Jay K wrote: > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > jair:m3core jay$ for a in \ > > IntLlvmObj \ > > IntLlvmAsm \ > > ExtLlvmObj \ > > ExtLlvmAsm \ > > StAloneLlvmObj \ > > StAloneLlvmAsm; \ > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > IntLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > *** > > > > IntLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** > > > > ExtLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > ExtLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > > > StAloneLlvmObj > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > => success? > > > > except for: > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > StAloneLlvmAsm > > close, maybe just a config problem > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > RT0.is:5:2: error: unknown directive > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 13 02:13:27 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 12 Sep 2015 19:13:27 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: , <55F335F7.9010108@lcwb.coop> Message-ID: <55F4BFA7.7090805@lcwb.coop> My inference has been that "internal" means the whole compilation job is done by code linked in (not necessarily statically) to cm3, e.g., using the integrated x86 backend. "external" means there are one or more separate executables to run after the front end, e.g., m3cc and as, or m3llvm, llc, asm, etc. I definitely want the possibility of keeping assembly form for development work, even when 'as' is run. And we should assume that occasional development work will continue forever. Disassembling object code will lose sometimes very useful information, compared to keeping assembly as originally generated as a side output. Actually, I'd also like to be able to set it up so m3cgcat gets run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It would just save a couple of manual steps that I am typing over and over, slowing down the debug process. On 09/12/2015 06:50 PM, Jay K wrote: > Those comments were from me, since I went through > and 1) added the C mode 2) tried to clean up what > I percieve as rather messy, and still messy. > > > You helped my vocabulary here -- it should be "cartesian factored". > Rather than every "product" be its own case. > > > I moved it toward that, but it still confusing. > > > Basically you want a set of available steps, and > then just sequence them, possibly omitting some of them. > > > I can see how you might have roughly 10 choices, > depending on if you are outputing bitcode or LLVM assembly > or building LLVM data structures, and linking to LLVM or not, > and LLVM is producing assembly or object files. > > > I would suggest that LLVM produing assembly is not a useful choice. > Remove that from the matrix. > Have it produce object files if you can. > For debugging, you can fiddle with the switches or dump the files > afterward. > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > You can use the "external object" mode plus config file stuff. > I started that way for C and didn't change the builder initially. > But it was slower. I don't remember why i needed builder support, > vs. the existing "internal object" mode, > maybe to more easily reuse the existing support for compiling C. > My mode is kind of "internal plus run some quake code". > I guess I could setup linkage so the C backend could call the quake > code w/o the builder knowing about it. > Maybe I should try that, so we can get back to the original set of modes. > > > Basically "external" means "produce cm3cg IR files". > "Internal" means don't produce them. > And then "assembly" means "run assembler after backend" > and "object" means don't run assembler because we already have objects. > > > Whether or not the m3cg IR files are being converted to C and then objects > or gcc internals and then objects, or LLVM internals and then objects, > the builder doesn't need to know or act differently. > It just runs the quake function m3_backend or such, and it decides, > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > Basically, let's say we have n different external backends producing > object files. That doesn't imply any new modes. But some other variable > that affects the quake code that runs cm3cg. > > > Thank you for the confirmation as to which mode you are using. > > > - Jay > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > to do it when you are familiar with the code than refamiliarize later. There were > > already some with comments like "No such back end". Only the StAloneLlvm ones > > have an actual back end. The others are intended for modes that link in at least > > some of llvm into cm3. As discussed, we may not want to do this. > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > on small examples and passing a good few of the compiler tests. Right now, I am working > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > jair:m3core jay$ for a in \ > > > IntLlvmObj \ > > > IntLlvmAsm \ > > > ExtLlvmObj \ > > > ExtLlvmAsm \ > > > StAloneLlvmObj \ > > > StAloneLlvmAsm; \ > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > IntLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > *** > > > > > > IntLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** > > > > > > ExtLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > ExtLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > StAloneLlvmObj > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > => success? > > > > > > except for: > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > StAloneLlvmAsm > > > close, maybe just a config problem > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > RT0.is:5:2: error: unknown directive > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > - Jay > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From dabenavidesd at yahoo.es Mon Sep 14 03:39:21 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 14 Sep 2015 01:39:21 +0000 (UTC) Subject: [M3devel] Missing something? Message-ID: <548758070.2577233.1442194761020.JavaMail.yahoo@mail.yahoo.com> Hi all:I was wondering whether RT machinery is running faster on cross-compiled code in C, or JITed? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 08:14:52 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 06:14:52 +0000 Subject: [M3devel] addition of GPL stuff Message-ID: Um, sorry to bring up such annoying matters, but maybe we should not be licensingnew files as GPL, assuming they aren't derived from GPL files? e.g. the LLVM m3makfiles and Makefiles? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 09:05:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 07:05:21 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F4BFA7.7090805@lcwb.coop> References: , , <55F335F7.9010108@lcwb.coop>, , <55F4BFA7.7090805@lcwb.coop> Message-ID: I agree development never ends.One should always keep in mind active development and not treat it as rare/unusual. The way to get assembly output, I claim, is to fiddle with the config file. For example, add -S or -s on the C compiler command line. I don't believe it is ever useful/required to output assembly for the entire treeusing a C compiler or LLVM. It is only needed very temporarily/modally for focused debugging.Even if development never ends. Unless maybe you are analysing picking up a new LLVM or such? You usually trust the C compiler and LLVM to translate correctlyfrom your output. Usually you only care to look at your output.Therefore -keep. I can be flexible on this.Still, it doesn't require a mode I Think. Yes I have about same meaning of "internal" as you.I guess there are multiple variables: separate process? separate .so/dll? output the cm3cg files? Up until now, they have always coincided. In terms of the builder though, it would not likely do anything different for static linking vs. dynamic linking of LLVM. It should still just make the same function calls. Unless maybe you want to use dlopen/dlsym? Which is really yet another mode "dynamic dynamic". I think it is really just if the cm3cg files are output. I should see if I can eliminate the C mode, in order to "prove"I'm not being hypocritical here. I think it was just for easieraccess to the RunCC or such. I'll try to pass that address tothe C backend and then go back to InternalObject mode.. If C mode is just InternalObject, then probably LLVM should be too.Again, builder should just decide to produce the IR files or not.The other decisions are other variables that Builder doesn't care about.One does have to instantiate the right backend somewhere. It isn't a huge deal though. Builder can temporarily be a mess and we canclean it up later. > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. I think this is just a matter of a few lines of quake. Can you look at what I did for m3cgcat and the C mode? Here: cm3cg.common: if not defined("USE_C_BACKEND_VIA_M3CGCAT") USE_C_BACKEND_VIA_M3CGCAT = FALSEend and then: if USE_C_BACKEND_VIA_M3CGCATM3_BACKEND_MODE = "ExternalObject" proc m3_backend(source, object, optimize, debug) is ret_code = try_exec("m3cgcat", "-in-binary:" & source, "-out-c:" & source & ".c") if not equal(ret_code, 0) return ret_code end return compile_c(source & ".c", object, [ ], FALSE, TRUE)endend I don't remember any problems with this, just that it is slower to output the files. If you wanted to see the C compiler output, here you'd add -S or -s or -FA or such.No need to change the mode.There is something subtle here I'm not explaining well, how this is usefulbut doesn't need to be automated, because it is used rarely and only at small scale. Like, I had no need for a CToAssembly mode. Ultimately "internal" modes are probably always faster, and the draw is strongfor that reason. The C backend is slow enough w/o "external" making it slower. - Jay > Date: Sat, 12 Sep 2015 19:13:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > My inference has been that "internal" means the whole compilation job > is done by code linked in (not necessarily statically) to cm3, e.g., > using the integrated x86 backend. "external" means there are one or > more separate executables to run after the front end, e.g., m3cc > and as, or m3llvm, llc, asm, etc. > > I definitely want the possibility of keeping assembly form for development work, > even when 'as' is run. And we should assume that occasional development work > will continue forever. Disassembling object code will lose sometimes very > useful information, compared to keeping assembly as originally generated as > a side output. > > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. > > On 09/12/2015 06:50 PM, Jay K wrote: > > Those comments were from me, since I went through > > and 1) added the C mode 2) tried to clean up what > > I percieve as rather messy, and still messy. > > > > > > You helped my vocabulary here -- it should be "cartesian factored". > > Rather than every "product" be its own case. > > > > > > I moved it toward that, but it still confusing. > > > > > > Basically you want a set of available steps, and > > then just sequence them, possibly omitting some of them. > > > > > > I can see how you might have roughly 10 choices, > > depending on if you are outputing bitcode or LLVM assembly > > or building LLVM data structures, and linking to LLVM or not, > > and LLVM is producing assembly or object files. > > > > > > I would suggest that LLVM produing assembly is not a useful choice. > > Remove that from the matrix. > > Have it produce object files if you can. > > For debugging, you can fiddle with the switches or dump the files > > afterward. > > > > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > > You can use the "external object" mode plus config file stuff. > > I started that way for C and didn't change the builder initially. > > But it was slower. I don't remember why i needed builder support, > > vs. the existing "internal object" mode, > > maybe to more easily reuse the existing support for compiling C. > > My mode is kind of "internal plus run some quake code". > > I guess I could setup linkage so the C backend could call the quake > > code w/o the builder knowing about it. > > Maybe I should try that, so we can get back to the original set of modes. > > > > > > Basically "external" means "produce cm3cg IR files". > > "Internal" means don't produce them. > > And then "assembly" means "run assembler after backend" > > and "object" means don't run assembler because we already have objects. > > > > > > Whether or not the m3cg IR files are being converted to C and then objects > > or gcc internals and then objects, or LLVM internals and then objects, > > the builder doesn't need to know or act differently. > > It just runs the quake function m3_backend or such, and it decides, > > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > > > > Basically, let's say we have n different external backends producing > > object files. That doesn't imply any new modes. But some other variable > > that affects the quake code that runs cm3cg. > > > > > > Thank you for the confirmation as to which mode you are using. > > > > > > - Jay > > > > > > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > > to do it when you are familiar with the code than refamiliarize later. There were > > > already some with comments like "No such back end". Only the StAloneLlvm ones > > > have an actual back end. The others are intended for modes that link in at least > > > some of llvm into cm3. As discussed, we may not want to do this. > > > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > > on small examples and passing a good few of the compiler tests. Right now, I am working > > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > > > > jair:m3core jay$ for a in \ > > > > IntLlvmObj \ > > > > IntLlvmAsm \ > > > > ExtLlvmObj \ > > > > ExtLlvmAsm \ > > > > StAloneLlvmObj \ > > > > StAloneLlvmAsm; \ > > > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > > > IntLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > *** > > > > > > > > IntLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** > > > > > > > > ExtLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > > > ExtLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > > > StAloneLlvmObj > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > > > => success? > > > > > > > > except for: > > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > > > > StAloneLlvmAsm > > > > close, maybe just a config problem > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > > RT0.is:5:2: error: unknown directive > > > > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 15 10:44:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 15 Sep 2015 08:44:31 +0000 Subject: [M3devel] Target.Aligned_procedures (again) Message-ID: How about this: To check for a closure: 1 Clear the lower bit of the address. Or maybe the lower 2 bits. 2 Check for a 4 byte -1. Not necessarily a full INTEGER. Marker always consumes an INTEGER's worth of space, but only ever check 4 bytes.Clearing the lower bit is strictly to accommodate Thumb -- does it take alignment faults?The conditional branch in the current code, at least for 64bit non-amd64 platforms would be gone.All targets would be the same. Clearing the lower two bits would make the address always aligned.The assumption is that if is a closure, it is already aligned and this is value preserving.If it is not a closure, this still yields a valid accessible address, just not the one to call. Or too risky in terms of portability? Better to check for alignment, assume unaligned is not a closure, and just check for a full INTEGER -1? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:47:39 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:47:39 +0000 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? Message-ID: As I remove Target variation: 1) Just assign the variables all the same in Target.i3/Target.m3?2) Change them to constants in Target.i3?3) Remove them entirely in Target and m3front? i.e. Do you think anyone will ever go back and re-diverge?And need a super local handy guide?Or can go back into history or figure it out anew? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:53:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:53:43 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned Message-ID: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 20:59:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 13:59:55 -0500 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? In-Reply-To: References: Message-ID: <55FF022B.8090101@lcwb.coop> I think it far easier for someone in the future if there is evidence right in the current source, perhaps commented out and preferably commented about, than to try to ferret out history, even when the CM system preserves it. Some future maintainer may not even know that relevant history even exists, let alone where to look. On 09/20/2015 11:47 AM, Jay K wrote: > As I remove Target variation: > > > 1) Just assign the variables all the same in Target.i3/Target.m3? > 2) Change them to constants in Target.i3? > 3) Remove them entirely in Target and m3front? > > > i.e. Do you think anyone will ever go back and re-diverge? > And need a super local handy guide? > Or can go back into history or figure it out anew? > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Sep 20 21:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 14:05:51 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: Message-ID: <55FF038F.8030603@lcwb.coop> On 09/20/2015 11:53 AM, Jay K wrote: > I propose that Allow_packed_byte_aligned be set to FALSE for all targets. > > Or make it const. > > > Or remove it. > Also, removing it altogether from Target.i3 means the value will be copied out the place(s) where it is now used, making it even harder for someone to find. Making it all the same is OK with me, but>>> > > This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths. > Specifically, if you manage to create a packed type with unaligned fields, > presumably loads/stores get converted to multiple aligned loads/stores followed > by putting stuff back together. The occurrence of the unaligned fields should > be very rare in the first place. > Not sure what you are saying, nor whether this is true of all targets, but at least some targets will refuse a packed field that crosses a word boundary, at the point of type definition. The language explicitly permits an implementation to do this. So multi-word loads/stores wouldn't happen anyway. Within a native word, bit-twiddling works fine. > > I does not affect any other target. > > > Granted, x86 and amd64 are the most common targets. > > > Ok? > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 20 22:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 20:41:03 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? Message-ID: containsLazyAlignments m3front/Module.m3: PROCEDURE SetLazyAlignment (on: BOOLEAN) = BEGIN IF curModule # NIL THEN curModule.lazyAligned := on; IF on THEN curModule.containsLazyAlignments := TRUE; END; END; END SetLazyAlignment; PROCEDURE Compile (t: T) = VAR save: T; zz: Scope.T; yy: Revelation.Set; BEGIN (* ETimer.Push (M3Timers.emit); *) Target.Allow_packed_byte_aligned := t.containsLazyAlignments; This feels wrong.Surely, Allow_packed_byte_aligned should be constant for any given target?Surely a pragma cannot change a target's characteristics?Or, rather, I found this by changing it to a constant false for all targets. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 23:00:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 16:00:35 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? In-Reply-To: References: Message-ID: <55FF1E73.1010206@lcwb.coop> Hmm, this sounds like a mechanism we had a discussion of several years ago. I don't remember how it ended, but I think there was a general consensus that it was an idea with some unresolved problems. However, one big class of use cases for packing things is to make it possible to define a record layout that matches one already specified externally, a network protocol packet, for example. We had a much bigger problem with that regarding different endianness, which we never did anything about. I have some unimplemented notes somewhere on how to do opposite-endian record layout. It was mind-boggling, to put it gently, and left me even more negative on so-called "little endian" than previously. Anyway, this is where different rules for different records on the same target may be wanted. On 09/20/2015 03:41 PM, Jay K wrote: > containsLazyAlignments > > m3front/Module.m3: > > PROCEDURE SetLazyAlignment (on: BOOLEAN) = > BEGIN > IF curModule # NIL THEN > curModule.lazyAligned := on; > IF on THEN > curModule.containsLazyAlignments := TRUE; > END; > END; > END SetLazyAlignment; > > > PROCEDURE Compile (t: T) = > VAR save: T; zz: Scope.T; yy: Revelation.Set; > BEGIN > (* ETimer.Push (M3Timers.emit); *) > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > > This feels wrong. > Surely, Allow_packed_byte_aligned should be constant for any given target? > Surely a pragma cannot change a target's characteristics? > Or, rather, I found this by changing it to a constant false for all targets. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Sep 21 04:19:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:19:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , Message-ID: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:22:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:22:25 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Message-ID: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:41:03 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG Message-ID: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:49:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:49:27 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG In-Reply-To: <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> References: , <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> Message-ID: Not gratuitous but I think I found a workaround.I will change the various M3x86/M3C/other. New functions to return M3CG_Ops.Public instead of M3CG.T.So that cm3/M3Backend.m3 can set the additional data fields, and then just returnit as the new same old fully opaque M3CG.T to the rest of cm3. I think a type with very limited visible is about as good as a type with nothing visible. When I was exploring the system, I also found this separation a slightspeed bump into learning. You look at M3CG for something useful and find nothing there.I suppose that is maybe the Modula-3 way, but it seems more obscure than usual.Usually there is just one interface for this sort of construct I think, not two. - Jay Subject: Re: [M3devel] M3CG_Ops vs. M3CG From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:44:10 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Jay, this seems like a gratuitous change. I strongly suggest not for now. Sent from my iPhone On Sep 21, 2015, at 12:41 PM, Jay K wrote: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:51:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:51:08 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> Message-ID: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:58:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:58:42 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> Message-ID: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:00:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:00:44 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, Message-ID: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:06:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:06:44 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, Message-ID: I suppose it is more convenient to have the C mode than I say -- it alsoputs the bootstrap vs. non-bootstrap decision in a convienent place,and the management of keep_files, and the deleting of temporary files. All that logic can be easily moved elsewhere, or not. But then we have like 6 LLVM modes.I set a bad example, and now it is even worse. Arguably we only need 2 modes -- internal and external.Any backend, internal or example, that produces assembly, canrun the assembler itself -- we could expose that in M3CG justlike I want to expose running the C compiler. I like it. Or even just one mode -- call the backend.We just need to provide in M3CG_Ops a function to write out the .mc files,that the backend can call if it needs it. i.e. for the backend call into quake/config.Each backend can put together its specific sequence.There is some commonality currently factored into Builder, but not reallymuch and it is a mess. I'm undecided at the moment. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 02:58:42 +0000 I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:21:09 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:21:09 +0000 Subject: [M3devel] globals vs. instance variables Message-ID: We have: cm3/MODULE M3Backend; ... VAR obj_file : M3ObjFile.T := NIL; obj_wr : Wr.T := NIL; obj_name : TEXT := NIL; log : Wr.T := NIL; log_name : TEXT := NIL; ...PROCEDURE Close (<*UNUSED*> cg: M3CG.T) = BEGIN IF obj_file # NIL THEN TRY NTObjFile.Dump (obj_file, obj_wr); EXCEPT Wr.Failure, Thread.Alerted => Msg.FatalError (NIL, "problem writing object file: ", obj_name); END; Utils.CloseWriter (log, log_name); obj_file := NIL; obj_wr := NIL; obj_name := NIL; log := NIL; log_name := NIL; END; END Close; Surely we should add those variables to M3CG_Ops in orderto get them to Close for this specific M3CG? i.e. prefer "instance variables" over "module variables" aka globals. Perhaps in a record called M3Backend to give an appearance of opacityor perhaps in an actual opaque type, though I hate to pay for theextra heap allocation just to achieve opacity. (Larger point is the cm3 is written an old unfortunate thread-unsafe style with many globals.) And this is a good reason then to smush M3CG.T and M3CG_Ops.T together,in order for M3Backend.New() result to be passed back to Close. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:40:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:40:46 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , Message-ID: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:45:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:45:37 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> Message-ID: There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:41:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:41:38 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> Message-ID: Sorry, clarification: struct A { char a; int b; }; is defined and fairly portable in C, and means generally the same thing onall platforms (ignoring 16bit int), the same thing as: RECORD a: Ctypes.char b: Ctypes.intEND; what is not portable in C is expecting this to have other than size 8. Strictly speaking, its size is completely not portable, but on the vast vast majorityof systems its size is 8, and getting it less requires compiler-specific mechanisms. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu But it us defined in C, just target-specific. Sent from my iPhone On Sep 21, 2015, at 2:00 PM, Jay K wrote: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:56:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:56:00 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:04:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? Message-ID: For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:07:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: Message-ID: Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:21:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:21:47 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: , Message-ID: er, ok, I partly did this. https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3.diff?r1=1.95;r2=1.96;f=u Revision 1.96: download - view: text, markup, annotated - select for diffsThu Apr 1 17:56:37 2010 (5 years, 5 months ago) by jkrellBranches: MAINDiff to: previous 1.95: preferred, unifiedChanges since revision 1.95: +1 -9 lines leave max_align = 64 for all platformsthis effects: I386_FREEBSD / FreeBSD4 I386_LINUX / LINUXLIBC6 NetBSD2_i386 It does not affect any others -- not PPC, not Darwin, not 64bit, not Solaris, not NT. but it wasn't consistent across 32bit platforms or x86.Nor should it necessarily -- I recall Cygwin wants to alignmore than cm3, implying NT does. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: Re: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 16:41:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 14:41:02 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, Message-ID: I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:12 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , Message-ID: I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:53 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu>, Message-ID: I guess it makes sense to be in Builder, just that it can be a separate booland not a mode. The code will almost be identical to today, and not clearlydemonstrate the lack of a need for a mode. Alas. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 04:45:37 +0000 There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:37:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:37:18 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , Message-ID: Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. Perhaps m3front can behave similarly?For all targets? But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 16:31:12 +0000 I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Sep 21 22:25:49 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 21 Sep 2015 16:25:49 -0400 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: <20150921202548.GB20914@topoi.pooq.com> On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > Perhaps m3front can behave similarly?For all targets? > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > - Jay The fancy packing should remain in the language, for backward compatibility if nothing else. modula 3 is, after all, a systems language. But is likely no need to worry it in the bootstrap. If the bootstrap doesn't use it, it will still work portably as a bootstrap, no matter how machine-dependent fancy packing happens to be. -- hendrik From jay.krell at cornell.edu Tue Sep 22 02:15:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <20150921202548.GB20914@topoi.pooq.com> References: , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , <20150921202548.GB20914@topoi.pooq.com> Message-ID: Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 22 06:55:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 04:55:50 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , , , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , , , , , , <20150921202548.GB20914@topoi.pooq.com>, Message-ID: There is a proposal in here..here is a variation of it.. Allow packing of records tightly on all targets. All targets get the same layout (modulo word size), the same things can compile successfully. Remove the caveat here that certain code will only compile for certain targets. If Allow_packed_byte_aligned is false, and there is an unaligned read/write, there are a few choices. m3front could transform into multiple inlined read/writes byte-wise taking the value apart of putting it back together. Or m3front could issue function calls. The C implementation could be #ifdefed such that amd64/x86 just do the read/write, albeit still with a function call. C backend could set allow_packed_byte_aligned the same always. In that case, the functions would be output into the file and inlined or macroed. A warning might be issued about the lack of atomicity. Heck, let's invent a feature...a pragma could be allowed on integral/floating point values as to endianness and compiler could output the byte swaps for you. The endianness could actually be some sort of runtime expression. That's how some formats work -- they get written native, with an indicator as to which that is, and swapped if necessary upon reads, and maybe swapped back for writes. You know -- if you really want a powerful system for efficient binary persistance.. Or just leave it alone. ? - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: Re: [M3devel] Target.Allow_packed_byte_aligned Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Sep 26 21:23:27 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sat, 26 Sep 2015 12:23:27 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150926122327.F8299C6A@m0087794.ppops.net> An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Sep 26 22:30:26 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 16:30:26 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: <20150926203026.GB29515@topoi.pooq.com> Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Sat Sep 26 23:57:46 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 26 Sep 2015 21:57:46 +0000 (UTC) Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926203026.GB29515@topoi.pooq.com> References: <20150926203026.GB29515@topoi.pooq.com> Message-ID: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> not here El S?bado 26 de septiembre de 2015 15:32, Hendrik Boom escribi?: Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Sep 27 01:46:27 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 19:46:27 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> References: <20150926203026.GB29515@topoi.pooq.com> <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> Message-ID: <20150926234627.GB2740@topoi.pooq.com> On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik From lists at darko.org Sun Sep 27 03:27:59 2015 From: lists at darko.org (Darko Volaric) Date: Sat, 26 Sep 2015 18:27:59 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: > In trying to better understand the issue, I see, in Module.m3:895, this > code: > > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > From other places, containsLazyAlignments derives from LAZYALIGN > pragmas, so this means it will turn on Allow_packed_byte_aligned, > regardless of the target, if there is a LAZYALIGN pragma. I think this > will generate code that will alignfault, if the target CPU doesn't > support it. That would be a bug. > > > -Rodney Bates > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:17:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:17:14 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, Message-ID: I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:21:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:21:34 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, , , Message-ID: I forgot to mention/ask: "packed" basically means "no padding", right? - Jay From: jay.krell at cornell.edu To: lists at darko.org; rodney_bates at lcwb.coop Date: Sun, 27 Sep 2015 05:17:14 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:03:20 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:03:20 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927080320.F82B920B@m0087798.ppops.net> An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:10:28 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:10:28 -0700 Subject: [M3devel] HTML mail (was Re: Another Allow_packed_byte_aligned issue) Message-ID: <20150927081028.F82B92BC@m0087798.ppops.net> FWIW, I am traveling and using a webmail that is different from my usual email client. I did find and change a preference setting that sounded like it might switch to plain text. -Rodney Bates --- hendrik at topoi.pooq.com wrote: From: Hendrik Boom To: "m3devel at elegosoft.com" Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue Date: Sat, 26 Sep 2015 19:46:27 -0400 On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Sun Sep 27 17:21:06 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:21:06 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927082106.F82B9288@m0087798.ppops.net> -Rodney Bates --- jay.krell at cornell.edu wrote: From: Jay K To: Darko Volaric , Rodney Bates CC: m3devel Subject: RE: [M3devel] Another Allow_packed_byte_aligned issue Date: Sun, 27 Sep 2015 05:17:14 +0000 I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. I'm just speculating here, but I would think that if it crosses a boundary of the hardware RAM word, (not necessarily the same as a CPU native word) it would be non-atomic anyway, regardless of whether the compiler or the CPU took care of juggling the separate pieces. In fact, I get the impression that, even within a single RAM word, you can't count on atomicity of even an aligned, single-word load or store, with modern multi-CPUs, multi-level hardware caching, etc. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Sep 1 01:44:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 23:44:23 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: <55E34A77.1050001@lcwb.coop> References: , <55E34A77.1050001@lcwb.coop> Message-ID: Agreed -- I forgot to acknowledge/repeat that this is relatedto the first instruction in a function, not arbitrary instructions. On the matter of 4 or 8 bytes though -- the thing is -- many but not all64bit architectures are the exact same instruction set as 32 bits,just a different selection of instructions. Same size, same encodings, same alignment. I believe this is true for alpha, mipa, ppc, sparc, hppa. It is almost true for x86, though that kinda doesn't count,since there is no fixed size or alignment requirement. Fyi: x86/amd64 instructions are between 1 and 15 bytes in size,inclusive, and all sizes in between. All the "lower" sizesincluding 1 are common. Upper sizes are rare, maybe starting around 7 bytes.The encoding scheme allows for longer instructions, butthere is a deliberate documented limit. Not sure about arm -- arm64 at least gets rid of thumb.So arm instructions either 4-aligned or at odd addresses,and arm64 is like all the rest -- fixed size 4 byte aligned. So 4 byte -1 e.g. on ppc32 has the same meaning as 8 byte -1 on ppc64,just that it is two in a row. It looks like we have agreement -- we can set this to a constant. The marker value is -1 I checked. I still find the function If_closure a little unclear/subtle.I read through my comments in Target.m3 again. It is false if: - the system ever has alignment faults - AND functions/instructions might be less aligned than closures Some of the commented I added in the code are incorrect. The comments say that a misaligned function pointer is readone byte at a time. I believe the actual behavior is thata misaligned function pointer is deemed not a closure,and reading the marker is skipped. The code is all still there, but branched around.When aligned_procedures == true, less code is generated,no alignment check is generated. Most 32bit platforms set Aligned_procedures = trueand most 64bit platforms set it to false. The exceptions: amd64 is also true. arm32 is false, due to Thumb mode. Because most platforms, 32bit and 64bit, have fixed size4 byte aligned instructions. So 32bit platforms haveinstructions (and functions) thereforealigned the same as closures. 64bit platforms generallyhave closures more aligned than instructions. I think if we ignored arm32, and penalized only amd64,which granted is the most common platform,we could set aligned_procedures = false for all 64bit,true for all 32bit. And having another variablethat coincided with word size is ok. But arm32's thumb instructions mean a function pointermight be unaligned. I'll make it always false. > Method body procedures are required to be top-level Kind of a language limitation compared to "more dynamic" languages. Definitely not trivial to do otherwise -- heap allocated garbage collected stack frames and such.. Thank you, - Jay > Date: Sun, 30 Aug 2015 13:24:55 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Aligned_procedures and closure markers? > > > > On 08/30/2015 02:45 AM, Jay K wrote: > > The agenda remains seeing if Target variables can be made constants. > > > > The discussion in this case is more complicated and some facts are unclear. > > > > > > Background: > > > > > > Nested functions are a problem. > > In particular, if you can take their address. > > Taking the address of a nested function presents a problem. > > You are presented with two or three solutions. > > > > > > - runtime code gen > > - either on the stack > > > > - or somewhere more expensive, possibly with garbage collection > > > > - possibly "templated" instead of "arbitrary"; the meaning > > of this is a lot to explain -- related to libffi and mremap, which > > isn't entirely portable, but is e.g. portable to Windows > > > > > > - or instead of runtime codegen, altering how function pointers are > > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > > The sequence is roughly: > > Check if it is a "regular" function pointer or a "closure". > > If it is a "regular" function pointer, just call it. > > If it is a "closure", it contains a function pointer and a "static link". > > Call the function pointer, passing the static link. > > > > > > To tell if it is "regular" function pointer or a "closure", roughly > > what is done is the data at the function pointer is read and compared > > against a marker value. If it equals the marker value, it is a closure. > > > > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > > The value of the marker checked for is 0 or -1, I'd have to check. > > The alignment of the pointer might be a factor. In particular, on most > > architectures, all instructions have a certain alignment. If the pointer has > > less alignment, it can't be an instruction. Or maybe on those architectures, > > the bytes are read one at a time to avoid alignment faults. > > > > > > In particular, as far as I know, the following: > > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > > 4 bytes and 4 byte aligned, so functions are at least also as much > > > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > > pointer is actually odd as well, and the low bit is removed to really find the instructions > > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > > > > I could use confirmation on much of this. > > > > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > > value that works on all. > > > > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > > then checking for two in a row reduces the odds of a false positive. > > > > > > However, given that the closure is a marker and two pointers, it isn't like you are going > > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > > > > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then > it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part > of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it > by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the > space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a > valid instruction or sequence thereof. > > So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized > alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity > here. > > Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. > It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for > certain (especially if we want to be able to call procedures produced by other compilers) but much less > likely to result in a problem. > > Just a wild guess, but I would not be surprised if ELF and other object formats would require the > machine code of a function/procedure to begin on a native word boundary, even if the hardware > instruction set does not. Where so, this would obviate checking the alignment before checking > for a closure, though probably target-dependently. > > > > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > > > > However, I want less target-variation not more. > > > > > > Here are some my lingering questions: > > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > > - Is a 64bit marker value actually sufficient on IA64? > > The way to help here, I think, is to ensure that a 64bit marker, > > not a 128bit marker, contains the "template", and an invalid "template". > > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > > > > i believe all of these function pointers are rare. > > I hope/believe the object method calls do not check for closures -- though actually > > that is related to a useful language construct, that I doubt we have. > > > > Method body procedures are required to be top-level, ensured statically, so there is no > need for method call code to consider the possibility of the pointer in the object type > to be a closure. > > > > > The simplest solution is likely: > > - ignore IA64, or research it further > > - keep marker size at integer > > Pointer would be target-independent in getting the following two pointers aligned. > > > - for the C backend, assume no alignment of function pointers -- give up > > any of the optimization, esp. x86/amd64. > > I think this optimization both applies to a low-frequency situation and has a very > small benefit, so I would not worry about giving up on it. > > > > > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > > > > Thoughts? > > > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 07:52:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 05:52:16 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816134306.GB5008@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com> Message-ID: old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd "stress test" my output.Digital Mars might be another.And yes the Intel compiler -- which is a proxy for a set of compilers -- the EDG frontend.(There are approx four C++ frontends in practical use these days: Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front end.) - Jay > Date: Sun, 16 Aug 2015 13:43:06 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > > Autoconf output has little/none dependency.Sun's SPARC optimization are > > mostly irrelevant as we have little C code,unless you use the C backend. > > Ok, well, it sounds like it will be increasingly important as you go to your > C backend though. > > > > somewhat of a test of C portability > > This I appreciate. We have some C code and the C backend.More compilers > > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > > now clang -- having found and worked around a bug in its assembler.Some > > time soon I'll expand to Metrowerks and Digital Mars.. > > I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have > the latest versions. C/C++ are not my thing but if I can help by just > building stuff to see what messages we get or if it breaks etc. let me know > and I will try to participate. > > I didn't know Metrowerks was still around. I thought they got swallowed up > by Motorola and then removed from all the non-embedded space. > > > > - Jay > > > > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > > From: microcode at zoho.com > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] Build Server - Plan > > > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > > The output of autoconf/automake should have lightweight dependencies.They > > > > might stress make, might require GNU make.They might stress the sh, but I > > > > think there are adequate shells out there. > > > > > > That is typically one issue with autoconf, requiring gnu tools in the > > > path. On Solaris this can be annoying since most Solaris people don't use > > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > > are up to date or even current by any stretch of the imagination. > > > > > > > They are meant to be easy for people building stuff to use.They aren't > > > > meant to be easy for people developing stuff to use. > > > > > > Not sure what you meant here... > > > > > > > Look at this way..while people complain and there are widelyused > > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > > Aix, Irix, etc. > > > > > > They often don't "work" for Solaris as-installed but they can most often be > > > made to work. Increasingly, as automake and its prereqs get version bumps > > > there are problems building apps on Solaris because Solaris installs with a > > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > > ran into a problem with the last year where Solaris awk was not good enough > > > to install(!) an app that compiled on Solaris as part of autotools so I had > > > to download a newish copy of gawk and install it. So really, it is much > > > better not to go there if you can avoid it. > > > > > > I am not suggesting an alternative to autotools but just pointing out it is > > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > > Linux. I also run BSD on several platforms and because of the same issues as > > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > > shells, etc. not being good enough for autotools) it is sometimes > > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > > no good/easy solution to this, just to point out it is not the slam-dunk as > > > might be thought. > > > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > > buildingthat with autoconf/automake. > > > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > > is in a non-standard location and is often not used. Ideally, apps to be run > > > on Solaris should be able to be built with native (non-gnu) tools. The > > > Studio compilers are very good and have optimizations for SPARC that should > > > be better than what gcc can provide and it is also somewhat of a test of C > > > portability since Studio doesn't necessarily provide all non-standard gcc > > > extensions. > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 08:22:24 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 06:22:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> <20150816134306.GB5008@zoho.com> Message-ID: <20150901062224.GA3050@zoho.com> On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > "stress test" my output.Digital Mars might be another.And yes the Intel > compiler -- which is a proxy for a set of compilers -- the EDG > frontend. Hi, You lost me here. What is EDG? As far as I know, the Intel compiler is a proprietary (and complete, self-contained) product of Intel. I think I read on Intel's forums in a post by Steve Lionel that earlier versions were based on code that made its way from DEC to COMPAQ but was later reworked so that it is almost all new. An interesting tangent: I ran some tests earlier and can't remember for sure but it seems to me there is some Open64 code in Intel's compilers. Whether Intel wrote it and it made its way to Open64, or whether it was part of Open64 and Intel took some of it, I do not know. > (There are approx four C++ frontends in practical use these days: > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > end.) From jay.krell at cornell.edu Tue Sep 1 08:23:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:23:42 +0000 Subject: [M3devel] m3front -unfold_nested_procs Message-ID: It'd be nice, if someone has the time, to let -unfold_nested_procs work with cm3cg.I tried just silently ignoring it, but then it crashes. -unfold_nested_procs is required by the C backend.I and I think the NTx86 backend. And required to be omitted by the gcc/cm3cg backend. Ideally, even if they aren't ABI compatible (due to how static link is passed,unfortunate, I wish I could fix it), they could at least operate on identical M3 IR.To ease moving between them. And granted, it'd be nice if the C backend didn't care. I think I can do that,by moving the functions around afterward.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 08:27:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:27:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150901062224.GA3050@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com>, , <20150901062224.GA3050@zoho.com> Message-ID: edg.comWhile almost nobody writes their own C++ compiler, even fewer people write their own C++ frontend.Edison Design Group.A small company whose main/only product is a C++ front end that they license in sourceform to people who are interested in, say, writing a backend only. - Jay > Date: Tue, 1 Sep 2015 06:22:24 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > > "stress test" my output.Digital Mars might be another.And yes the Intel > > compiler -- which is a proxy for a set of compilers -- the EDG > > frontend. > > Hi, > > You lost me here. What is EDG? > > As far as I know, the Intel compiler is a proprietary (and complete, > self-contained) product of Intel. I think I read on Intel's forums in a post > by Steve Lionel that earlier versions were based on code that made its way > from DEC to COMPAQ but was later reworked so that it is almost all new. > > An interesting tangent: I ran some tests earlier and can't remember for sure > but it seems to me there is some Open64 code in Intel's compilers. Whether > Intel wrote it and it made its way to Open64, or whether it was part of > Open64 and Intel took some of it, I do not know. > > > (There are approx four C++ frontends in practical use these days: > > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > > end.) > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 09:18:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 07:18:02 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] 4537cb: remove unused locals In-Reply-To: References: <55e53db0990a0_3be33fd3fe0af2a068547@hookshot-fe2-cp1-prd.iad.github.net.mail>, <7C1D0FBD-5C9A-4883-A47C-57E613FA171E@purdue.edu>, <4E019C37-773B-42AE-AC49-F53C52249195@purdue.edu>, <2E8C2B23-F814-46FF-A59A-C92F486EC273@purdue.edu>, Message-ID: I use git pull to update my local repository to be "like" github.I use git commit to commit my local edits to my local repository.It use git push to push my local repository changes to the master.I don't know what rebasing is or how/why to use it. - Jay Subject: Re: [M3commit] [modula3/cm3] 4537cb: remove unused locals From: hosking at purdue.edu Date: Tue, 1 Sep 2015 17:10:51 +1000 CC: jay.krell at cornell.edu; m3commit at elegosoft.com; m3devel at elegosoft.com To: hosking at purdue.edu It appears to be this merge action which is causing the problem.I don?t have time to explore right now. If anyone else can it would be most appreciated. On 1 Sep 2015, at 5:08 pm, Antony Hosking wrote:Whatever you are doing I now don?t know how to recover from.It appears that somehow you are rebasing the history on the master branch, but I don?t quite know how. On 1 Sep 2015, at 5:00 pm, Antony Hosking wrote:In your work flow are you using "git push" to push your local commits? Or are you doing something else? On 1 Sep 2015, at 4:58 pm, Antony Hosking wrote:Jay, it appears that your commits are still messing up the history. As of your most recent commit we have lost history again. What exactly are you doing when you use git against the repo? For instance, if you take a look at RTCollector we see a history with the oldest commit labelled "initial diff from ../config/PPC_LINUX" from Jan 4, 2008. Whereas it should have the oldest commit labeled as follows from when Olaf first imported from cvs to svn. commit cfac2c79141d61c291a40177f0d5490f774c032fAuthor: Olaf Wagner Date: Wed Jan 24 12:24:49 2001 +0000 This commit was generated by cvs2svn to compensate for changes in r20, which included commits to RCS files with non-trunk default branches. On 1 Sep 2015, at 3:54 pm, jaykrell wrote: Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 4537cb804fbf864a076235495db77eebf9c5dab6 https://github.com/modula3/cm3/commit/4537cb804fbf864a076235495db77eebf9c5dab6 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- remove unused locals Commit: 477c3d058a95ba8a06c90ac5d36dceefd3e7f39a https://github.com/modula3/cm3/commit/477c3d058a95ba8a06c90ac5d36dceefd3e7f39a Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- Fix regression from "7/31 Initial integration of llvm back end into cm3's build system." when bootstrapping using cm3cg. .c files pass through for bootstrap, along with any .h or checked in .s/.o etc. bootstrapping is using cm3 -boot. Commit: 459512da7848b534636c1fe4e1095eee76ad2d89 https://github.com/modula3/cm3/commit/459512da7848b534636c1fe4e1095eee76ad2d89 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: A m3-sys/llvm3.6.1/src/LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 A m3-sys/llvm3.6.1/src/Main.m3 A m3-sys/llvm3.6.1/src/m3makefile Log Message: ----------- Merge branch 'master' of github.com:modula3/cm3 pull from master Compare: https://github.com/modula3/cm3/compare/b068a306fe87...459512da7848_______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 18:28:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? Message-ID: I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 19:00:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 17:00:07 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: Sorry, this merits more discussion. - There are at least three ways compilers chose between C and C++. invocation name: gcc vs. g++, lowercase cc vs. uppercase CC I think clang vs. clang++. Visual C++ only has one invocation: cl Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. Command line switches: gcc -x c++ clang -x C++ cl -Tc means next source file is C -TC means all are C -Tp means next is C++ -TP means all are C++ We'd have to read up on others. One of the combinations implemented by gcc is deprecated by clang. I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). So I switched to use -x c++. "cpp" is ambiguous between C preprocessor and C++. "cxx" is often therefore used to indicate C++. x is a rotated +. We could have config files specify SYSTEM_CC and SYSTEM_CXX. And we could have functions compile_c and compile_cxx. We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe others, error if there are multiple, and chose compile_c vs. compile_cxx. Or compile_c could chose based on the extension. In any case, makes changes to the C backend path, and in the llvm binding/backend directories should be easy. We don't *have* to change the meanings everywhere to effect them. I do not actually intend to rewrite all of our little bits of C using C++ specific features. But the LLVM stuff uses C++ and the C backend imho ought to. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 19:08:58 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 17:08:58 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <20150901170858.GA6140@zoho.com> Since you or somebody else mentioned autotools, I think the way this is normally handled is for the script to use cc or CC and it expects you to set those values in your shell to whatever compiler you want to use. On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > Sorry, this merits more discussion. > > - There are at least three ways compilers chose between C and C++. > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > I think clang vs. clang++. > Visual C++ only has one invocation: cl > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > Command line switches: > gcc -x c++ > clang -x C++ > cl -Tc means next source file is C > -TC means all are C > -Tp means next is C++ > -TP means all are C++ > We'd have to read up on others. > > One of the combinations implemented by gcc is deprecated by clang. > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > So I switched to use -x c++. > > "cpp" is ambiguous between C preprocessor and C++. > "cxx" is often therefore used to indicate C++. x is a rotated +. > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > And we could have functions compile_c and compile_cxx. > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > others, error if there are multiple, and chose compile_c vs. compile_cxx. > Or compile_c could chose based on the extension. > > In any case, makes changes to the C backend path, and in the llvm binding/backend > directories should be easy. We don't *have* to change the meanings everywhere > to effect them. I do not actually intend to rewrite all of our little bits > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > imho ought to. > > - Jay > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Tue, 1 Sep 2015 16:28:10 +0000 > Subject: [M3devel] gcc vs. g++ in config files? > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- From jay.krell at cornell.edu Tue Sep 1 20:02:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 18:02:27 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: <20150901170858.GA6140@zoho.com> References: , , <20150901170858.GA6140@zoho.com> Message-ID: For now we aren't using autotools. I remain on the fence. I want some of what they do, but they really are fairly ugly and slow and overkill.I kinda want some other small portable scripting language.The way they find the C compiler is pretty simple really.Maybe python. Embedding it and adding functions? I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? libtool supports more systems than our config files, but is alsokinda ugly/overkill/slow and somewhat philosophically poor.For example, the fact that they default to compiling everything twice,pic and non-pic, bothers me. Surely it is not worth a) the double timeto build b) the slight loss of security in executables.All code should just be pic.They do make it an option, but when people foist that default decisionon the masses, it makes me wonder.Apple has no such modes.Windows has no such modes (though Windows doesn't do quitewhat I want either). In particular, I want someone else to - find C and C++ compiler -- but what autotools does here is pretty simple - pick out int8, int16, int32, int64 for me, with greater portability to older and newer systems -- but there is inttypes.h on most systems anyway - figure out how to link -- for static there is just one portable way (I adjusted the config files recently), but for shared, libtool or cmake do it better?easier? - generate portable makefiles for bootstrapping (or for regular building?) Or just GNU makefiles? That is one some guidance I've seen -- give up on portable makefiles and just use GNU make. cmake?? I don't want to lose the fine grained incrementality the integrated builder has,but it is tempting to make the system just generate makefiles. I wonder which parts of the system are worth maintaining, vs. giving in to the largerecosystem of stuff being maintained for us. cm3 is ahead of its time for 20 years ago but these days seems largely redundant.Perhaps we can/should whittle it down to a more unique and worthwhile core?I don't know. - Jay > Date: Tue, 1 Sep 2015 17:08:58 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] gcc vs. g++ in config files? > > Since you or somebody else mentioned autotools, I think the way this is > normally handled is for the script to use cc or CC and it expects you to > set those values in your shell to whatever compiler you want to use. > > On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > > Sorry, this merits more discussion. > > > > - There are at least three ways compilers chose between C and C++. > > > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > > I think clang vs. clang++. > > Visual C++ only has one invocation: cl > > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > > Command line switches: > > gcc -x c++ > > clang -x C++ > > cl -Tc means next source file is C > > -TC means all are C > > -Tp means next is C++ > > -TP means all are C++ > > We'd have to read up on others. > > > > One of the combinations implemented by gcc is deprecated by clang. > > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > > So I switched to use -x c++. > > > > "cpp" is ambiguous between C preprocessor and C++. > > "cxx" is often therefore used to indicate C++. x is a rotated +. > > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > > And we could have functions compile_c and compile_cxx. > > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > > others, error if there are multiple, and chose compile_c vs. compile_cxx. > > Or compile_c could chose based on the extension. > > > > In any case, makes changes to the C backend path, and in the llvm binding/backend > > directories should be easy. We don't *have* to change the meanings everywhere > > to effect them. I do not actually intend to rewrite all of our little bits > > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > > imho ought to. > > > > - Jay > > > > > > > > > > From: jay.krell at cornell.edu > > To: m3devel at elegosoft.com > > Date: Tue, 1 Sep 2015 16:28:10 +0000 > > Subject: [M3devel] gcc vs. g++ in config files? > > > > > > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Sep 1 20:32:00 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 1 Sep 2015 14:32:00 -0400 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: <20150901170858.GA6140@zoho.com> Message-ID: <20150901183200.GA10167@topoi.pooq.com> On Tue, Sep 01, 2015 at 06:02:27PM +0000, Jay K wrote: ... ... > I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? It might be scons you're looking for. It's a build system that's made out of python. It is not a plug-compatible make replacement. It is completely different. -- hendrik From rodney_bates at lcwb.coop Wed Sep 2 17:32:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 02 Sep 2015 10:32:03 -0500 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <55E71673.4010508@lcwb.coop> On 09/01/2015 11:28 AM, Jay K wrote: > I suggest we replace "gcc" with "g++". > In all the config files (and in the bootstrap archives). > And similar for Sun CC, and Visual C++ use -TP. > > > rationale: > It will likely cleanup our LLVM stuff, which currently > uses a custom Makefile and drops derived files into the src directory. There are a couple of problems here. I initially tried putting compiled C/C++ code in the build directory. Since it is target-dependent, it obviously does not belong in src. But a bug in the builder just wipes out the file with a circular symbolic link: % import_obj needs work!! % import_obj() puts a symlink in ../BUILD_DIR/.o, pointing to , % located relative to the src directory. Later, it links the symlink into the package. % If the file is target-dependent, (as these and most .o files are), we want to put them % in ../BUILD_DIR, the target build directory, and only target-dependent place we have. % But import_obj("../BUILD_DIR/") just replaces it with a circular symlink. % When this is fixed, the Makefile will have to be changed to put it in the right place. The bug shouldn't be hard to fix, it just hasn't gotten to the top of my list. Then the .o files can be moved to the build directory. But that's still using make to build the .o files. The other problem is the cm3 build system. It's really quite sophisticated about dependencies within Modula3 code, but things like 'c_source' don't, as far as I can see, do any dependency analysis at all. They work fine for the odd .c and .h files here and there in the cm3 tree. But the bindings to llvm (the parts we must write in C & C++) bring in large #include closures from the llvm source tree and then link in several compiled llvm libraries. I don't think cm3's build system is at all up to this. So I used make for that. Actually, as far as I can find, llvm has not provided any dependency information about what libraries will be needed, in what order (which, BTW, undergoes reversal of certain segments). So far, it has been a very dicey and frustrating game of wild guesses and experiments. I am sure the lists I have come up with must have some undetected unnecessary things. I haven't even figured out a way to find out what library a link name declaration is in. > It will ease/allow using C++ exception handling in the C backend. > I've been using g++ for months/years on Darwin. > counter-rationale: > Not all C code is valid C++, and sometimes the meanings are different. > I converted the coverage code only recently. > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Sep 3 01:56:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 2 Sep 2015 23:56:06 +0000 Subject: [M3devel] build_standalone errors Message-ID: import_lib("dl", "/usr/lib")import_lib("ncurses", "/usr/lib") + build_standalone() => on Darwin, where is only /usr/lib/libdl.dylib and /usr/lib/libncurses.dylb. clang: error: no such file or directory: '/usr/lib/libncurses.a'clang: error: no such file or directory: '/usr/lib/libdl.a' build_standalone should probably be much smarterand at least probe file existance; if found specify it;if not found use -lfoo. The idea being to be standalone with respect to available static libs,not all libs. More so, really, stand-aloneness should be specifiable on a libraryby library, or directory basis. This is while building m3-sys/llvm, making it standalone, whichI expect *might* be desired. I'll go ahead without standalone. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 4 23:48:42 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 04 Sep 2015 16:48:42 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail> Message-ID: <55EA11BA.7010704@lcwb.coop> Hmm. As near as I remember, I assumed the front end was unnesting calls just because it did it on one example (using m3cc). Looking at the code, I see command line options -nested_calls and -no_nested_calls (the default). However, the only use of this I can find affects whether an independent check by a M3CG_Check.T instance verifies that there is no nesting. So it looks like it's actually unconditional after all. I'll leave the comments as they are unless this changes. I would expect a back end not to have to cope with them, unless it was generating final code for a bytecode or stack-like machine that could directly handle them nested. On 09/03/2015 05:38 PM, Jay wrote: > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > - Jay > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > >> Branch: refs/heads/master >> Home: https://github.com/modula3/cm3 >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/m3middle/src/M3CG_Ops.i3 >> >> Log Message: >> ----------- >> Comment no nested calls occur in cm3 IR. >> >> Comment the fact that nested calls in source code are unnested in >> cm3 intermediate representation. This is an essential invariant >> to both producers and consumers of this IR. >> >> Changes to be committed: >> >> modified: src/M3CG_Ops.i3 >> >> >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Revert an incorrect fix due to misdiagnosis. >> >> The actual problem here was not a lack of a set_source_file, >> it was reading binary cm3 IR with the ascii reader. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 >> >> Lots of changes happened to this file in package llvm, while the version >> in package llvm3.6.1 has been undergoing integration with the greatly >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 >> found in package llvm3.6.1. Also some random manual review and fixes. >> >> Compiles and runs to completion, with superficially believable output, >> on one small test case involving separate temporaries in and out of >> a FINALLY procedure. >> >> Works with cm3 command, when its executable m3llvm has been shipped. >> >> The test compile fails to link, not finding alloca. m3llvm needs to >> detect calls on alloca and translate internally into llvm alloca >> instructions, rather than letting them through to be linked in. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Sep 5 03:51:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 5 Sep 2015 01:51:16 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: <55EA11BA.7010704@lcwb.coop> References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail>, , <55EA11BA.7010704@lcwb.coop> Message-ID: -unfold_nested_procs is what I'm referring to. Sorry, I comprehend better now -- "procs" vs. "calls". nested call: foo(bar()) unnested call: temp = bar() foo(temp) nested proc: void foo(){ void bar() { }} unnested proc: void foo(){ } void bar(){} And then everything I said. :)I want fewer variables and more constants. :)I want one (or fewer) persisted form of the IR and an ability for a backend to easily transform.I need to push the M3CG_MultiPass stuff around more I think.Then again, the IR doesn't tend to live long and there isn't likely a cause for it to, so variation is ok.The only time it lives long is when iterating tightly on a backend.It could be useful for bootstrapping perhaps, but I think we have a better option already -- C/C++. - Jay > Date: Fri, 4 Sep 2015 16:48:42 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. > > Hmm. As near as I remember, I assumed the front end was unnesting calls just because it > did it on one example (using m3cc). Looking at the code, I see command line options > -nested_calls and -no_nested_calls (the default). However, the only use of this I can > find affects whether an independent check by a M3CG_Check.T instance verifies that > there is no nesting. So it looks like it's actually unconditional after all. I'll > leave the comments as they are unless this changes. > > I would expect a back end not to have to cope with them, unless it was generating > final code for a bytecode or stack-like machine that could directly handle them nested. > > On 09/03/2015 05:38 PM, Jay wrote: > > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > > > - Jay > > > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > > > >> Branch: refs/heads/master > >> Home: https://github.com/modula3/cm3 > >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 > >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/m3middle/src/M3CG_Ops.i3 > >> > >> Log Message: > >> ----------- > >> Comment no nested calls occur in cm3 IR. > >> > >> Comment the fact that nested calls in source code are unnested in > >> cm3 intermediate representation. This is an essential invariant > >> to both producers and consumers of this IR. > >> > >> Changes to be committed: > >> > >> modified: src/M3CG_Ops.i3 > >> > >> > >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 > >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Revert an incorrect fix due to misdiagnosis. > >> > >> The actual problem here was not a lack of a set_source_file, > >> it was reading binary cm3 IR with the ascii reader. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 > >> > >> Lots of changes happened to this file in package llvm, while the version > >> in package llvm3.6.1 has been undergoing integration with the greatly > >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 > >> found in package llvm3.6.1. Also some random manual review and fixes. > >> > >> Compiles and runs to completion, with superficially believable output, > >> on one small test case involving separate temporaries in and out of > >> a FINALLY procedure. > >> > >> Works with cm3 command, when its executable m3llvm has been shipped. > >> > >> The test compile fails to link, not finding alloca. m3llvm needs to > >> detect calls on alloca and translate internally into llvm alloca > >> instructions, rather than letting them through to be linked in. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 > >> _______________________________________________ > >> M3commit mailing list > >> M3commit at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Sep 11 03:53:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 11 Sep 2015 01:53:36 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone Message-ID: Here is what I get: some "crash", some "fail", some get the file format mixed up, some work!6 modes. 4 crash, 1 appears to work, 1 appears close jair:m3core jay$ for a in \IntLlvmObj \IntLlvmAsm \ExtLlvmObj \ExtLlvmAsm \StAloneLlvmObj \StAloneLlvmAsm; \ do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done IntLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3*** IntLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** ExtLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 ExtLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL StAloneLlvmObjm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io => success? except for:error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) StAloneLlvmAsmclose, maybe just a config problemm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.isfg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.ioRT0.is:5:2: error: unknown directive I assume we should ignore Int and Ext and focus on StAlone for now. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 11 22:13:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 11 Sep 2015 15:13:43 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: Message-ID: <55F335F7.9010108@lcwb.coop> Yes, I put in all the modes I thought of while working in there, as it's a lot easier to do it when you are familiar with the code than refamiliarize later. There were already some with comments like "No such back end". Only the StAloneLlvm ones have an actual back end. The others are intended for modes that link in at least some of llvm into cm3. As discussed, we may not want to do this. The only mode I have tried is StAloneLlvmAsm. This is running to compile completion on small examples and passing a good few of the compiler tests. Right now, I am working on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. On 09/10/2015 08:53 PM, Jay K wrote: > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > jair:m3core jay$ for a in \ > IntLlvmObj \ > IntLlvmAsm \ > ExtLlvmObj \ > ExtLlvmAsm \ > StAloneLlvmObj \ > StAloneLlvmAsm; \ > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > IntLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > *** > > IntLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** > > ExtLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > ExtLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > > StAloneLlvmObj > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > => success? > > except for: > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > StAloneLlvmAsm > close, maybe just a config problem > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > RT0.is:5:2: error: unknown directive > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 13 01:26:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:26:35 +0000 Subject: [M3devel] builder behavior upon error -- subsequent files not fully compiled Message-ID: fyi, I observe the following: When compiling multiple files, and one fails, the failure inappropriately affects later files, in that, one or more of: - all outputs are deleted - or -keep isn't honored - or the "build pipeline" is truncated, i.e. we might check the validity of .m3/.i3 files but we don't progress all the way to generate assembly/object files It used to be worse, we just discussed, I had regressed it years ago by accident, in that compilation stopped and no further errors are reported. But now it can largely succeed, but not have the intended outputs, so you fix the one error, and then recompile everything. There is just one boolean "compile_failed" that is set by the first failure, and left set, and the code doesn't know if it was from an earlier file or the current file. It either needs to be a counter, and captured before certain steps and checked for > after them, or there should be two booleans, one that is set/cleared per file, one that is set whenever the first becomes set and is never cleared, or functions should return a boolean, for the current file, and either also set the "global", or set the global near the top of the call tree. It isn't global, but it is nearly so. Making this code multithreaded will be an unfortunately large task.. I'll probably fix this soon. This is just a heads up. I'm not addressing the single-threadedness, just the other. It shouldn't be a big change. This is related to: What I'm actually seeing with one of the LLVM passes is the first invocation of some tool crashes, so the rest truncate their build pipeline and claim m3front failed, which isn't true, either it didn't run or didn't fail. The failure is from the first file. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 13 01:50:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:50:08 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F335F7.9010108@lcwb.coop> References: , <55F335F7.9010108@lcwb.coop> Message-ID: Those comments were from me, since I went through and 1) added the C mode 2) tried to clean up what I percieve as rather messy, and still messy. You helped my vocabulary here -- it should be "cartesian factored". Rather than every "product" be its own case. I moved it toward that, but it still confusing. Basically you want a set of available steps, and then just sequence them, possibly omitting some of them. I can see how you might have roughly 10 choices, depending on if you are outputing bitcode or LLVM assembly or building LLVM data structures, and linking to LLVM or not, and LLVM is producing assembly or object files. I would suggest that LLVM produing assembly is not a useful choice. Remove that from the matrix. Have it produce object files if you can. For debugging, you can fiddle with the switches or dump the files afterward. If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. You can use the "external object" mode plus config file stuff. I started that way for C and didn't change the builder initially. But it was slower. I don't remember why i needed builder support, vs. the existing "internal object" mode, maybe to more easily reuse the existing support for compiling C. My mode is kind of "internal plus run some quake code". I guess I could setup linkage so the C backend could call the quake code w/o the builder knowing about it. Maybe I should try that, so we can get back to the original set of modes. Basically "external" means "produce cm3cg IR files". "Internal" means don't produce them. And then "assembly" means "run assembler after backend" and "object" means don't run assembler because we already have objects. Whether or not the m3cg IR files are being converted to C and then objects or gcc internals and then objects, or LLVM internals and then objects, the builder doesn't need to know or act differently. It just runs the quake function m3_backend or such, and it decides, via a separate factor from mode to run llvm or cm3cg on the IR file. Basically, let's say we have n different external backends producing object files. That doesn't imply any new modes. But some other variable that affects the quake code that runs cm3cg. Thank you for the confirmation as to which mode you are using. - Jay > Date: Fri, 11 Sep 2015 15:13:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > to do it when you are familiar with the code than refamiliarize later. There were > already some with comments like "No such back end". Only the StAloneLlvm ones > have an actual back end. The others are intended for modes that link in at least > some of llvm into cm3. As discussed, we may not want to do this. > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > on small examples and passing a good few of the compiler tests. Right now, I am working > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > On 09/10/2015 08:53 PM, Jay K wrote: > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > jair:m3core jay$ for a in \ > > IntLlvmObj \ > > IntLlvmAsm \ > > ExtLlvmObj \ > > ExtLlvmAsm \ > > StAloneLlvmObj \ > > StAloneLlvmAsm; \ > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > IntLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > *** > > > > IntLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** > > > > ExtLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > ExtLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > > > StAloneLlvmObj > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > => success? > > > > except for: > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > StAloneLlvmAsm > > close, maybe just a config problem > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > RT0.is:5:2: error: unknown directive > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 13 02:13:27 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 12 Sep 2015 19:13:27 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: , <55F335F7.9010108@lcwb.coop> Message-ID: <55F4BFA7.7090805@lcwb.coop> My inference has been that "internal" means the whole compilation job is done by code linked in (not necessarily statically) to cm3, e.g., using the integrated x86 backend. "external" means there are one or more separate executables to run after the front end, e.g., m3cc and as, or m3llvm, llc, asm, etc. I definitely want the possibility of keeping assembly form for development work, even when 'as' is run. And we should assume that occasional development work will continue forever. Disassembling object code will lose sometimes very useful information, compared to keeping assembly as originally generated as a side output. Actually, I'd also like to be able to set it up so m3cgcat gets run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It would just save a couple of manual steps that I am typing over and over, slowing down the debug process. On 09/12/2015 06:50 PM, Jay K wrote: > Those comments were from me, since I went through > and 1) added the C mode 2) tried to clean up what > I percieve as rather messy, and still messy. > > > You helped my vocabulary here -- it should be "cartesian factored". > Rather than every "product" be its own case. > > > I moved it toward that, but it still confusing. > > > Basically you want a set of available steps, and > then just sequence them, possibly omitting some of them. > > > I can see how you might have roughly 10 choices, > depending on if you are outputing bitcode or LLVM assembly > or building LLVM data structures, and linking to LLVM or not, > and LLVM is producing assembly or object files. > > > I would suggest that LLVM produing assembly is not a useful choice. > Remove that from the matrix. > Have it produce object files if you can. > For debugging, you can fiddle with the switches or dump the files > afterward. > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > You can use the "external object" mode plus config file stuff. > I started that way for C and didn't change the builder initially. > But it was slower. I don't remember why i needed builder support, > vs. the existing "internal object" mode, > maybe to more easily reuse the existing support for compiling C. > My mode is kind of "internal plus run some quake code". > I guess I could setup linkage so the C backend could call the quake > code w/o the builder knowing about it. > Maybe I should try that, so we can get back to the original set of modes. > > > Basically "external" means "produce cm3cg IR files". > "Internal" means don't produce them. > And then "assembly" means "run assembler after backend" > and "object" means don't run assembler because we already have objects. > > > Whether or not the m3cg IR files are being converted to C and then objects > or gcc internals and then objects, or LLVM internals and then objects, > the builder doesn't need to know or act differently. > It just runs the quake function m3_backend or such, and it decides, > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > Basically, let's say we have n different external backends producing > object files. That doesn't imply any new modes. But some other variable > that affects the quake code that runs cm3cg. > > > Thank you for the confirmation as to which mode you are using. > > > - Jay > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > to do it when you are familiar with the code than refamiliarize later. There were > > already some with comments like "No such back end". Only the StAloneLlvm ones > > have an actual back end. The others are intended for modes that link in at least > > some of llvm into cm3. As discussed, we may not want to do this. > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > on small examples and passing a good few of the compiler tests. Right now, I am working > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > jair:m3core jay$ for a in \ > > > IntLlvmObj \ > > > IntLlvmAsm \ > > > ExtLlvmObj \ > > > ExtLlvmAsm \ > > > StAloneLlvmObj \ > > > StAloneLlvmAsm; \ > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > IntLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > *** > > > > > > IntLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** > > > > > > ExtLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > ExtLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > StAloneLlvmObj > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > => success? > > > > > > except for: > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > StAloneLlvmAsm > > > close, maybe just a config problem > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > RT0.is:5:2: error: unknown directive > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > - Jay > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From dabenavidesd at yahoo.es Mon Sep 14 03:39:21 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 14 Sep 2015 01:39:21 +0000 (UTC) Subject: [M3devel] Missing something? Message-ID: <548758070.2577233.1442194761020.JavaMail.yahoo@mail.yahoo.com> Hi all:I was wondering whether RT machinery is running faster on cross-compiled code in C, or JITed? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 08:14:52 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 06:14:52 +0000 Subject: [M3devel] addition of GPL stuff Message-ID: Um, sorry to bring up such annoying matters, but maybe we should not be licensingnew files as GPL, assuming they aren't derived from GPL files? e.g. the LLVM m3makfiles and Makefiles? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 09:05:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 07:05:21 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F4BFA7.7090805@lcwb.coop> References: , , <55F335F7.9010108@lcwb.coop>, , <55F4BFA7.7090805@lcwb.coop> Message-ID: I agree development never ends.One should always keep in mind active development and not treat it as rare/unusual. The way to get assembly output, I claim, is to fiddle with the config file. For example, add -S or -s on the C compiler command line. I don't believe it is ever useful/required to output assembly for the entire treeusing a C compiler or LLVM. It is only needed very temporarily/modally for focused debugging.Even if development never ends. Unless maybe you are analysing picking up a new LLVM or such? You usually trust the C compiler and LLVM to translate correctlyfrom your output. Usually you only care to look at your output.Therefore -keep. I can be flexible on this.Still, it doesn't require a mode I Think. Yes I have about same meaning of "internal" as you.I guess there are multiple variables: separate process? separate .so/dll? output the cm3cg files? Up until now, they have always coincided. In terms of the builder though, it would not likely do anything different for static linking vs. dynamic linking of LLVM. It should still just make the same function calls. Unless maybe you want to use dlopen/dlsym? Which is really yet another mode "dynamic dynamic". I think it is really just if the cm3cg files are output. I should see if I can eliminate the C mode, in order to "prove"I'm not being hypocritical here. I think it was just for easieraccess to the RunCC or such. I'll try to pass that address tothe C backend and then go back to InternalObject mode.. If C mode is just InternalObject, then probably LLVM should be too.Again, builder should just decide to produce the IR files or not.The other decisions are other variables that Builder doesn't care about.One does have to instantiate the right backend somewhere. It isn't a huge deal though. Builder can temporarily be a mess and we canclean it up later. > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. I think this is just a matter of a few lines of quake. Can you look at what I did for m3cgcat and the C mode? Here: cm3cg.common: if not defined("USE_C_BACKEND_VIA_M3CGCAT") USE_C_BACKEND_VIA_M3CGCAT = FALSEend and then: if USE_C_BACKEND_VIA_M3CGCATM3_BACKEND_MODE = "ExternalObject" proc m3_backend(source, object, optimize, debug) is ret_code = try_exec("m3cgcat", "-in-binary:" & source, "-out-c:" & source & ".c") if not equal(ret_code, 0) return ret_code end return compile_c(source & ".c", object, [ ], FALSE, TRUE)endend I don't remember any problems with this, just that it is slower to output the files. If you wanted to see the C compiler output, here you'd add -S or -s or -FA or such.No need to change the mode.There is something subtle here I'm not explaining well, how this is usefulbut doesn't need to be automated, because it is used rarely and only at small scale. Like, I had no need for a CToAssembly mode. Ultimately "internal" modes are probably always faster, and the draw is strongfor that reason. The C backend is slow enough w/o "external" making it slower. - Jay > Date: Sat, 12 Sep 2015 19:13:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > My inference has been that "internal" means the whole compilation job > is done by code linked in (not necessarily statically) to cm3, e.g., > using the integrated x86 backend. "external" means there are one or > more separate executables to run after the front end, e.g., m3cc > and as, or m3llvm, llc, asm, etc. > > I definitely want the possibility of keeping assembly form for development work, > even when 'as' is run. And we should assume that occasional development work > will continue forever. Disassembling object code will lose sometimes very > useful information, compared to keeping assembly as originally generated as > a side output. > > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. > > On 09/12/2015 06:50 PM, Jay K wrote: > > Those comments were from me, since I went through > > and 1) added the C mode 2) tried to clean up what > > I percieve as rather messy, and still messy. > > > > > > You helped my vocabulary here -- it should be "cartesian factored". > > Rather than every "product" be its own case. > > > > > > I moved it toward that, but it still confusing. > > > > > > Basically you want a set of available steps, and > > then just sequence them, possibly omitting some of them. > > > > > > I can see how you might have roughly 10 choices, > > depending on if you are outputing bitcode or LLVM assembly > > or building LLVM data structures, and linking to LLVM or not, > > and LLVM is producing assembly or object files. > > > > > > I would suggest that LLVM produing assembly is not a useful choice. > > Remove that from the matrix. > > Have it produce object files if you can. > > For debugging, you can fiddle with the switches or dump the files > > afterward. > > > > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > > You can use the "external object" mode plus config file stuff. > > I started that way for C and didn't change the builder initially. > > But it was slower. I don't remember why i needed builder support, > > vs. the existing "internal object" mode, > > maybe to more easily reuse the existing support for compiling C. > > My mode is kind of "internal plus run some quake code". > > I guess I could setup linkage so the C backend could call the quake > > code w/o the builder knowing about it. > > Maybe I should try that, so we can get back to the original set of modes. > > > > > > Basically "external" means "produce cm3cg IR files". > > "Internal" means don't produce them. > > And then "assembly" means "run assembler after backend" > > and "object" means don't run assembler because we already have objects. > > > > > > Whether or not the m3cg IR files are being converted to C and then objects > > or gcc internals and then objects, or LLVM internals and then objects, > > the builder doesn't need to know or act differently. > > It just runs the quake function m3_backend or such, and it decides, > > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > > > > Basically, let's say we have n different external backends producing > > object files. That doesn't imply any new modes. But some other variable > > that affects the quake code that runs cm3cg. > > > > > > Thank you for the confirmation as to which mode you are using. > > > > > > - Jay > > > > > > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > > to do it when you are familiar with the code than refamiliarize later. There were > > > already some with comments like "No such back end". Only the StAloneLlvm ones > > > have an actual back end. The others are intended for modes that link in at least > > > some of llvm into cm3. As discussed, we may not want to do this. > > > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > > on small examples and passing a good few of the compiler tests. Right now, I am working > > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > > > > jair:m3core jay$ for a in \ > > > > IntLlvmObj \ > > > > IntLlvmAsm \ > > > > ExtLlvmObj \ > > > > ExtLlvmAsm \ > > > > StAloneLlvmObj \ > > > > StAloneLlvmAsm; \ > > > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > > > IntLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > *** > > > > > > > > IntLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** > > > > > > > > ExtLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > > > ExtLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > > > StAloneLlvmObj > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > > > => success? > > > > > > > > except for: > > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > > > > StAloneLlvmAsm > > > > close, maybe just a config problem > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > > RT0.is:5:2: error: unknown directive > > > > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 15 10:44:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 15 Sep 2015 08:44:31 +0000 Subject: [M3devel] Target.Aligned_procedures (again) Message-ID: How about this: To check for a closure: 1 Clear the lower bit of the address. Or maybe the lower 2 bits. 2 Check for a 4 byte -1. Not necessarily a full INTEGER. Marker always consumes an INTEGER's worth of space, but only ever check 4 bytes.Clearing the lower bit is strictly to accommodate Thumb -- does it take alignment faults?The conditional branch in the current code, at least for 64bit non-amd64 platforms would be gone.All targets would be the same. Clearing the lower two bits would make the address always aligned.The assumption is that if is a closure, it is already aligned and this is value preserving.If it is not a closure, this still yields a valid accessible address, just not the one to call. Or too risky in terms of portability? Better to check for alignment, assume unaligned is not a closure, and just check for a full INTEGER -1? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:47:39 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:47:39 +0000 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? Message-ID: As I remove Target variation: 1) Just assign the variables all the same in Target.i3/Target.m3?2) Change them to constants in Target.i3?3) Remove them entirely in Target and m3front? i.e. Do you think anyone will ever go back and re-diverge?And need a super local handy guide?Or can go back into history or figure it out anew? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:53:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:53:43 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned Message-ID: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 20:59:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 13:59:55 -0500 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? In-Reply-To: References: Message-ID: <55FF022B.8090101@lcwb.coop> I think it far easier for someone in the future if there is evidence right in the current source, perhaps commented out and preferably commented about, than to try to ferret out history, even when the CM system preserves it. Some future maintainer may not even know that relevant history even exists, let alone where to look. On 09/20/2015 11:47 AM, Jay K wrote: > As I remove Target variation: > > > 1) Just assign the variables all the same in Target.i3/Target.m3? > 2) Change them to constants in Target.i3? > 3) Remove them entirely in Target and m3front? > > > i.e. Do you think anyone will ever go back and re-diverge? > And need a super local handy guide? > Or can go back into history or figure it out anew? > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Sep 20 21:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 14:05:51 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: Message-ID: <55FF038F.8030603@lcwb.coop> On 09/20/2015 11:53 AM, Jay K wrote: > I propose that Allow_packed_byte_aligned be set to FALSE for all targets. > > Or make it const. > > > Or remove it. > Also, removing it altogether from Target.i3 means the value will be copied out the place(s) where it is now used, making it even harder for someone to find. Making it all the same is OK with me, but>>> > > This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths. > Specifically, if you manage to create a packed type with unaligned fields, > presumably loads/stores get converted to multiple aligned loads/stores followed > by putting stuff back together. The occurrence of the unaligned fields should > be very rare in the first place. > Not sure what you are saying, nor whether this is true of all targets, but at least some targets will refuse a packed field that crosses a word boundary, at the point of type definition. The language explicitly permits an implementation to do this. So multi-word loads/stores wouldn't happen anyway. Within a native word, bit-twiddling works fine. > > I does not affect any other target. > > > Granted, x86 and amd64 are the most common targets. > > > Ok? > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 20 22:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 20:41:03 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? Message-ID: containsLazyAlignments m3front/Module.m3: PROCEDURE SetLazyAlignment (on: BOOLEAN) = BEGIN IF curModule # NIL THEN curModule.lazyAligned := on; IF on THEN curModule.containsLazyAlignments := TRUE; END; END; END SetLazyAlignment; PROCEDURE Compile (t: T) = VAR save: T; zz: Scope.T; yy: Revelation.Set; BEGIN (* ETimer.Push (M3Timers.emit); *) Target.Allow_packed_byte_aligned := t.containsLazyAlignments; This feels wrong.Surely, Allow_packed_byte_aligned should be constant for any given target?Surely a pragma cannot change a target's characteristics?Or, rather, I found this by changing it to a constant false for all targets. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 23:00:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 16:00:35 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? In-Reply-To: References: Message-ID: <55FF1E73.1010206@lcwb.coop> Hmm, this sounds like a mechanism we had a discussion of several years ago. I don't remember how it ended, but I think there was a general consensus that it was an idea with some unresolved problems. However, one big class of use cases for packing things is to make it possible to define a record layout that matches one already specified externally, a network protocol packet, for example. We had a much bigger problem with that regarding different endianness, which we never did anything about. I have some unimplemented notes somewhere on how to do opposite-endian record layout. It was mind-boggling, to put it gently, and left me even more negative on so-called "little endian" than previously. Anyway, this is where different rules for different records on the same target may be wanted. On 09/20/2015 03:41 PM, Jay K wrote: > containsLazyAlignments > > m3front/Module.m3: > > PROCEDURE SetLazyAlignment (on: BOOLEAN) = > BEGIN > IF curModule # NIL THEN > curModule.lazyAligned := on; > IF on THEN > curModule.containsLazyAlignments := TRUE; > END; > END; > END SetLazyAlignment; > > > PROCEDURE Compile (t: T) = > VAR save: T; zz: Scope.T; yy: Revelation.Set; > BEGIN > (* ETimer.Push (M3Timers.emit); *) > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > > This feels wrong. > Surely, Allow_packed_byte_aligned should be constant for any given target? > Surely a pragma cannot change a target's characteristics? > Or, rather, I found this by changing it to a constant false for all targets. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Sep 21 04:19:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:19:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , Message-ID: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:22:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:22:25 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Message-ID: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:41:03 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG Message-ID: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:49:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:49:27 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG In-Reply-To: <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> References: , <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> Message-ID: Not gratuitous but I think I found a workaround.I will change the various M3x86/M3C/other. New functions to return M3CG_Ops.Public instead of M3CG.T.So that cm3/M3Backend.m3 can set the additional data fields, and then just returnit as the new same old fully opaque M3CG.T to the rest of cm3. I think a type with very limited visible is about as good as a type with nothing visible. When I was exploring the system, I also found this separation a slightspeed bump into learning. You look at M3CG for something useful and find nothing there.I suppose that is maybe the Modula-3 way, but it seems more obscure than usual.Usually there is just one interface for this sort of construct I think, not two. - Jay Subject: Re: [M3devel] M3CG_Ops vs. M3CG From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:44:10 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Jay, this seems like a gratuitous change. I strongly suggest not for now. Sent from my iPhone On Sep 21, 2015, at 12:41 PM, Jay K wrote: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:51:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:51:08 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> Message-ID: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:58:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:58:42 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> Message-ID: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:00:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:00:44 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, Message-ID: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:06:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:06:44 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, Message-ID: I suppose it is more convenient to have the C mode than I say -- it alsoputs the bootstrap vs. non-bootstrap decision in a convienent place,and the management of keep_files, and the deleting of temporary files. All that logic can be easily moved elsewhere, or not. But then we have like 6 LLVM modes.I set a bad example, and now it is even worse. Arguably we only need 2 modes -- internal and external.Any backend, internal or example, that produces assembly, canrun the assembler itself -- we could expose that in M3CG justlike I want to expose running the C compiler. I like it. Or even just one mode -- call the backend.We just need to provide in M3CG_Ops a function to write out the .mc files,that the backend can call if it needs it. i.e. for the backend call into quake/config.Each backend can put together its specific sequence.There is some commonality currently factored into Builder, but not reallymuch and it is a mess. I'm undecided at the moment. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 02:58:42 +0000 I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:21:09 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:21:09 +0000 Subject: [M3devel] globals vs. instance variables Message-ID: We have: cm3/MODULE M3Backend; ... VAR obj_file : M3ObjFile.T := NIL; obj_wr : Wr.T := NIL; obj_name : TEXT := NIL; log : Wr.T := NIL; log_name : TEXT := NIL; ...PROCEDURE Close (<*UNUSED*> cg: M3CG.T) = BEGIN IF obj_file # NIL THEN TRY NTObjFile.Dump (obj_file, obj_wr); EXCEPT Wr.Failure, Thread.Alerted => Msg.FatalError (NIL, "problem writing object file: ", obj_name); END; Utils.CloseWriter (log, log_name); obj_file := NIL; obj_wr := NIL; obj_name := NIL; log := NIL; log_name := NIL; END; END Close; Surely we should add those variables to M3CG_Ops in orderto get them to Close for this specific M3CG? i.e. prefer "instance variables" over "module variables" aka globals. Perhaps in a record called M3Backend to give an appearance of opacityor perhaps in an actual opaque type, though I hate to pay for theextra heap allocation just to achieve opacity. (Larger point is the cm3 is written an old unfortunate thread-unsafe style with many globals.) And this is a good reason then to smush M3CG.T and M3CG_Ops.T together,in order for M3Backend.New() result to be passed back to Close. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:40:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:40:46 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , Message-ID: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:45:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:45:37 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> Message-ID: There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:41:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:41:38 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> Message-ID: Sorry, clarification: struct A { char a; int b; }; is defined and fairly portable in C, and means generally the same thing onall platforms (ignoring 16bit int), the same thing as: RECORD a: Ctypes.char b: Ctypes.intEND; what is not portable in C is expecting this to have other than size 8. Strictly speaking, its size is completely not portable, but on the vast vast majorityof systems its size is 8, and getting it less requires compiler-specific mechanisms. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu But it us defined in C, just target-specific. Sent from my iPhone On Sep 21, 2015, at 2:00 PM, Jay K wrote: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:56:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:56:00 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:04:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? Message-ID: For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:07:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: Message-ID: Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:21:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:21:47 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: , Message-ID: er, ok, I partly did this. https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3.diff?r1=1.95;r2=1.96;f=u Revision 1.96: download - view: text, markup, annotated - select for diffsThu Apr 1 17:56:37 2010 (5 years, 5 months ago) by jkrellBranches: MAINDiff to: previous 1.95: preferred, unifiedChanges since revision 1.95: +1 -9 lines leave max_align = 64 for all platformsthis effects: I386_FREEBSD / FreeBSD4 I386_LINUX / LINUXLIBC6 NetBSD2_i386 It does not affect any others -- not PPC, not Darwin, not 64bit, not Solaris, not NT. but it wasn't consistent across 32bit platforms or x86.Nor should it necessarily -- I recall Cygwin wants to alignmore than cm3, implying NT does. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: Re: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 16:41:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 14:41:02 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, Message-ID: I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:12 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , Message-ID: I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:53 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu>, Message-ID: I guess it makes sense to be in Builder, just that it can be a separate booland not a mode. The code will almost be identical to today, and not clearlydemonstrate the lack of a need for a mode. Alas. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 04:45:37 +0000 There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:37:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:37:18 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , Message-ID: Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. Perhaps m3front can behave similarly?For all targets? But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 16:31:12 +0000 I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Sep 21 22:25:49 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 21 Sep 2015 16:25:49 -0400 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: <20150921202548.GB20914@topoi.pooq.com> On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > Perhaps m3front can behave similarly?For all targets? > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > - Jay The fancy packing should remain in the language, for backward compatibility if nothing else. modula 3 is, after all, a systems language. But is likely no need to worry it in the bootstrap. If the bootstrap doesn't use it, it will still work portably as a bootstrap, no matter how machine-dependent fancy packing happens to be. -- hendrik From jay.krell at cornell.edu Tue Sep 22 02:15:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <20150921202548.GB20914@topoi.pooq.com> References: , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , <20150921202548.GB20914@topoi.pooq.com> Message-ID: Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 22 06:55:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 04:55:50 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , , , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , , , , , , <20150921202548.GB20914@topoi.pooq.com>, Message-ID: There is a proposal in here..here is a variation of it.. Allow packing of records tightly on all targets. All targets get the same layout (modulo word size), the same things can compile successfully. Remove the caveat here that certain code will only compile for certain targets. If Allow_packed_byte_aligned is false, and there is an unaligned read/write, there are a few choices. m3front could transform into multiple inlined read/writes byte-wise taking the value apart of putting it back together. Or m3front could issue function calls. The C implementation could be #ifdefed such that amd64/x86 just do the read/write, albeit still with a function call. C backend could set allow_packed_byte_aligned the same always. In that case, the functions would be output into the file and inlined or macroed. A warning might be issued about the lack of atomicity. Heck, let's invent a feature...a pragma could be allowed on integral/floating point values as to endianness and compiler could output the byte swaps for you. The endianness could actually be some sort of runtime expression. That's how some formats work -- they get written native, with an indicator as to which that is, and swapped if necessary upon reads, and maybe swapped back for writes. You know -- if you really want a powerful system for efficient binary persistance.. Or just leave it alone. ? - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: Re: [M3devel] Target.Allow_packed_byte_aligned Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Sep 26 21:23:27 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sat, 26 Sep 2015 12:23:27 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150926122327.F8299C6A@m0087794.ppops.net> An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Sep 26 22:30:26 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 16:30:26 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: <20150926203026.GB29515@topoi.pooq.com> Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Sat Sep 26 23:57:46 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 26 Sep 2015 21:57:46 +0000 (UTC) Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926203026.GB29515@topoi.pooq.com> References: <20150926203026.GB29515@topoi.pooq.com> Message-ID: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> not here El S?bado 26 de septiembre de 2015 15:32, Hendrik Boom escribi?: Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Sep 27 01:46:27 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 19:46:27 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> References: <20150926203026.GB29515@topoi.pooq.com> <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> Message-ID: <20150926234627.GB2740@topoi.pooq.com> On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik From lists at darko.org Sun Sep 27 03:27:59 2015 From: lists at darko.org (Darko Volaric) Date: Sat, 26 Sep 2015 18:27:59 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: > In trying to better understand the issue, I see, in Module.m3:895, this > code: > > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > From other places, containsLazyAlignments derives from LAZYALIGN > pragmas, so this means it will turn on Allow_packed_byte_aligned, > regardless of the target, if there is a LAZYALIGN pragma. I think this > will generate code that will alignfault, if the target CPU doesn't > support it. That would be a bug. > > > -Rodney Bates > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:17:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:17:14 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, Message-ID: I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:21:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:21:34 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, , , Message-ID: I forgot to mention/ask: "packed" basically means "no padding", right? - Jay From: jay.krell at cornell.edu To: lists at darko.org; rodney_bates at lcwb.coop Date: Sun, 27 Sep 2015 05:17:14 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:03:20 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:03:20 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927080320.F82B920B@m0087798.ppops.net> An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:10:28 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:10:28 -0700 Subject: [M3devel] HTML mail (was Re: Another Allow_packed_byte_aligned issue) Message-ID: <20150927081028.F82B92BC@m0087798.ppops.net> FWIW, I am traveling and using a webmail that is different from my usual email client. I did find and change a preference setting that sounded like it might switch to plain text. -Rodney Bates --- hendrik at topoi.pooq.com wrote: From: Hendrik Boom To: "m3devel at elegosoft.com" Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue Date: Sat, 26 Sep 2015 19:46:27 -0400 On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Sun Sep 27 17:21:06 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:21:06 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927082106.F82B9288@m0087798.ppops.net> -Rodney Bates --- jay.krell at cornell.edu wrote: From: Jay K To: Darko Volaric , Rodney Bates CC: m3devel Subject: RE: [M3devel] Another Allow_packed_byte_aligned issue Date: Sun, 27 Sep 2015 05:17:14 +0000 I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. I'm just speculating here, but I would think that if it crosses a boundary of the hardware RAM word, (not necessarily the same as a CPU native word) it would be non-atomic anyway, regardless of whether the compiler or the CPU took care of juggling the separate pieces. In fact, I get the impression that, even within a single RAM word, you can't count on atomicity of even an aligned, single-word load or store, with modern multi-CPUs, multi-level hardware caching, etc. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From jay.krell at cornell.edu Tue Sep 1 01:44:23 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 31 Aug 2015 23:44:23 +0000 Subject: [M3devel] Target.Aligned_procedures and closure markers? In-Reply-To: <55E34A77.1050001@lcwb.coop> References: , <55E34A77.1050001@lcwb.coop> Message-ID: Agreed -- I forgot to acknowledge/repeat that this is relatedto the first instruction in a function, not arbitrary instructions. On the matter of 4 or 8 bytes though -- the thing is -- many but not all64bit architectures are the exact same instruction set as 32 bits,just a different selection of instructions. Same size, same encodings, same alignment. I believe this is true for alpha, mipa, ppc, sparc, hppa. It is almost true for x86, though that kinda doesn't count,since there is no fixed size or alignment requirement. Fyi: x86/amd64 instructions are between 1 and 15 bytes in size,inclusive, and all sizes in between. All the "lower" sizesincluding 1 are common. Upper sizes are rare, maybe starting around 7 bytes.The encoding scheme allows for longer instructions, butthere is a deliberate documented limit. Not sure about arm -- arm64 at least gets rid of thumb.So arm instructions either 4-aligned or at odd addresses,and arm64 is like all the rest -- fixed size 4 byte aligned. So 4 byte -1 e.g. on ppc32 has the same meaning as 8 byte -1 on ppc64,just that it is two in a row. It looks like we have agreement -- we can set this to a constant. The marker value is -1 I checked. I still find the function If_closure a little unclear/subtle.I read through my comments in Target.m3 again. It is false if: - the system ever has alignment faults - AND functions/instructions might be less aligned than closures Some of the commented I added in the code are incorrect. The comments say that a misaligned function pointer is readone byte at a time. I believe the actual behavior is thata misaligned function pointer is deemed not a closure,and reading the marker is skipped. The code is all still there, but branched around.When aligned_procedures == true, less code is generated,no alignment check is generated. Most 32bit platforms set Aligned_procedures = trueand most 64bit platforms set it to false. The exceptions: amd64 is also true. arm32 is false, due to Thumb mode. Because most platforms, 32bit and 64bit, have fixed size4 byte aligned instructions. So 32bit platforms haveinstructions (and functions) thereforealigned the same as closures. 64bit platforms generallyhave closures more aligned than instructions. I think if we ignored arm32, and penalized only amd64,which granted is the most common platform,we could set aligned_procedures = false for all 64bit,true for all 32bit. And having another variablethat coincided with word size is ok. But arm32's thumb instructions mean a function pointermight be unaligned. I'll make it always false. > Method body procedures are required to be top-level Kind of a language limitation compared to "more dynamic" languages. Definitely not trivial to do otherwise -- heap allocated garbage collected stack frames and such.. Thank you, - Jay > Date: Sun, 30 Aug 2015 13:24:55 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Aligned_procedures and closure markers? > > > > On 08/30/2015 02:45 AM, Jay K wrote: > > The agenda remains seeing if Target variables can be made constants. > > > > The discussion in this case is more complicated and some facts are unclear. > > > > > > Background: > > > > > > Nested functions are a problem. > > In particular, if you can take their address. > > Taking the address of a nested function presents a problem. > > You are presented with two or three solutions. > > > > > > - runtime code gen > > - either on the stack > > > > - or somewhere more expensive, possibly with garbage collection > > > > - possibly "templated" instead of "arbitrary"; the meaning > > of this is a lot to explain -- related to libffi and mremap, which > > isn't entirely portable, but is e.g. portable to Windows > > > > > > - or instead of runtime codegen, altering how function pointers are > > called; you can only do this from Modula-3 code, not e.g. C or C++. > > > > > > The solution Modula-3 has taken is to alter how funtion pointers are called. > > The sequence is roughly: > > Check if it is a "regular" function pointer or a "closure". > > If it is a "regular" function pointer, just call it. > > If it is a "closure", it contains a function pointer and a "static link". > > Call the function pointer, passing the static link. > > > > > > To tell if it is "regular" function pointer or a "closure", roughly > > what is done is the data at the function pointer is read and compared > > against a marker value. If it equals the marker value, it is a closure. > > > > > > The size of the marker is the size of an integer or pointer (4 or 8 bytes). > > The value of the marker checked for is 0 or -1, I'd have to check. > > The alignment of the pointer might be a factor. In particular, on most > > architectures, all instructions have a certain alignment. If the pointer has > > less alignment, it can't be an instruction. Or maybe on those architectures, > > the bytes are read one at a time to avoid alignment faults. > > > > > > In particular, as far as I know, the following: > > x86/amd64: no alignment of instructions, but functions maybe, but Modula-3 assumes functions aren't aligned > > > > ppc32/ppc64/alpha32/alpha64/mips32/mipa64/sparc32/sparc64/arm64/hppa32/hppa64 -- instructions are all > > 4 bytes and 4 byte aligned, so functions are at least also as much > > > > arm32 -- instructions are 2 or 4 bytes; if they are 2 bytes, then the instruction > > pointer is actually odd as well, and the low bit is removed to really find the instructions > > That is -- instruction pointer is either odd or 4-aligned, never 2-aligned. > > > > ia64 -- instructions come in bundles of 3, they are 41 bits each, with a 5 bit "template" in each > > bundle, for a total of 128 bits per bundle, likely always 128-bit-aligned > > > > > > I could use confirmation on much of this. > > > > > > I find the use of a marker value a little dubious. It'd be good to research if there is one > > value that works on all. > > > > > > I find the choice of a marker size to be pointer-sized dubious on most platforms. > > In particular, most 64bit platforms have a 32bit instruction size, so using more than 32 bits > > for the marker value doesn't buy much. If the marker value is actually a legal instruction, > > then checking for two in a row reduces the odds of a false positive. > > > > > > However, given that the closure is a marker and two pointers, it isn't like you are going > > to pack the second and third 64bit field right up against a 32bit field. You'd want padding for alignmet. > > > > Right. If the marker had smaller alignment than a pointer, say 32-bit marker, 64-bit pointers, then > it would be necessary to start the closure on an odd multiple of 32 bits--a rule that is not part > of anybody's alignment system of any compiler that I am aware of. So then you'd have to finesse it > by giving the closure 64-bit alignment and starting with a pad word, which would fail to gain the > space benefits. Moreover, fewer bits of marker increase the likelihood of its accidentally being a > valid instruction or sequence thereof. > > So I think making the marker the same size as a pointer, and giving the whole closure pointer-sized > alignment is the best way, unless/until we find a machine instruction set that has a known ambiguity > here. > > Also, it is not necessary that there be no valid instruction sequence that starts with 32 or 64 1-bits. > It is enough that no compiler produces it at the beginning of a prologue. Much harder to ascertain for > certain (especially if we want to be able to call procedures produced by other compilers) but much less > likely to result in a problem. > > Just a wild guess, but I would not be surprised if ELF and other object formats would require the > machine code of a function/procedure to begin on a native word boundary, even if the hardware > instruction set does not. Where so, this would obviate checking the alignment before checking > for a closure, though probably target-dependently. > > > > > If we are aiming for all out target-specificity, I'd suggest marker size be a target aspect, > > and set it to 4 bytes for ppc64/mips64/sparc64/alpha64/arm64/hppa64. > > > > > > However, I want less target-variation not more. > > > > > > Here are some my lingering questions: > > - Is the marker value actually invalid code on every platform? Does its value need to be target-specific? > > - Is a 64bit marker value actually sufficient on IA64? > > The way to help here, I think, is to ensure that a 64bit marker, > > not a 128bit marker, contains the "template", and an invalid "template". > > - Barring the previous, a solution might be to use a 128 bit marker on all platforms. > > > > > > i believe all of these function pointers are rare. > > I hope/believe the object method calls do not check for closures -- though actually > > that is related to a useful language construct, that I doubt we have. > > > > Method body procedures are required to be top-level, ensured statically, so there is no > need for method call code to consider the possibility of the pointer in the object type > to be a closure. > > > > > The simplest solution is likely: > > - ignore IA64, or research it further > > - keep marker size at integer > > Pointer would be target-independent in getting the following two pointers aligned. > > > - for the C backend, assume no alignment of function pointers -- give up > > any of the optimization, esp. x86/amd64. > > I think this optimization both applies to a low-frequency situation and has a very > small benefit, so I would not worry about giving up on it. > > > > > > > For other than the C backend, maybe dial back marker size to 4 bytes for mips64/sparc64/alpha64/arm64/hppa64. > > While I don't like target-specificity, notice this wouldn't check linux vs. bsd vs. solaris, etc. It isn't a cross produce thing. > > > > > > Thoughts? > > > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 07:52:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 05:52:16 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150816134306.GB5008@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com> Message-ID: old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd "stress test" my output.Digital Mars might be another.And yes the Intel compiler -- which is a proxy for a set of compilers -- the EDG frontend.(There are approx four C++ frontends in practical use these days: Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front end.) - Jay > Date: Sun, 16 Aug 2015 13:43:06 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Sun, Aug 16, 2015 at 10:11:15AM +0000, Jay K wrote: > > Autoconf output has little/none dependency.Sun's SPARC optimization are > > mostly irrelevant as we have little C code,unless you use the C backend. > > Ok, well, it sounds like it will be increasingly important as you go to your > C backend though. > > > > somewhat of a test of C portability > > This I appreciate. We have some C code and the C backend.More compilers > > have "helped".The Tru64 compiler was another.On HP-UX in-box I had only > > K&R so resorted to gcc (Including bootstrapping through gcc 3.x;see > > m3-sys/m3cc/src/m3makefile...)We work with Microsoft Visual C++ too.And > > now clang -- having found and worked around a bug in its assembler.Some > > time soon I'll expand to Metrowerks and Digital Mars.. > > I have the Intel C/C++ and Fortran compilers on a Linux box but I don't have > the latest versions. C/C++ are not my thing but if I can help by just > building stuff to see what messages we get or if it breaks etc. let me know > and I will try to participate. > > I didn't know Metrowerks was still around. I thought they got swallowed up > by Motorola and then removed from all the non-embedded space. > > > > - Jay > > > > > > > Date: Sun, 16 Aug 2015 08:24:09 +0000 > > > From: microcode at zoho.com > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] Build Server - Plan > > > > > > On Sun, Aug 16, 2015 at 07:52:13AM +0000, Jay K wrote: > > > > The output of autoconf/automake should have lightweight dependencies.They > > > > might stress make, might require GNU make.They might stress the sh, but I > > > > think there are adequate shells out there. > > > > > > That is typically one issue with autoconf, requiring gnu tools in the > > > path. On Solaris this can be annoying since most Solaris people don't use > > > gcc or bash or have them in their path and not all (none of?) the gnu pieces > > > are up to date or even current by any stretch of the imagination. > > > > > > > They are meant to be easy for people building stuff to use.They aren't > > > > meant to be easy for people developing stuff to use. > > > > > > Not sure what you meant here... > > > > > > > Look at this way..while people complain and there are widelyused > > > > alternatives like cmake, autoconf/automake are in widespreaduse, and they > > > > do provide things that work for Linux, Solaris, BSD, MacOSX, Cygwin,HPUX, > > > > Aix, Irix, etc. > > > > > > They often don't "work" for Solaris as-installed but they can most often be > > > made to work. Increasingly, as automake and its prereqs get version bumps > > > there are problems building apps on Solaris because Solaris installs with a > > > very back-level version of gcc and a rather incomplete set of gnu tools. I > > > ran into a problem with the last year where Solaris awk was not good enough > > > to install(!) an app that compiled on Solaris as part of autotools so I had > > > to download a newish copy of gawk and install it. So really, it is much > > > better not to go there if you can avoid it. > > > > > > I am not suggesting an alternative to autotools but just pointing out it is > > > not really accurate to say they work on Linux, Solaris, BSD, etc. except for > > > Linux. I also run BSD on several platforms and because of the same issues as > > > with Solaris (old copy of gcc and gnu tools and POSIX-compliant awk, sed, > > > shells, etc. not being good enough for autotools) it is sometimes > > > non-trivial and painful to get Linux apps built on BSD. There is obviously > > > no good/easy solution to this, just to point out it is not the slam-dunk as > > > might be thought. > > > > > > > Furthermore, consider any platform that has a native gcc, is likely > > > > buildingthat with autoconf/automake. > > > > > > Conceded, yet on Solaris it is not clear why gcc is there. It is old and it > > > is in a non-standard location and is often not used. Ideally, apps to be run > > > on Solaris should be able to be built with native (non-gnu) tools. The > > > Studio compilers are very good and have optimizations for SPARC that should > > > be better than what gcc can provide and it is also somewhat of a test of C > > > portability since Studio doesn't necessarily provide all non-standard gcc > > > extensions. > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 08:22:24 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 06:22:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: References: <20150814063629.GA3086@zoho.com> <20150816072730.GB3479@zoho.com> <20150816082409.GC3479@zoho.com> <20150816134306.GB5008@zoho.com> Message-ID: <20150901062224.GA3050@zoho.com> On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > "stress test" my output.Digital Mars might be another.And yes the Intel > compiler -- which is a proxy for a set of compilers -- the EDG > frontend. Hi, You lost me here. What is EDG? As far as I know, the Intel compiler is a proprietary (and complete, self-contained) product of Intel. I think I read on Intel's forums in a post by Steve Lionel that earlier versions were based on code that made its way from DEC to COMPAQ but was later reworked so that it is almost all new. An interesting tangent: I ran some tests earlier and can't remember for sure but it seems to me there is some Open64 code in Intel's compilers. Whether Intel wrote it and it made its way to Open64, or whether it was part of Open64 and Intel took some of it, I do not know. > (There are approx four C++ frontends in practical use these days: > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > end.) From jay.krell at cornell.edu Tue Sep 1 08:23:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:23:42 +0000 Subject: [M3devel] m3front -unfold_nested_procs Message-ID: It'd be nice, if someone has the time, to let -unfold_nested_procs work with cm3cg.I tried just silently ignoring it, but then it crashes. -unfold_nested_procs is required by the C backend.I and I think the NTx86 backend. And required to be omitted by the gcc/cm3cg backend. Ideally, even if they aren't ABI compatible (due to how static link is passed,unfortunate, I wish I could fix it), they could at least operate on identical M3 IR.To ease moving between them. And granted, it'd be nice if the C backend didn't care. I think I can do that,by moving the functions around afterward.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 08:27:24 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 06:27:24 +0000 Subject: [M3devel] Build Server - Plan In-Reply-To: <20150901062224.GA3050@zoho.com> References: , <20150814063629.GA3086@zoho.com>, , <20150816072730.GB3479@zoho.com>, , <20150816082409.GC3479@zoho.com>, , <20150816134306.GB5008@zoho.com>, , <20150901062224.GA3050@zoho.com> Message-ID: edg.comWhile almost nobody writes their own C++ compiler, even fewer people write their own C++ frontend.Edison Design Group.A small company whose main/only product is a C++ front end that they license in sourceform to people who are interested in, say, writing a backend only. - Jay > Date: Tue, 1 Sep 2015 06:22:24 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Build Server - Plan > > On Tue, Sep 01, 2015 at 05:52:16AM +0000, Jay K wrote: > > old mail, sorry: indeed Metrowerks isn't relevant, but I figured I'd > > "stress test" my output.Digital Mars might be another.And yes the Intel > > compiler -- which is a proxy for a set of compilers -- the EDG > > frontend. > > Hi, > > You lost me here. What is EDG? > > As far as I know, the Intel compiler is a proprietary (and complete, > self-contained) product of Intel. I think I read on Intel's forums in a post > by Steve Lionel that earlier versions were based on code that made its way > from DEC to COMPAQ but was later reworked so that it is almost all new. > > An interesting tangent: I ran some tests earlier and can't remember for sure > but it seems to me there is some Open64 code in Intel's compilers. Whether > Intel wrote it and it made its way to Open64, or whether it was part of > Open64 and Intel took some of it, I do not know. > > > (There are approx four C++ frontends in practical use these days: > > Microsoft, gcc, clang, EDG.Most compilers other than those use the EDG front > > end.) > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 09:18:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 07:18:02 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] 4537cb: remove unused locals In-Reply-To: References: <55e53db0990a0_3be33fd3fe0af2a068547@hookshot-fe2-cp1-prd.iad.github.net.mail>, <7C1D0FBD-5C9A-4883-A47C-57E613FA171E@purdue.edu>, <4E019C37-773B-42AE-AC49-F53C52249195@purdue.edu>, <2E8C2B23-F814-46FF-A59A-C92F486EC273@purdue.edu>, Message-ID: I use git pull to update my local repository to be "like" github.I use git commit to commit my local edits to my local repository.It use git push to push my local repository changes to the master.I don't know what rebasing is or how/why to use it. - Jay Subject: Re: [M3commit] [modula3/cm3] 4537cb: remove unused locals From: hosking at purdue.edu Date: Tue, 1 Sep 2015 17:10:51 +1000 CC: jay.krell at cornell.edu; m3commit at elegosoft.com; m3devel at elegosoft.com To: hosking at purdue.edu It appears to be this merge action which is causing the problem.I don?t have time to explore right now. If anyone else can it would be most appreciated. On 1 Sep 2015, at 5:08 pm, Antony Hosking wrote:Whatever you are doing I now don?t know how to recover from.It appears that somehow you are rebasing the history on the master branch, but I don?t quite know how. On 1 Sep 2015, at 5:00 pm, Antony Hosking wrote:In your work flow are you using "git push" to push your local commits? Or are you doing something else? On 1 Sep 2015, at 4:58 pm, Antony Hosking wrote:Jay, it appears that your commits are still messing up the history. As of your most recent commit we have lost history again. What exactly are you doing when you use git against the repo? For instance, if you take a look at RTCollector we see a history with the oldest commit labelled "initial diff from ../config/PPC_LINUX" from Jan 4, 2008. Whereas it should have the oldest commit labeled as follows from when Olaf first imported from cvs to svn. commit cfac2c79141d61c291a40177f0d5490f774c032fAuthor: Olaf Wagner Date: Wed Jan 24 12:24:49 2001 +0000 This commit was generated by cvs2svn to compensate for changes in r20, which included commits to RCS files with non-trunk default branches. On 1 Sep 2015, at 3:54 pm, jaykrell wrote: Branch: refs/heads/master Home: https://github.com/modula3/cm3 Commit: 4537cb804fbf864a076235495db77eebf9c5dab6 https://github.com/modula3/cm3/commit/4537cb804fbf864a076235495db77eebf9c5dab6 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- remove unused locals Commit: 477c3d058a95ba8a06c90ac5d36dceefd3e7f39a https://github.com/modula3/cm3/commit/477c3d058a95ba8a06c90ac5d36dceefd3e7f39a Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: M m3-sys/cm3/src/Builder.m3 Log Message: ----------- Fix regression from "7/31 Initial integration of llvm back end into cm3's build system." when bootstrapping using cm3cg. .c files pass through for bootstrap, along with any .h or checked in .s/.o etc. bootstrapping is using cm3 -boot. Commit: 459512da7848b534636c1fe4e1095eee76ad2d89 https://github.com/modula3/cm3/commit/459512da7848b534636c1fe4e1095eee76ad2d89 Author: jaykrell Date: 2015-08-31 (Mon, 31 Aug 2015) Changed paths: A m3-sys/llvm3.6.1/src/LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.i3 A m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 A m3-sys/llvm3.6.1/src/Main.m3 A m3-sys/llvm3.6.1/src/m3makefile Log Message: ----------- Merge branch 'master' of github.com:modula3/cm3 pull from master Compare: https://github.com/modula3/cm3/compare/b068a306fe87...459512da7848_______________________________________________ M3commit mailing list M3commit at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 18:28:10 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? Message-ID: I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 1 19:00:07 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 17:00:07 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: Sorry, this merits more discussion. - There are at least three ways compilers chose between C and C++. invocation name: gcc vs. g++, lowercase cc vs. uppercase CC I think clang vs. clang++. Visual C++ only has one invocation: cl Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. Command line switches: gcc -x c++ clang -x C++ cl -Tc means next source file is C -TC means all are C -Tp means next is C++ -TP means all are C++ We'd have to read up on others. One of the combinations implemented by gcc is deprecated by clang. I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). So I switched to use -x c++. "cpp" is ambiguous between C preprocessor and C++. "cxx" is often therefore used to indicate C++. x is a rotated +. We could have config files specify SYSTEM_CC and SYSTEM_CXX. And we could have functions compile_c and compile_cxx. We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe others, error if there are multiple, and chose compile_c vs. compile_cxx. Or compile_c could chose based on the extension. In any case, makes changes to the C backend path, and in the llvm binding/backend directories should be easy. We don't *have* to change the meanings everywhere to effect them. I do not actually intend to rewrite all of our little bits of C using C++ specific features. But the LLVM stuff uses C++ and the C backend imho ought to. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Tue, 1 Sep 2015 16:28:10 +0000 Subject: [M3devel] gcc vs. g++ in config files? I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Sep 1 19:08:58 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 1 Sep 2015 17:08:58 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <20150901170858.GA6140@zoho.com> Since you or somebody else mentioned autotools, I think the way this is normally handled is for the script to use cc or CC and it expects you to set those values in your shell to whatever compiler you want to use. On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > Sorry, this merits more discussion. > > - There are at least three ways compilers chose between C and C++. > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > I think clang vs. clang++. > Visual C++ only has one invocation: cl > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > Command line switches: > gcc -x c++ > clang -x C++ > cl -Tc means next source file is C > -TC means all are C > -Tp means next is C++ > -TP means all are C++ > We'd have to read up on others. > > One of the combinations implemented by gcc is deprecated by clang. > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > So I switched to use -x c++. > > "cpp" is ambiguous between C preprocessor and C++. > "cxx" is often therefore used to indicate C++. x is a rotated +. > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > And we could have functions compile_c and compile_cxx. > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > others, error if there are multiple, and chose compile_c vs. compile_cxx. > Or compile_c could chose based on the extension. > > In any case, makes changes to the C backend path, and in the llvm binding/backend > directories should be easy. We don't *have* to change the meanings everywhere > to effect them. I do not actually intend to rewrite all of our little bits > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > imho ought to. > > - Jay > > > > > From: jay.krell at cornell.edu > To: m3devel at elegosoft.com > Date: Tue, 1 Sep 2015 16:28:10 +0000 > Subject: [M3devel] gcc vs. g++ in config files? > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > - Jay > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -- From jay.krell at cornell.edu Tue Sep 1 20:02:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 1 Sep 2015 18:02:27 +0000 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: <20150901170858.GA6140@zoho.com> References: , , <20150901170858.GA6140@zoho.com> Message-ID: For now we aren't using autotools. I remain on the fence. I want some of what they do, but they really are fairly ugly and slow and overkill.I kinda want some other small portable scripting language.The way they find the C compiler is pretty simple really.Maybe python. Embedding it and adding functions? I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? libtool supports more systems than our config files, but is alsokinda ugly/overkill/slow and somewhat philosophically poor.For example, the fact that they default to compiling everything twice,pic and non-pic, bothers me. Surely it is not worth a) the double timeto build b) the slight loss of security in executables.All code should just be pic.They do make it an option, but when people foist that default decisionon the masses, it makes me wonder.Apple has no such modes.Windows has no such modes (though Windows doesn't do quitewhat I want either). In particular, I want someone else to - find C and C++ compiler -- but what autotools does here is pretty simple - pick out int8, int16, int32, int64 for me, with greater portability to older and newer systems -- but there is inttypes.h on most systems anyway - figure out how to link -- for static there is just one portable way (I adjusted the config files recently), but for shared, libtool or cmake do it better?easier? - generate portable makefiles for bootstrapping (or for regular building?) Or just GNU makefiles? That is one some guidance I've seen -- give up on portable makefiles and just use GNU make. cmake?? I don't want to lose the fine grained incrementality the integrated builder has,but it is tempting to make the system just generate makefiles. I wonder which parts of the system are worth maintaining, vs. giving in to the largerecosystem of stuff being maintained for us. cm3 is ahead of its time for 20 years ago but these days seems largely redundant.Perhaps we can/should whittle it down to a more unique and worthwhile core?I don't know. - Jay > Date: Tue, 1 Sep 2015 17:08:58 +0000 > From: microcode at zoho.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] gcc vs. g++ in config files? > > Since you or somebody else mentioned autotools, I think the way this is > normally handled is for the script to use cc or CC and it expects you to > set those values in your shell to whatever compiler you want to use. > > On Tue, Sep 01, 2015 at 05:00:07PM +0000, Jay K wrote: > > Sorry, this merits more discussion. > > > > - There are at least three ways compilers chose between C and C++. > > > > invocation name: gcc vs. g++, lowercase cc vs. uppercase CC > > I think clang vs. clang++. > > Visual C++ only has one invocation: cl > > Source file extension: .c vs. .cpp, .cxx, .cc, .C, etc.This is very common with Visual C++. > > Command line switches: > > gcc -x c++ > > clang -x C++ > > cl -Tc means next source file is C > > -TC means all are C > > -Tp means next is C++ > > -TP means all are C++ > > We'd have to read up on others. > > > > One of the combinations implemented by gcc is deprecated by clang. > > I think clang++ foo.c. And, actually g++ foo.c (Apple installs clang as gcc). > > So I switched to use -x c++. > > > > "cpp" is ambiguous between C preprocessor and C++. > > "cxx" is often therefore used to indicate C++. x is a rotated +. > > We could have config files specify SYSTEM_CC and SYSTEM_CXX. > > And we could have functions compile_c and compile_cxx. > > We could have builder c_source("foo") probe for foo.c, and foo.cpp, maybe > > others, error if there are multiple, and chose compile_c vs. compile_cxx. > > Or compile_c could chose based on the extension. > > > > In any case, makes changes to the C backend path, and in the llvm binding/backend > > directories should be easy. We don't *have* to change the meanings everywhere > > to effect them. I do not actually intend to rewrite all of our little bits > > of C using C++ specific features. But the LLVM stuff uses C++ and the C backend > > imho ought to. > > > > - Jay > > > > > > > > > > From: jay.krell at cornell.edu > > To: m3devel at elegosoft.com > > Date: Tue, 1 Sep 2015 16:28:10 +0000 > > Subject: [M3devel] gcc vs. g++ in config files? > > > > > > > > > > I suggest we replace "gcc" with "g++".In all the config files (and in the bootstrap archives).And similar for Sun CC, and Visual C++ use -TP. > > > > rationale: It will likely cleanup our LLVM stuff, which currently uses a custom Makefile and drops derived files into the src directory. It will ease/allow using C++ exception handling in the C backend. I've been using g++ for months/years on Darwin. counter-rationale: Not all C code is valid C++, and sometimes the meanings are different. I converted the coverage code only recently. > > > > - Jay > > > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > -- > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Sep 1 20:32:00 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 1 Sep 2015 14:32:00 -0400 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: <20150901170858.GA6140@zoho.com> Message-ID: <20150901183200.GA10167@topoi.pooq.com> On Tue, Sep 01, 2015 at 06:02:27PM +0000, Jay K wrote: ... ... > I wonder about the value of much of the cm3 system.Like, maybe quake would be better off replaced with Python? It might be scons you're looking for. It's a build system that's made out of python. It is not a plug-compatible make replacement. It is completely different. -- hendrik From rodney_bates at lcwb.coop Wed Sep 2 17:32:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 02 Sep 2015 10:32:03 -0500 Subject: [M3devel] gcc vs. g++ in config files? In-Reply-To: References: Message-ID: <55E71673.4010508@lcwb.coop> On 09/01/2015 11:28 AM, Jay K wrote: > I suggest we replace "gcc" with "g++". > In all the config files (and in the bootstrap archives). > And similar for Sun CC, and Visual C++ use -TP. > > > rationale: > It will likely cleanup our LLVM stuff, which currently > uses a custom Makefile and drops derived files into the src directory. There are a couple of problems here. I initially tried putting compiled C/C++ code in the build directory. Since it is target-dependent, it obviously does not belong in src. But a bug in the builder just wipes out the file with a circular symbolic link: % import_obj needs work!! % import_obj() puts a symlink in ../BUILD_DIR/.o, pointing to , % located relative to the src directory. Later, it links the symlink into the package. % If the file is target-dependent, (as these and most .o files are), we want to put them % in ../BUILD_DIR, the target build directory, and only target-dependent place we have. % But import_obj("../BUILD_DIR/") just replaces it with a circular symlink. % When this is fixed, the Makefile will have to be changed to put it in the right place. The bug shouldn't be hard to fix, it just hasn't gotten to the top of my list. Then the .o files can be moved to the build directory. But that's still using make to build the .o files. The other problem is the cm3 build system. It's really quite sophisticated about dependencies within Modula3 code, but things like 'c_source' don't, as far as I can see, do any dependency analysis at all. They work fine for the odd .c and .h files here and there in the cm3 tree. But the bindings to llvm (the parts we must write in C & C++) bring in large #include closures from the llvm source tree and then link in several compiled llvm libraries. I don't think cm3's build system is at all up to this. So I used make for that. Actually, as far as I can find, llvm has not provided any dependency information about what libraries will be needed, in what order (which, BTW, undergoes reversal of certain segments). So far, it has been a very dicey and frustrating game of wild guesses and experiments. I am sure the lists I have come up with must have some undetected unnecessary things. I haven't even figured out a way to find out what library a link name declaration is in. > It will ease/allow using C++ exception handling in the C backend. > I've been using g++ for months/years on Darwin. > counter-rationale: > Not all C code is valid C++, and sometimes the meanings are different. > I converted the coverage code only recently. > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Thu Sep 3 01:56:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 2 Sep 2015 23:56:06 +0000 Subject: [M3devel] build_standalone errors Message-ID: import_lib("dl", "/usr/lib")import_lib("ncurses", "/usr/lib") + build_standalone() => on Darwin, where is only /usr/lib/libdl.dylib and /usr/lib/libncurses.dylb. clang: error: no such file or directory: '/usr/lib/libncurses.a'clang: error: no such file or directory: '/usr/lib/libdl.a' build_standalone should probably be much smarterand at least probe file existance; if found specify it;if not found use -lfoo. The idea being to be standalone with respect to available static libs,not all libs. More so, really, stand-aloneness should be specifiable on a libraryby library, or directory basis. This is while building m3-sys/llvm, making it standalone, whichI expect *might* be desired. I'll go ahead without standalone. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 4 23:48:42 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 04 Sep 2015 16:48:42 -0500 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail> Message-ID: <55EA11BA.7010704@lcwb.coop> Hmm. As near as I remember, I assumed the front end was unnesting calls just because it did it on one example (using m3cc). Looking at the code, I see command line options -nested_calls and -no_nested_calls (the default). However, the only use of this I can find affects whether an independent check by a M3CG_Check.T instance verifies that there is no nesting. So it looks like it's actually unconditional after all. I'll leave the comments as they are unless this changes. I would expect a back end not to have to cope with them, unless it was generating final code for a bytecode or stack-like machine that could directly handle them nested. On 09/03/2015 05:38 PM, Jay wrote: > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > - Jay > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > >> Branch: refs/heads/master >> Home: https://github.com/modula3/cm3 >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/m3middle/src/M3CG_Ops.i3 >> >> Log Message: >> ----------- >> Comment no nested calls occur in cm3 IR. >> >> Comment the fact that nested calls in source code are unnested in >> cm3 intermediate representation. This is an essential invariant >> to both producers and consumers of this IR. >> >> Changes to be committed: >> >> modified: src/M3CG_Ops.i3 >> >> >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Revert an incorrect fix due to misdiagnosis. >> >> The actual problem here was not a lack of a set_source_file, >> it was reading binary cm3 IR with the ascii reader. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 >> Author: Rodney Bates >> Date: 2015-09-03 (Thu, 03 Sep 2015) >> >> Changed paths: >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 >> >> Log Message: >> ----------- >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 >> >> Lots of changes happened to this file in package llvm, while the version >> in package llvm3.6.1 has been undergoing integration with the greatly >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 >> found in package llvm3.6.1. Also some random manual review and fixes. >> >> Compiles and runs to completion, with superficially believable output, >> on one small test case involving separate temporaries in and out of >> a FINALLY procedure. >> >> Works with cm3 command, when its executable m3llvm has been shipped. >> >> The test compile fails to link, not finding alloca. m3llvm needs to >> detect calls on alloca and translate internally into llvm alloca >> instructions, rather than letting them through to be linked in. >> >> Changes to be committed: >> >> modified: M3CG_LLVM.m3 >> >> >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 >> _______________________________________________ >> M3commit mailing list >> M3commit at elegosoft.com >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sat Sep 5 03:51:16 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 5 Sep 2015 01:51:16 +0000 Subject: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. In-Reply-To: <55EA11BA.7010704@lcwb.coop> References: <55e89cdb30490_79243f820464d2bc82318@hookshot-fe6-cp1-prd.iad.github.net.mail>, , <55EA11BA.7010704@lcwb.coop> Message-ID: -unfold_nested_procs is what I'm referring to. Sorry, I comprehend better now -- "procs" vs. "calls". nested call: foo(bar()) unnested call: temp = bar() foo(temp) nested proc: void foo(){ void bar() { }} unnested proc: void foo(){ } void bar(){} And then everything I said. :)I want fewer variables and more constants. :)I want one (or fewer) persisted form of the IR and an ability for a backend to easily transform.I need to push the M3CG_MultiPass stuff around more I think.Then again, the IR doesn't tend to live long and there isn't likely a cause for it to, so variation is ok.The only time it lives long is when iterating tightly on a backend.It could be useful for bootstrapping perhaps, but I think we have a better option already -- C/C++. - Jay > Date: Fri, 4 Sep 2015 16:48:42 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com; m3commit at elegosoft.com > Subject: Re: [M3devel] [M3commit] [modula3/cm3] e7a87c: Comment no nested calls occur in cm3 IR. > > Hmm. As near as I remember, I assumed the front end was unnesting calls just because it > did it on one example (using m3cc). Looking at the code, I see command line options > -nested_calls and -no_nested_calls (the default). However, the only use of this I can > find affects whether an independent check by a M3CG_Check.T instance verifies that > there is no nesting. So it looks like it's actually unconditional after all. I'll > leave the comments as they are unless this changes. > > I would expect a back end not to have to cope with them, unless it was generating > final code for a bytecode or stack-like machine that could directly handle them nested. > > On 09/03/2015 05:38 PM, Jay wrote: > > I believe the lack of nested calls is optional. M3front has an option. Different backends have different requirements. The in-process backends could/should set the flag. There should be an M3CG pass or two to transform from either form to the other (or have m3front output one way, and optionally transform to the other). > > > > - Jay > > > > On Sep 3, 2015, at 12:17 PM, Rodney Bates wrote: > > > >> Branch: refs/heads/master > >> Home: https://github.com/modula3/cm3 > >> Commit: e7a87c74bd1b88d284408b760b22409459640d46 > >> https://github.com/modula3/cm3/commit/e7a87c74bd1b88d284408b760b22409459640d46 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/m3middle/src/M3CG_Ops.i3 > >> > >> Log Message: > >> ----------- > >> Comment no nested calls occur in cm3 IR. > >> > >> Comment the fact that nested calls in source code are unnested in > >> cm3 intermediate representation. This is an essential invariant > >> to both producers and consumers of this IR. > >> > >> Changes to be committed: > >> > >> modified: src/M3CG_Ops.i3 > >> > >> > >> Commit: 8bcccbb7ccaa0af0879c80014215daa571f95174 > >> https://github.com/modula3/cm3/commit/8bcccbb7ccaa0af0879c80014215daa571f95174 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Revert an incorrect fix due to misdiagnosis. > >> > >> The actual problem here was not a lack of a set_source_file, > >> it was reading binary cm3 IR with the ascii reader. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Commit: 379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> https://github.com/modula3/cm3/commit/379f750aa7e5fac4948be06d5bd7a748a7d85350 > >> Author: Rodney Bates > >> Date: 2015-09-03 (Thu, 03 Sep 2015) > >> > >> Changed paths: > >> M m3-sys/llvm3.6.1/src/M3CG_LLVM.m3 > >> > >> Log Message: > >> ----------- > >> Big merge of changes to M3CG_LLVM.m3 from package llvm into package llvm3.6.1 > >> > >> Lots of changes happened to this file in package llvm, while the version > >> in package llvm3.6.1 has been undergoing integration with the greatly > >> altered llvm interfaces in llvm 3.6.1. Merge these into the M3CG_LLVM.m3 > >> found in package llvm3.6.1. Also some random manual review and fixes. > >> > >> Compiles and runs to completion, with superficially believable output, > >> on one small test case involving separate temporaries in and out of > >> a FINALLY procedure. > >> > >> Works with cm3 command, when its executable m3llvm has been shipped. > >> > >> The test compile fails to link, not finding alloca. m3llvm needs to > >> detect calls on alloca and translate internally into llvm alloca > >> instructions, rather than letting them through to be linked in. > >> > >> Changes to be committed: > >> > >> modified: M3CG_LLVM.m3 > >> > >> > >> Compare: https://github.com/modula3/cm3/compare/909637def4c4...379f750aa7e5 > >> _______________________________________________ > >> M3commit mailing list > >> M3commit at elegosoft.com > >> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3commit > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Sep 11 03:53:36 2015 From: jay.krell at cornell.edu (Jay K) Date: Fri, 11 Sep 2015 01:53:36 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone Message-ID: Here is what I get: some "crash", some "fail", some get the file format mixed up, some work!6 modes. 4 crash, 1 appears to work, 1 appears close jair:m3core jay$ for a in \IntLlvmObj \IntLlvmAsm \ExtLlvmObj \ExtLlvmAsm \StAloneLlvmObj \StAloneLlvmAsm; \ do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done IntLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3*** IntLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** ExtLlvmObj*** runtime error:*** Segmentation violation - possible attempt to dereference NIL*** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 ExtLlvmAsm*** runtime error:*** Segmentation violation - possible attempt to dereference NIL StAloneLlvmObjm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io => success? except for:error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) StAloneLlvmAsmclose, maybe just a config problemm3llvm -b -d -o RT0.ib RT0.icllc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.isfg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.ioRT0.is:5:2: error: unknown directive I assume we should ignore Int and Ext and focus on StAlone for now. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Sep 11 22:13:43 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 11 Sep 2015 15:13:43 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: Message-ID: <55F335F7.9010108@lcwb.coop> Yes, I put in all the modes I thought of while working in there, as it's a lot easier to do it when you are familiar with the code than refamiliarize later. There were already some with comments like "No such back end". Only the StAloneLlvm ones have an actual back end. The others are intended for modes that link in at least some of llvm into cm3. As discussed, we may not want to do this. The only mode I have tried is StAloneLlvmAsm. This is running to compile completion on small examples and passing a good few of the compiler tests. Right now, I am working on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. On 09/10/2015 08:53 PM, Jay K wrote: > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > jair:m3core jay$ for a in \ > IntLlvmObj \ > IntLlvmAsm \ > ExtLlvmObj \ > ExtLlvmAsm \ > StAloneLlvmObj \ > StAloneLlvmAsm; \ > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > IntLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > *** > > IntLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** > > ExtLlvmObj > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > ExtLlvmAsm > *** runtime error: > *** Segmentation violation - possible attempt to dereference NIL > > StAloneLlvmObj > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > => success? > > except for: > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > StAloneLlvmAsm > close, maybe just a config problem > m3llvm -b -d -o RT0.ib RT0.ic > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > RT0.is:5:2: error: unknown directive > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 13 01:26:35 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:26:35 +0000 Subject: [M3devel] builder behavior upon error -- subsequent files not fully compiled Message-ID: fyi, I observe the following: When compiling multiple files, and one fails, the failure inappropriately affects later files, in that, one or more of: - all outputs are deleted - or -keep isn't honored - or the "build pipeline" is truncated, i.e. we might check the validity of .m3/.i3 files but we don't progress all the way to generate assembly/object files It used to be worse, we just discussed, I had regressed it years ago by accident, in that compilation stopped and no further errors are reported. But now it can largely succeed, but not have the intended outputs, so you fix the one error, and then recompile everything. There is just one boolean "compile_failed" that is set by the first failure, and left set, and the code doesn't know if it was from an earlier file or the current file. It either needs to be a counter, and captured before certain steps and checked for > after them, or there should be two booleans, one that is set/cleared per file, one that is set whenever the first becomes set and is never cleared, or functions should return a boolean, for the current file, and either also set the "global", or set the global near the top of the call tree. It isn't global, but it is nearly so. Making this code multithreaded will be an unfortunately large task.. I'll probably fix this soon. This is just a heads up. I'm not addressing the single-threadedness, just the other. It shouldn't be a big change. This is related to: What I'm actually seeing with one of the LLVM passes is the first invocation of some tool crashes, so the rest truncate their build pipeline and claim m3front failed, which isn't true, either it didn't run or didn't fail. The failure is from the first file. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 13 01:50:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 12 Sep 2015 23:50:08 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F335F7.9010108@lcwb.coop> References: , <55F335F7.9010108@lcwb.coop> Message-ID: Those comments were from me, since I went through and 1) added the C mode 2) tried to clean up what I percieve as rather messy, and still messy. You helped my vocabulary here -- it should be "cartesian factored". Rather than every "product" be its own case. I moved it toward that, but it still confusing. Basically you want a set of available steps, and then just sequence them, possibly omitting some of them. I can see how you might have roughly 10 choices, depending on if you are outputing bitcode or LLVM assembly or building LLVM data structures, and linking to LLVM or not, and LLVM is producing assembly or object files. I would suggest that LLVM produing assembly is not a useful choice. Remove that from the matrix. Have it produce object files if you can. For debugging, you can fiddle with the switches or dump the files afterward. If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. You can use the "external object" mode plus config file stuff. I started that way for C and didn't change the builder initially. But it was slower. I don't remember why i needed builder support, vs. the existing "internal object" mode, maybe to more easily reuse the existing support for compiling C. My mode is kind of "internal plus run some quake code". I guess I could setup linkage so the C backend could call the quake code w/o the builder knowing about it. Maybe I should try that, so we can get back to the original set of modes. Basically "external" means "produce cm3cg IR files". "Internal" means don't produce them. And then "assembly" means "run assembler after backend" and "object" means don't run assembler because we already have objects. Whether or not the m3cg IR files are being converted to C and then objects or gcc internals and then objects, or LLVM internals and then objects, the builder doesn't need to know or act differently. It just runs the quake function m3_backend or such, and it decides, via a separate factor from mode to run llvm or cm3cg on the IR file. Basically, let's say we have n different external backends producing object files. That doesn't imply any new modes. But some other variable that affects the quake code that runs cm3cg. Thank you for the confirmation as to which mode you are using. - Jay > Date: Fri, 11 Sep 2015 15:13:43 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > to do it when you are familiar with the code than refamiliarize later. There were > already some with comments like "No such back end". Only the StAloneLlvm ones > have an actual back end. The others are intended for modes that link in at least > some of llvm into cm3. As discussed, we may not want to do this. > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > on small examples and passing a good few of the compiler tests. Right now, I am working > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > On 09/10/2015 08:53 PM, Jay K wrote: > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > jair:m3core jay$ for a in \ > > IntLlvmObj \ > > IntLlvmAsm \ > > ExtLlvmObj \ > > ExtLlvmAsm \ > > StAloneLlvmObj \ > > StAloneLlvmAsm; \ > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > IntLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > *** > > > > IntLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** > > > > ExtLlvmObj > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > ExtLlvmAsm > > *** runtime error: > > *** Segmentation violation - possible attempt to dereference NIL > > > > StAloneLlvmObj > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > => success? > > > > except for: > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > StAloneLlvmAsm > > close, maybe just a config problem > > m3llvm -b -d -o RT0.ib RT0.ic > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > RT0.is:5:2: error: unknown directive > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > - Jay > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 13 02:13:27 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 12 Sep 2015 19:13:27 -0500 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: References: , <55F335F7.9010108@lcwb.coop> Message-ID: <55F4BFA7.7090805@lcwb.coop> My inference has been that "internal" means the whole compilation job is done by code linked in (not necessarily statically) to cm3, e.g., using the integrated x86 backend. "external" means there are one or more separate executables to run after the front end, e.g., m3cc and as, or m3llvm, llc, asm, etc. I definitely want the possibility of keeping assembly form for development work, even when 'as' is run. And we should assume that occasional development work will continue forever. Disassembling object code will lose sometimes very useful information, compared to keeping assembly as originally generated as a side output. Actually, I'd also like to be able to set it up so m3cgcat gets run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It would just save a couple of manual steps that I am typing over and over, slowing down the debug process. On 09/12/2015 06:50 PM, Jay K wrote: > Those comments were from me, since I went through > and 1) added the C mode 2) tried to clean up what > I percieve as rather messy, and still messy. > > > You helped my vocabulary here -- it should be "cartesian factored". > Rather than every "product" be its own case. > > > I moved it toward that, but it still confusing. > > > Basically you want a set of available steps, and > then just sequence them, possibly omitting some of them. > > > I can see how you might have roughly 10 choices, > depending on if you are outputing bitcode or LLVM assembly > or building LLVM data structures, and linking to LLVM or not, > and LLVM is producing assembly or object files. > > > I would suggest that LLVM produing assembly is not a useful choice. > Remove that from the matrix. > Have it produce object files if you can. > For debugging, you can fiddle with the switches or dump the files > afterward. > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > You can use the "external object" mode plus config file stuff. > I started that way for C and didn't change the builder initially. > But it was slower. I don't remember why i needed builder support, > vs. the existing "internal object" mode, > maybe to more easily reuse the existing support for compiling C. > My mode is kind of "internal plus run some quake code". > I guess I could setup linkage so the C backend could call the quake > code w/o the builder knowing about it. > Maybe I should try that, so we can get back to the original set of modes. > > > Basically "external" means "produce cm3cg IR files". > "Internal" means don't produce them. > And then "assembly" means "run assembler after backend" > and "object" means don't run assembler because we already have objects. > > > Whether or not the m3cg IR files are being converted to C and then objects > or gcc internals and then objects, or LLVM internals and then objects, > the builder doesn't need to know or act differently. > It just runs the quake function m3_backend or such, and it decides, > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > Basically, let's say we have n different external backends producing > object files. That doesn't imply any new modes. But some other variable > that affects the quake code that runs cm3cg. > > > Thank you for the confirmation as to which mode you are using. > > > - Jay > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > to do it when you are familiar with the code than refamiliarize later. There were > > already some with comments like "No such back end". Only the StAloneLlvm ones > > have an actual back end. The others are intended for modes that link in at least > > some of llvm into cm3. As discussed, we may not want to do this. > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > on small examples and passing a good few of the compiler tests. Right now, I am working > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > jair:m3core jay$ for a in \ > > > IntLlvmObj \ > > > IntLlvmAsm \ > > > ExtLlvmObj \ > > > ExtLlvmAsm \ > > > StAloneLlvmObj \ > > > StAloneLlvmAsm; \ > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > IntLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > *** > > > > > > IntLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** > > > > > > ExtLlvmObj > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > ExtLlvmAsm > > > *** runtime error: > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > StAloneLlvmObj > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > => success? > > > > > > except for: > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > StAloneLlvmAsm > > > close, maybe just a config problem > > > m3llvm -b -d -o RT0.ib RT0.ic > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > RT0.is:5:2: error: unknown directive > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > - Jay > > > > > > > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From dabenavidesd at yahoo.es Mon Sep 14 03:39:21 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 14 Sep 2015 01:39:21 +0000 (UTC) Subject: [M3devel] Missing something? Message-ID: <548758070.2577233.1442194761020.JavaMail.yahoo@mail.yahoo.com> Hi all:I was wondering whether RT machinery is running faster on cross-compiled code in C, or JITed? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 08:14:52 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 06:14:52 +0000 Subject: [M3devel] addition of GPL stuff Message-ID: Um, sorry to bring up such annoying matters, but maybe we should not be licensingnew files as GPL, assuming they aren't derived from GPL files? e.g. the LLVM m3makfiles and Makefiles? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 14 09:05:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 14 Sep 2015 07:05:21 +0000 Subject: [M3devel] LLVM builder modes Int/Ext/StAlone In-Reply-To: <55F4BFA7.7090805@lcwb.coop> References: , , <55F335F7.9010108@lcwb.coop>, , <55F4BFA7.7090805@lcwb.coop> Message-ID: I agree development never ends.One should always keep in mind active development and not treat it as rare/unusual. The way to get assembly output, I claim, is to fiddle with the config file. For example, add -S or -s on the C compiler command line. I don't believe it is ever useful/required to output assembly for the entire treeusing a C compiler or LLVM. It is only needed very temporarily/modally for focused debugging.Even if development never ends. Unless maybe you are analysing picking up a new LLVM or such? You usually trust the C compiler and LLVM to translate correctlyfrom your output. Usually you only care to look at your output.Therefore -keep. I can be flexible on this.Still, it doesn't require a mode I Think. Yes I have about same meaning of "internal" as you.I guess there are multiple variables: separate process? separate .so/dll? output the cm3cg files? Up until now, they have always coincided. In terms of the builder though, it would not likely do anything different for static linking vs. dynamic linking of LLVM. It should still just make the same function calls. Unless maybe you want to use dlopen/dlsym? Which is really yet another mode "dynamic dynamic". I think it is really just if the cm3cg files are output. I should see if I can eliminate the C mode, in order to "prove"I'm not being hypocritical here. I think it was just for easieraccess to the RunCC or such. I'll try to pass that address tothe C backend and then go back to InternalObject mode.. If C mode is just InternalObject, then probably LLVM should be too.Again, builder should just decide to produce the IR files or not.The other decisions are other variables that Builder doesn't care about.One does have to instantiate the right backend somewhere. It isn't a huge deal though. Builder can temporarily be a mess and we canclean it up later. > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. I think this is just a matter of a few lines of quake. Can you look at what I did for m3cgcat and the C mode? Here: cm3cg.common: if not defined("USE_C_BACKEND_VIA_M3CGCAT") USE_C_BACKEND_VIA_M3CGCAT = FALSEend and then: if USE_C_BACKEND_VIA_M3CGCATM3_BACKEND_MODE = "ExternalObject" proc m3_backend(source, object, optimize, debug) is ret_code = try_exec("m3cgcat", "-in-binary:" & source, "-out-c:" & source & ".c") if not equal(ret_code, 0) return ret_code end return compile_c(source & ".c", object, [ ], FALSE, TRUE)endend I don't remember any problems with this, just that it is slower to output the files. If you wanted to see the C compiler output, here you'd add -S or -s or -FA or such.No need to change the mode.There is something subtle here I'm not explaining well, how this is usefulbut doesn't need to be automated, because it is used rarely and only at small scale. Like, I had no need for a CToAssembly mode. Ultimately "internal" modes are probably always faster, and the draw is strongfor that reason. The C backend is slow enough w/o "external" making it slower. - Jay > Date: Sat, 12 Sep 2015 19:13:27 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > My inference has been that "internal" means the whole compilation job > is done by code linked in (not necessarily statically) to cm3, e.g., > using the integrated x86 backend. "external" means there are one or > more separate executables to run after the front end, e.g., m3cc > and as, or m3llvm, llc, asm, etc. > > I definitely want the possibility of keeping assembly form for development work, > even when 'as' is run. And we should assume that occasional development work > will continue forever. Disassembling object code will lose sometimes very > useful information, compared to keeping assembly as originally generated as > a side output. > > Actually, I'd also like to be able to set it up so m3cgcat gets > run on .mc/.mc files and llvm-dis on .mb/.ib files, by cm3. It > would just save a couple of manual steps that I am typing over > and over, slowing down the debug process. > > On 09/12/2015 06:50 PM, Jay K wrote: > > Those comments were from me, since I went through > > and 1) added the C mode 2) tried to clean up what > > I percieve as rather messy, and still messy. > > > > > > You helped my vocabulary here -- it should be "cartesian factored". > > Rather than every "product" be its own case. > > > > > > I moved it toward that, but it still confusing. > > > > > > Basically you want a set of available steps, and > > then just sequence them, possibly omitting some of them. > > > > > > I can see how you might have roughly 10 choices, > > depending on if you are outputing bitcode or LLVM assembly > > or building LLVM data structures, and linking to LLVM or not, > > and LLVM is producing assembly or object files. > > > > > > I would suggest that LLVM produing assembly is not a useful choice. > > Remove that from the matrix. > > Have it produce object files if you can. > > For debugging, you can fiddle with the switches or dump the files > > afterward. > > > > > > If you are producing cm3cg IR files like for m3cc, likely you don't need builder support. > > You can use the "external object" mode plus config file stuff. > > I started that way for C and didn't change the builder initially. > > But it was slower. I don't remember why i needed builder support, > > vs. the existing "internal object" mode, > > maybe to more easily reuse the existing support for compiling C. > > My mode is kind of "internal plus run some quake code". > > I guess I could setup linkage so the C backend could call the quake > > code w/o the builder knowing about it. > > Maybe I should try that, so we can get back to the original set of modes. > > > > > > Basically "external" means "produce cm3cg IR files". > > "Internal" means don't produce them. > > And then "assembly" means "run assembler after backend" > > and "object" means don't run assembler because we already have objects. > > > > > > Whether or not the m3cg IR files are being converted to C and then objects > > or gcc internals and then objects, or LLVM internals and then objects, > > the builder doesn't need to know or act differently. > > It just runs the quake function m3_backend or such, and it decides, > > via a separate factor from mode to run llvm or cm3cg on the IR file. > > > > > > Basically, let's say we have n different external backends producing > > object files. That doesn't imply any new modes. But some other variable > > that affects the quake code that runs cm3cg. > > > > > > Thank you for the confirmation as to which mode you are using. > > > > > > - Jay > > > > > > > > > > > Date: Fri, 11 Sep 2015 15:13:43 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com > > > Subject: Re: [M3devel] LLVM builder modes Int/Ext/StAlone > > > > > > Yes, I put in all the modes I thought of while working in there, as it's a lot easier > > > to do it when you are familiar with the code than refamiliarize later. There were > > > already some with comments like "No such back end". Only the StAloneLlvm ones > > > have an actual back end. The others are intended for modes that link in at least > > > some of llvm into cm3. As discussed, we may not want to do this. > > > > > > The only mode I have tried is StAloneLlvmAsm. This is running to compile completion > > > on small examples and passing a good few of the compiler tests. Right now, I am working > > > on the ones that pass using m3cc and fail using m3llvm, before trying to build the real code. > > > > > > On 09/10/2015 08:53 PM, Jay K wrote: > > > > Here is what I get: some "crash", some "fail", some get the file format mixed up, some work! > > > > 6 modes. 4 crash, 1 appears to work, 1 appears close > > > > > > > > > > > > jair:m3core jay$ for a in \ > > > > IntLlvmObj \ > > > > IntLlvmAsm \ > > > > ExtLlvmObj \ > > > > ExtLlvmAsm \ > > > > StAloneLlvmObj \ > > > > StAloneLlvmAsm; \ > > > > > > > > do rm AMD64_DARWIN/RT0.io ; echo $a;cm3 -DM3_BACKEND_MODE=$a -keep -commands; done > > > > > > > > IntLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x1030edca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > *** > > > > > > > > IntLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** > > > > > > > > ExtLlvmObj > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > *** pc = 0x10e8b0ca3 = set_error_handler + 0x5b in ../src/M3CG.m3 > > > > > > > > ExtLlvmAsm > > > > *** runtime error: > > > > *** Segmentation violation - possible attempt to dereference NIL > > > > > > > > StAloneLlvmObj > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=obj -relocation-model=pic RT0.ib -o RT0.io > > > > > > > > => success? > > > > > > > > except for: > > > > error: /Library/Developer/CommandLineTools/usr/bin/libtool: file: RT0.io is not an object file (not allowed in a library) > > > > > > > > > > > > StAloneLlvmAsm > > > > close, maybe just a config problem > > > > m3llvm -b -d -o RT0.ib RT0.ic > > > > llc -disable-fp-elim -filetype=asm -relocation-model=pic RT0.ib -o RT0.is > > > > fg++ -m64 -arch x86_64 -c -x assembler RT0.is -o RT0.io > > > > RT0.is:5:2: error: unknown directive > > > > > > > > > > > > I assume we should ignore Int and Ext and focus on StAlone for now. > > > > > > > > > > > > - Jay > > > > > > > > > > > > _______________________________________________ > > > > M3devel mailing list > > > > M3devel at elegosoft.com > > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > > _______________________________________________ > > > M3devel mailing list > > > M3devel at elegosoft.com > > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 15 10:44:31 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 15 Sep 2015 08:44:31 +0000 Subject: [M3devel] Target.Aligned_procedures (again) Message-ID: How about this: To check for a closure: 1 Clear the lower bit of the address. Or maybe the lower 2 bits. 2 Check for a 4 byte -1. Not necessarily a full INTEGER. Marker always consumes an INTEGER's worth of space, but only ever check 4 bytes.Clearing the lower bit is strictly to accommodate Thumb -- does it take alignment faults?The conditional branch in the current code, at least for 64bit non-amd64 platforms would be gone.All targets would be the same. Clearing the lower two bits would make the address always aligned.The assumption is that if is a closure, it is already aligned and this is value preserving.If it is not a closure, this still yields a valid accessible address, just not the one to call. Or too risky in terms of portability? Better to check for alignment, assume unaligned is not a closure, and just check for a full INTEGER -1? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:47:39 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:47:39 +0000 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? Message-ID: As I remove Target variation: 1) Just assign the variables all the same in Target.i3/Target.m3?2) Change them to constants in Target.i3?3) Remove them entirely in Target and m3front? i.e. Do you think anyone will ever go back and re-diverge?And need a super local handy guide?Or can go back into history or figure it out anew? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 20 18:53:43 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 16:53:43 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned Message-ID: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 20:59:55 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 13:59:55 -0500 Subject: [M3devel] how far to reduce Target.i3/.m3? var/const/gone? In-Reply-To: References: Message-ID: <55FF022B.8090101@lcwb.coop> I think it far easier for someone in the future if there is evidence right in the current source, perhaps commented out and preferably commented about, than to try to ferret out history, even when the CM system preserves it. Some future maintainer may not even know that relevant history even exists, let alone where to look. On 09/20/2015 11:47 AM, Jay K wrote: > As I remove Target variation: > > > 1) Just assign the variables all the same in Target.i3/Target.m3? > 2) Change them to constants in Target.i3? > 3) Remove them entirely in Target and m3front? > > > i.e. Do you think anyone will ever go back and re-diverge? > And need a super local handy guide? > Or can go back into history or figure it out anew? > > > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sun Sep 20 21:05:51 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 14:05:51 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: Message-ID: <55FF038F.8030603@lcwb.coop> On 09/20/2015 11:53 AM, Jay K wrote: > I propose that Allow_packed_byte_aligned be set to FALSE for all targets. > > Or make it const. > > > Or remove it. > Also, removing it altogether from Target.i3 means the value will be copied out the place(s) where it is now used, making it even harder for someone to find. Making it all the same is OK with me, but>>> > > This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths. > Specifically, if you manage to create a packed type with unaligned fields, > presumably loads/stores get converted to multiple aligned loads/stores followed > by putting stuff back together. The occurrence of the unaligned fields should > be very rare in the first place. > Not sure what you are saying, nor whether this is true of all targets, but at least some targets will refuse a packed field that crosses a word boundary, at the point of type definition. The language explicitly permits an implementation to do this. So multi-word loads/stores wouldn't happen anyway. Within a native word, bit-twiddling works fine. > > I does not affect any other target. > > > Granted, x86 and amd64 are the most common targets. > > > Ok? > - Jay > > > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Sun Sep 20 22:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 20 Sep 2015 20:41:03 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? Message-ID: containsLazyAlignments m3front/Module.m3: PROCEDURE SetLazyAlignment (on: BOOLEAN) = BEGIN IF curModule # NIL THEN curModule.lazyAligned := on; IF on THEN curModule.containsLazyAlignments := TRUE; END; END; END SetLazyAlignment; PROCEDURE Compile (t: T) = VAR save: T; zz: Scope.T; yy: Revelation.Set; BEGIN (* ETimer.Push (M3Timers.emit); *) Target.Allow_packed_byte_aligned := t.containsLazyAlignments; This feels wrong.Surely, Allow_packed_byte_aligned should be constant for any given target?Surely a pragma cannot change a target's characteristics?Or, rather, I found this by changing it to a constant false for all targets. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 20 23:00:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 20 Sep 2015 16:00:35 -0500 Subject: [M3devel] Target.Allow_packed_byte_aligned const or var per target? In-Reply-To: References: Message-ID: <55FF1E73.1010206@lcwb.coop> Hmm, this sounds like a mechanism we had a discussion of several years ago. I don't remember how it ended, but I think there was a general consensus that it was an idea with some unresolved problems. However, one big class of use cases for packing things is to make it possible to define a record layout that matches one already specified externally, a network protocol packet, for example. We had a much bigger problem with that regarding different endianness, which we never did anything about. I have some unimplemented notes somewhere on how to do opposite-endian record layout. It was mind-boggling, to put it gently, and left me even more negative on so-called "little endian" than previously. Anyway, this is where different rules for different records on the same target may be wanted. On 09/20/2015 03:41 PM, Jay K wrote: > containsLazyAlignments > > m3front/Module.m3: > > PROCEDURE SetLazyAlignment (on: BOOLEAN) = > BEGIN > IF curModule # NIL THEN > curModule.lazyAligned := on; > IF on THEN > curModule.containsLazyAlignments := TRUE; > END; > END; > END SetLazyAlignment; > > > PROCEDURE Compile (t: T) = > VAR save: T; zz: Scope.T; yy: Revelation.Set; > BEGIN > (* ETimer.Push (M3Timers.emit); *) > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > > This feels wrong. > Surely, Allow_packed_byte_aligned should be constant for any given target? > Surely a pragma cannot change a target's characteristics? > Or, rather, I found this by changing it to a constant false for all targets. > > > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Sep 21 04:19:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:19:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , Message-ID: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:22:25 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:22:25 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Message-ID: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:41:03 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:41:03 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG Message-ID: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:49:27 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:49:27 +0000 Subject: [M3devel] M3CG_Ops vs. M3CG In-Reply-To: <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> References: , <32FAF623-B437-44F6-848D-51F9D9CF61CC@purdue.edu> Message-ID: Not gratuitous but I think I found a workaround.I will change the various M3x86/M3C/other. New functions to return M3CG_Ops.Public instead of M3CG.T.So that cm3/M3Backend.m3 can set the additional data fields, and then just returnit as the new same old fully opaque M3CG.T to the rest of cm3. I think a type with very limited visible is about as good as a type with nothing visible. When I was exploring the system, I also found this separation a slightspeed bump into learning. You look at M3CG for something useful and find nothing there.I suppose that is maybe the Modula-3 way, but it seems more obscure than usual.Usually there is just one interface for this sort of construct I think, not two. - Jay Subject: Re: [M3devel] M3CG_Ops vs. M3CG From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:44:10 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Jay, this seems like a gratuitous change. I strongly suggest not for now. Sent from my iPhone On Sep 21, 2015, at 12:41 PM, Jay K wrote: Is the distinction between M3CG.T and M3CG_Ops.T useful? Might as well just move M3CG_Ops.T and Public into M3CG? And then maybe for compatibility M3CG_Ops.T = M3CG.T; M3CG_Ops.Public = M3CG.Public; - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:51:08 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:51:08 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> Message-ID: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 04:58:42 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 02:58:42 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu> Message-ID: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:00:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:00:44 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, Message-ID: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:06:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:06:44 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, Message-ID: I suppose it is more convenient to have the C mode than I say -- it alsoputs the bootstrap vs. non-bootstrap decision in a convienent place,and the management of keep_files, and the deleting of temporary files. All that logic can be easily moved elsewhere, or not. But then we have like 6 LLVM modes.I set a bad example, and now it is even worse. Arguably we only need 2 modes -- internal and external.Any backend, internal or example, that produces assembly, canrun the assembler itself -- we could expose that in M3CG justlike I want to expose running the C compiler. I like it. Or even just one mode -- call the backend.We just need to provide in M3CG_Ops a function to write out the .mc files,that the backend can call if it needs it. i.e. for the backend call into quake/config.Each backend can put together its specific sequence.There is some commonality currently factored into Builder, but not reallymuch and it is a mess. I'm undecided at the moment. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 02:58:42 +0000 I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:21:09 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:21:09 +0000 Subject: [M3devel] globals vs. instance variables Message-ID: We have: cm3/MODULE M3Backend; ... VAR obj_file : M3ObjFile.T := NIL; obj_wr : Wr.T := NIL; obj_name : TEXT := NIL; log : Wr.T := NIL; log_name : TEXT := NIL; ...PROCEDURE Close (<*UNUSED*> cg: M3CG.T) = BEGIN IF obj_file # NIL THEN TRY NTObjFile.Dump (obj_file, obj_wr); EXCEPT Wr.Failure, Thread.Alerted => Msg.FatalError (NIL, "problem writing object file: ", obj_name); END; Utils.CloseWriter (log, log_name); obj_file := NIL; obj_wr := NIL; obj_name := NIL; log := NIL; log_name := NIL; END; END Close; Surely we should add those variables to M3CG_Ops in orderto get them to Close for this specific M3CG? i.e. prefer "instance variables" over "module variables" aka globals. Perhaps in a record called M3Backend to give an appearance of opacityor perhaps in an actual opaque type, though I hate to pay for theextra heap allocation just to achieve opacity. (Larger point is the cm3 is written an old unfortunate thread-unsafe style with many globals.) And this is a good reason then to smush M3CG.T and M3CG_Ops.T together,in order for M3Backend.New() result to be passed back to Close. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:40:46 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:40:46 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , Message-ID: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 06:45:37 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 04:45:37 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu> Message-ID: There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:41:38 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:41:38 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , <3872F2D2-832C-4E22-B911-B0F122B9A054@purdue.edu> Message-ID: Sorry, clarification: struct A { char a; int b; }; is defined and fairly portable in C, and means generally the same thing onall platforms (ignoring 16bit int), the same thing as: RECORD a: Ctypes.char b: Ctypes.intEND; what is not portable in C is expecting this to have other than size 8. Strictly speaking, its size is completely not portable, but on the vast vast majorityof systems its size is 8, and getting it less requires compiler-specific mechanisms. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:50 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu But it us defined in C, just target-specific. Sent from my iPhone On Sep 21, 2015, at 2:00 PM, Jay K wrote: ps, you can't even do this portably in C. struct A { char a; int b; }; will have padding between a and b on all systems. The way to eliminate it varies by compiler, #pragma pack for Visual C++, something else with gcc. And might result in an alignment fault. And we do have this wacky LAZYALIGN pragma that that might make it work. Really that pragma is weird. You might get the packed layout, but you'll also get alignment faults on some targets, unless we decompose the reads into byte by byte -- assuming this isn't hardware access that demands a single access. In C you have a chance with: struct A{ char a; char b[4];}; and then maybeA a; *(int*)&a.b; but it will fault on most target -- but not x86/amd64. If you don't need one read, you can do: int c = a.b[0] b <<= 8; b |= a.b[1]; b <<= 8; b |= a.b[2]; b <<= 8; b |= a.b[3]; or such. You can do like that in Modula-3 also. So I don't see a strong need to support these things. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 02:51:08 +0000 I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 07:56:00 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 05:56:00 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:04:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? Message-ID: For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:07:21 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: Message-ID: Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 09:21:47 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 07:21:47 +0000 Subject: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? In-Reply-To: References: , Message-ID: er, ok, I partly did this. https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3.diff?r1=1.95;r2=1.96;f=u Revision 1.96: download - view: text, markup, annotated - select for diffsThu Apr 1 17:56:37 2010 (5 years, 5 months ago) by jkrellBranches: MAINDiff to: previous 1.95: preferred, unifiedChanges since revision 1.95: +1 -9 lines leave max_align = 64 for all platformsthis effects: I386_FREEBSD / FreeBSD4 I386_LINUX / LINUXLIBC6 NetBSD2_i386 It does not affect any others -- not PPC, not Darwin, not 64bit, not Solaris, not NT. but it wasn't consistent across 32bit platforms or x86.Nor should it necessarily -- I recall Cygwin wants to alignmore than cm3, implying NT does. - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:07:21 +0000 Subject: Re: [M3devel] LONGINT/LONGREAL alignment on 32bit targets? Hm. Same thing with double/LONGREAL.Do we need to fix this??Or just use it as evidence that our interop isn't perfect but adequate? jair:1 jay$ cat 1.c src/Main.m3#include int main(){printf("%d\n", (int)sizeof(struct {int a; double b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGREAL; END)));IO.Put("\n");END Main. jair:1 jay$ rm -rf I386_DARWIN/jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a16jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12jair:1 jay$ - Jay From: jay.krell at cornell.edu To: m3devel at elegosoft.com Date: Mon, 21 Sep 2015 07:04:49 +0000 Subject: [M3devel] LONGINT alignment on 32bit targets? For the record, I didn't do this, way back even in: https://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-sys/m3middle/src/Target.m3?rev=1.1;content-type=text%2Fplain we had: (* build a generic 32-bit/IEEE system description *) Int64.cg_type := CGType.Int64; Int64.size := 64; Int64.align := 64; Int64.min.x := IChunks { 00, 00, 00, 16_8000 }; Int64.max.x := IChunks { FF, FF, FF, 16_7fff }; This is likely wrong on many/most/all 32bit targets. Int64 is likely to be loaded/stored with two 32bit operations, therefore align probably should be 32. At the time, I don't think there was any way to use this type, i.e.there was no LONGINT. Once LONGINT came into being, if you wrote: foo.m3RECORD a: Ctypes.int; b: LONGINTEND; and foo.c:struct { int a; long long b;}; you would probably get different layout. Let's see: jair:src jay$ cat 1.cint a = sizeof(struct { int a; long long b;});jair:src jay$ cc -c -S -m32 1.cjair:src jay$ cat 1.s .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 10 .section __DATA,__data .globl _a ## @a .align 2_a: .long 12 ## 0xc jair:1 jay$ cat 1.c src/Main.m3 src/m3makefile #include int main(){printf("%d\n", (int)sizeof(struct {int a; long long b;}));return 0;} MODULE Main;IMPORT IO, Fmt, Ctypes; BEGINIO.Put(Fmt.Int(BYTESIZE(RECORD a:Ctypes.int; b:LONGINT; END)));IO.Put("\n");END Main. import("m3core")import("libm3")implementation("Main")build_standalone()program("a") jair:1 jay$ cc -m32 1.cjair:1 jay$ ./a.out12 jair:1 jay$ cm3 -DTARGET=I386_DARWIN --- building in I386_DARWIN --- new source -> compiling Main.m3 -> linking ajair:1 jay$ ./I386_DARWIN/a 16 Now, if you are going to use x86 interlocked instructions on this,you might need higher alignment anyway. Might. (Which all goes to my point -- while layout rules can't actually be removed,they have to exist, it does behoove one to try to layout data that hasobvious portable layout, if you need interop. i.e. you just would not everhave a record like this. You would make the likely padding explicit.Or sort by size. Though once you put records within records, it becomesdifficult/impossible.) - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 16:41:02 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 14:41:02 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, Message-ID: I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:12 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:12 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , Message-ID: I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:31:53 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:31:53 +0000 Subject: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) In-Reply-To: References: , <512508A0-4E9F-4857-9981-3C3E63A96C36@purdue.edu>, , <566AE45A-AFA8-4DB2-9561-D1D97A66C28B@purdue.edu>, Message-ID: I guess it makes sense to be in Builder, just that it can be a separate booland not a mode. The code will almost be identical to today, and not clearlydemonstrate the lack of a need for a mode. Alas. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) Date: Mon, 21 Sep 2015 04:45:37 +0000 There is more value in putting this in builder -- logic around file name formation, "keep", and "bootstrap".I'm undecided. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:28:34 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Oh ok. Sent from my iPhone On Sep 21, 2015, at 12:58 PM, Jay K wrote: I believe we really only need 4 modes.Internal or Externalproduces object or assembly All the other modes are unnecessary.I will do my part and eliminate the one I added.Hopefully leading the way to eliminating all the LLVM modes. C backend certainly supported, I just didn't expose it in the ideal way. - Jay Subject: Re: [M3devel] elimining builder.C mode (instead IntegratedObject and another boolean) From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:49:19 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu What is the benefit of doing this? It would seem to me to be nicer to retain the existing modes. Sent from my iPhone On Sep 21, 2015, at 12:22 PM, Jay K wrote: I want remove the C mode from the builder and just use IntegratedObject.The goal is to expose Builder.RunCC to M3C. Something like: add some data to MC3CG_Op.TI wouldn't bother with "set" functions. I'm also not making them module global -- this could be globals in M3C.But I want to preserve the illusion that things can be easily made multi-threaded (m3front structuring very much to the contrary...) The data will be: declare Builder.State <: REFANY in Builder.i3 M3CG_Ops should get RunCCState: Builder.State; OR BuilderState: Builder.State; keep_files: BOOLEAN; PROCEDURE RunCC (s: State; source, object: TEXT; debug, optimize: BOOLEAN; include_path: Arg.List): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source, object: TEXT): BOOLEAN (* Success. *); or possibly: PROCEDURE RunCC (s: State; source: TEXT; u: M3Unit.T): BOOLEAN (* Success. *) include_path isn't really needed here. also: M3Backend.PROCEDURE Open (target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; should be changed to: PROCEDURE Open (m3file: TEXT; target: Wr.T; target_name: TEXT; backend_mode: M3BackendMode_t): M3CG.T; ok? - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Sep 21 18:37:18 2015 From: jay.krell at cornell.edu (Jay K) Date: Mon, 21 Sep 2015 16:37:18 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , Message-ID: Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. Perhaps m3front can behave similarly?For all targets? But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 16:31:12 +0000 I guess we have other problems here -- packed records in the C backend. But maybe we can provide some good value here. In particular: - I can put in the compiler-specific stuff, like: #if _MSC_VER #pragma pack(1) #endif at the top of every file. Something for other compilers. - In the frontend, doing layout, if any padding is inserted in a record, insert an array of chars. - generate static asserts that front end layout matches C layout OR maybe just accept C layout. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 14:41:02 +0000 I was asked what the point is: To have just one or two C outputs for distribution/bootstrapping. Another option is I can verify that the n different outputs are the same and then eliminate them afterward.i.e. verify this setting makes no different for wide swaths of code, but still leave it there for other code. Again, per-target packing rules don't allow the records to be as easily written by one target and read by another. Granted, such binary I/O is always problematic. Textual I/O is commonly used instead. - Jay From: jay.krell at cornell.edu To: hosking at purdue.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] Target.Allow_packed_byte_aligned Date: Mon, 21 Sep 2015 05:56:00 +0000 "stat" used to be packed and it was bad.It is no longer. Let's say I'm interfacing with hardware.A lot of hardware can be presented to a variety of processors.For example, every x86, amd64, arm32, and arm64 device probably has USB.So target-specific rules don't help. Let's say I'm interfacing with a file format.Again, target-specific rules are clearly wrong. The only place this really makes sense is interfacingwith things like x86 interrupt descriptor tables and such (which do haveunnatural layout I believe).This can be handled by using chars and combing them, orcoming up with C code that uses whatever compiler-specific mechanism. What is difficult to achieve is unnatural alignment and atomic access,even though x86/amd64 I guess support that -- at least as long as youaren't crossing a cache line. If you are crossing a cache line, thenI think one instruction is still going to exhibit non atomic access, at whichpoint you might as well read individual byte and reconstruct the larger integer. And Modula-3 does have pragmas for types.<* lazyalign *> and <* strictalign *> Arguably that is a better method -- as it might still give the same IR and Cacross all targets. As well, let's say I am defining a file format.For this, it helps to have the same layout/alignment rules across all targets.Though I'm torn on this -- the alignment of a 64bit integer shouldtypically be only 4 bytes on a 32bit target, so something like: RECORD a: Ctypes.int; b:LONGINT END; should/can be smaller on a 32bit target. I believe with the changes I made years ago,they will have the same layout, for better and worse.Given that I can't likely unify IR/C for word size, maybe we shouldfix that. I should review the Win32 .i3 files for this, though it probablydoesn't occur, given the recentness of LONGINT. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:57:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Sent from my iPhone On Sep 21, 2015, at 2:40 PM, Jay K wrote: You mean, the source code will be in target-specific directories?Isn't expected to be portable anyway?Yet we should still provide a way to do it? Yes to all. M3 is meant to be a systems programming language and packed types are the way you handle layout for specific targets. Are you sure we don't have packed types in the libraries? I thought we used to. I'm kinda keen on just not having them.Very little code needs them. I'd like to eliminate target-specificity, so that C code from the C backendcan be used to bootstrap any platform. I'm resigned to not quite make it there,but I don't want unnecessary variation anyway. I think can I decree the C backend is always big endian.But word size I can't eliminate due to access to external data structureswith size_t/integer. So we'd always have at least two C targets -- 32bitand 64bit. Non-C targets can retain their "correct" endianness.They don't interoperate anyway because of nested function implementation varying. Again, this was only true for x86/amd64.And there is a strange pragma to allow it on modules or types.Instead of always on the target. Allowing it on types is kinda close to what Visual C++ offers -- something scopedinstead of global. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 14:30:15 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed will generally be target-specific no? Sent from my iPhone On Sep 21, 2015, at 12:51 PM, Jay K wrote: I don't believe this is portable.Such code would only compile for x86/amd64 targets.I think.We want to cater to such target-specific constructs?I should confirm it is target-specific?I want to eliminate gratuitous target-specific aspects and devolve the IR to having essentially 3 variables -- word size and endian and NT vs. posix. Fewer if I can manage to. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 12:47:07 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu Packed types are part of the language and their behavior should be preserved. Just because we don't use it on the libraries others do! Sent from my iPhone On Sep 21, 2015, at 12:19 PM, Jay K wrote: I don't think so.I built the system with it.Granted, only one target. If we had something likeTYPE T = BITS BITSIZE(INTEGER) + 8 FOR PACKED RECORD a: CHAR b:INTEGEREND; I think it'd break that.I'm not sure we any longer use packed types.And such things would only work for x86/amd64 I believe. - Jay Subject: Re: [M3devel] Target.Allow_packed_byte_aligned From: hosking at purdue.edu Date: Mon, 21 Sep 2015 10:44:59 +1000 CC: m3devel at elegosoft.com To: jay.krell at cornell.edu We should be careful here. Might break something, no? Sent from my iPhone On Sep 21, 2015, at 2:53 AM, Jay K wrote: I propose that Allow_packed_byte_aligned be set to FALSE for all targets. Or make it const. Or remove it. This will degrade slightly x86/amd64 but in, I speculate, fairly rare paths.Specifically, if you manage to create a packed type with unaligned fields,presumably loads/stores get converted to multiple aligned loads/stores followedby putting stuff back together. The occurrence of the unaligned fields shouldbe very rare in the first place. I does not affect any other target. Granted, x86 and amd64 are the most common targets. Ok? - Jay ! _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Sep 21 22:25:49 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 21 Sep 2015 16:25:49 -0400 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu> <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu> Message-ID: <20150921202548.GB20914@topoi.pooq.com> On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > Perhaps m3front can behave similarly?For all targets? > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > - Jay The fancy packing should remain in the language, for backward compatibility if nothing else. modula 3 is, after all, a systems language. But is likely no need to worry it in the bootstrap. If the bootstrap doesn't use it, it will still work portably as a bootstrap, no matter how machine-dependent fancy packing happens to be. -- hendrik From jay.krell at cornell.edu Tue Sep 22 02:15:05 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: <20150921202548.GB20914@topoi.pooq.com> References: , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , <20150921202548.GB20914@topoi.pooq.com> Message-ID: Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Sep 22 06:55:50 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 22 Sep 2015 04:55:50 +0000 Subject: [M3devel] Target.Allow_packed_byte_aligned In-Reply-To: References: , , <108D09B7-F5AE-48D5-B908-64B99D9846BE@purdue.edu>, , , , , , , , <3C416648-07EE-472D-9625-A7891A05F9E5@purdue.edu>, , , , , , , , , , <20150921202548.GB20914@topoi.pooq.com>, Message-ID: There is a proposal in here..here is a variation of it.. Allow packing of records tightly on all targets. All targets get the same layout (modulo word size), the same things can compile successfully. Remove the caveat here that certain code will only compile for certain targets. If Allow_packed_byte_aligned is false, and there is an unaligned read/write, there are a few choices. m3front could transform into multiple inlined read/writes byte-wise taking the value apart of putting it back together. Or m3front could issue function calls. The C implementation could be #ifdefed such that amd64/x86 just do the read/write, albeit still with a function call. C backend could set allow_packed_byte_aligned the same always. In that case, the functions would be output into the file and inlined or macroed. A warning might be issued about the lack of atomicity. Heck, let's invent a feature...a pragma could be allowed on integral/floating point values as to endianness and compiler could output the byte swaps for you. The endianness could actually be some sort of runtime expression. That's how some formats work -- they get written native, with an indicator as to which that is, and swapped if necessary upon reads, and maybe swapped back for writes. You know -- if you really want a powerful system for efficient binary persistance.. Or just leave it alone. ? - Jay From: jay.krell at cornell.edu To: hendrik at topoi.pooq.com; m3devel at elegosoft.com Date: Tue, 22 Sep 2015 00:15:05 +0000 Subject: Re: [M3devel] Target.Allow_packed_byte_aligned Right -- I had what I think is the same idea -- I can build the n bootstrap archives, compare them, and verify where they are the same and where they are different, and then end up with fewer archives, maybe 6, maybe 2. For that matter, I guess I should get this stuff to work in the C backend --- providing the #pragma pack(1) or whatever gcc/clang equivalent. - Jay > Date: Mon, 21 Sep 2015 16:25:49 -0400 > From: hendrik at topoi.pooq.com > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Target.Allow_packed_byte_aligned > > On Mon, Sep 21, 2015 at 04:37:18PM +0000, Jay K wrote: > > Also -- if anyone has a proposal that would allow Allow_packed_byte_aligned to always be true, that would also fit my agenda. I don't care true or false, I just want it the same across all targets. > > In particular, on IA64/NT you can say stuff like: #pragma pack(1) struct { char a; __unaligned int b; } A; A.b > > and you do get a 5 byte struct and b is at offset 1.The generated code will read b with multiple reads of bytes or shorts and reassemble the 4 byte integer. > > Perhaps m3front can behave similarly?For all targets? > > But for now I'll put it back and later when I generate bootstraps I'll see if they are equal, i.e. contain no packed types. > > - Jay > > The fancy packing should remain in the language, for backward > compatibility if nothing else. modula 3 is, after all, a systems > language. > > But is likely no need to worry it in the bootstrap. > > If the bootstrap doesn't use it, it will still work portably as a > bootstrap, no matter how machine-dependent fancy packing happens to be. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Sep 26 21:23:27 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sat, 26 Sep 2015 12:23:27 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150926122327.F8299C6A@m0087794.ppops.net> An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Sep 26 22:30:26 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 16:30:26 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: <20150926203026.GB29515@topoi.pooq.com> Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From dabenavidesd at yahoo.es Sat Sep 26 23:57:46 2015 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 26 Sep 2015 21:57:46 +0000 (UTC) Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926203026.GB29515@topoi.pooq.com> References: <20150926203026.GB29515@topoi.pooq.com> Message-ID: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> not here El S?bado 26 de septiembre de 2015 15:32, Hendrik Boom escribi?: Your mailer seems to have slipped into HTML mode. On Sat, Sep 26, 2015 at 12:23:27PM -0700, Rodney Bates wrote: >
In trying to better understand the issue, I see, in Module.m3:895, this code:

    Target.Allow_packed_byte_aligned := t.containsLazyAlignments;

From other places, containsLazyAlignments derives from LAZYALIGN
pragmas, so this means it will turn on Allow_packed_byte_aligned,
regardless of the target, if there is a LAZYALIGN pragma.  I think this
will generate code that will alignfault, if the target CPU doesn't
support it.  That would be a bug.


-Rodney Bates
> _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Sep 27 01:46:27 2015 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 26 Sep 2015 19:46:27 -0400 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> References: <20150926203026.GB29515@topoi.pooq.com> <420248670.1870164.1443304666445.JavaMail.yahoo@mail.yahoo.com> Message-ID: <20150926234627.GB2740@topoi.pooq.com> On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik From lists at darko.org Sun Sep 27 03:27:59 2015 From: lists at darko.org (Darko Volaric) Date: Sat, 26 Sep 2015 18:27:59 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: <20150926122327.F8299C6A@m0087794.ppops.net> References: <20150926122327.F8299C6A@m0087794.ppops.net> Message-ID: I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: > In trying to better understand the issue, I see, in Module.m3:895, this > code: > > Target.Allow_packed_byte_aligned := t.containsLazyAlignments; > > From other places, containsLazyAlignments derives from LAZYALIGN > pragmas, so this means it will turn on Allow_packed_byte_aligned, > regardless of the target, if there is a LAZYALIGN pragma. I think this > will generate code that will alignfault, if the target CPU doesn't > support it. That would be a bug. > > > -Rodney Bates > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:17:14 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:17:14 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, Message-ID: I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sun Sep 27 07:21:34 2015 From: jay.krell at cornell.edu (Jay K) Date: Sun, 27 Sep 2015 05:21:34 +0000 Subject: [M3devel] Another Allow_packed_byte_aligned issue In-Reply-To: References: <20150926122327.F8299C6A@m0087794.ppops.net>, , , Message-ID: I forgot to mention/ask: "packed" basically means "no padding", right? - Jay From: jay.krell at cornell.edu To: lists at darko.org; rodney_bates at lcwb.coop Date: Sun, 27 Sep 2015 05:17:14 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:03:20 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:03:20 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927080320.F82B920B@m0087798.ppops.net> An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Sep 27 17:10:28 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:10:28 -0700 Subject: [M3devel] HTML mail (was Re: Another Allow_packed_byte_aligned issue) Message-ID: <20150927081028.F82B92BC@m0087798.ppops.net> FWIW, I am traveling and using a webmail that is different from my usual email client. I did find and change a preference setting that sounded like it might switch to plain text. -Rodney Bates --- hendrik at topoi.pooq.com wrote: From: Hendrik Boom To: "m3devel at elegosoft.com" Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue Date: Sat, 26 Sep 2015 19:46:27 -0400 On Sat, Sep 26, 2015 at 09:57:46PM +0000, Daniel Alejandro Benavides D. wrote: > not here So you got it as plaintext. Interesting. My email reader operates on a text-only terminal. I got a two-part message. The first was HTML, starting with
and ending with
, though I could cozy it into a browser and read it properly. The second part was the mailing list's signature. And that was all. -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Sun Sep 27 17:21:06 2015 From: rodney_bates at lcwb.coop (Rodney Bates) Date: Sun, 27 Sep 2015 08:21:06 -0700 Subject: [M3devel] Another Allow_packed_byte_aligned issue Message-ID: <20150927082106.F82B9288@m0087798.ppops.net> -Rodney Bates --- jay.krell at cornell.edu wrote: From: Jay K To: Darko Volaric , Rodney Bates CC: m3devel Subject: RE: [M3devel] Another Allow_packed_byte_aligned issue Date: Sun, 27 Sep 2015 05:17:14 +0000 I agree it is probably messed up, but I wonder if another approach or approaches can/should be taken. 1) I wonder if the boolean should be true for all targets.And the pragmas removed, or changed to do nothing. And, here comes the important part: for any unaligned load/store, m3front break it down into multiple smaller loads/stores Upsides: same predictable layout for all targets same does-it-compile/does-it-not-compile behavior for all targets Downsides: possibly atomic accesses are no longer atomic possibly "efficient" accesses are no longer atomic. I'm just speculating here, but I would think that if it crosses a boundary of the hardware RAM word, (not necessarily the same as a CPU native word) it would be non-atomic anyway, regardless of whether the compiler or the CPU took care of juggling the separate pieces. In fact, I get the impression that, even within a single RAM word, you can't count on atomicity of even an aligned, single-word load or store, with modern multi-CPUs, multi-level hardware caching, etc. 2)Similar to before: allow same layouts on all architecures but on architectures where alignment doesn't matter much, just do the unaligned loads/stores Keep in mind that "interlocked" alignment does reportedly matter on x86 -- x86 is not really entirely alignment-agnostic.I believe alignment of avx loads/stores sometimes matters too -- again, x86 does care sometimes. And the rules are more complicated than m3front/m3middle can encode. Not that we have any avx data types.I'd have to experiment more on the sse2 stuff which we do presumably use. And yes I agree it generates alignment faults on some targets.We should be able to reproduce this easily enough e.g. on SPARC. (Not to mention that x86 can be asked to take alignment faults...it is a mode bit in the processor, that justtends to be off.) - Jay Date: Sat, 26 Sep 2015 18:27:59 -0700 From: lists at darko.org To: rodney_bates at lcwb.coop CC: m3devel at elegosoft.com Subject: Re: [M3devel] Another Allow_packed_byte_aligned issue I implemented the byte aligned packing functionality and the Allow_packed_byte_aligned flag was supposed to be a constant that signified that the target CPU allows byte alignment thus relaxing the normal packing restrictions. I can't speak for the implementation of the pragma (or later changes), but I think it should be removed since (in my mind) it doesn't serve any useful purpose. On Sat, Sep 26, 2015 at 12:23 PM, Rodney Bates wrote: In trying to better understand the issue, I see, in Module.m3:895, this code: Target.Allow_packed_byte_aligned := t.containsLazyAlignments; >From other places, containsLazyAlignments derives from LAZYALIGN pragmas, so this means it will turn on Allow_packed_byte_aligned, regardless of the target, if there is a LAZYALIGN pragma. I think this will generate code that will alignfault, if the target CPU doesn't support it. That would be a bug. -Rodney Bates _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel