From jayk123 at hotmail.com Mon Mar 1 05:24:11 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 04:24:11 +0000 Subject: [M3devel] etimer Message-ID: Sorry to be against change but I think the ETimer change broke the usual bootstrap order. That is, m3quake requires newer libm3. Maybe no big deal. You can update libm3 first. Maybe cause to make more releases. I was going to suggest a new bootstrap procedure, where we just build the world, but that violates another feature, that of which, which language features can be used where. The current procedure enables m3core/libm3 to use new language features, because they do not have to be built with old compiler. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 1 06:37:32 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 05:37:32 +0000 Subject: [M3devel] etimer In-Reply-To: References: Message-ID: By my estimation, you must update libm3 to commit c62c53ddfc204b0623db343469db6418d37e527d prior to upgrade past c62c53ddfc204b0623db343469db6418d37e527d. In general commit arbitrary X to commit arbitrary Y is NOT supported. But very many do happen work and people get used to it. In general commit arbitrary X to commit X + 1 IS supported. And there is no exception here. That was not broken. You checkout c62c53ddfc204b0623db343469db6418d37e527d. Upgrade.py (from "something"). Checkout master. Upgrade.py. We do NOT TEND to document required waypoints. But there have been a few through the years. One suggestion is we make some sort of release at commit c62c53ddfc204b0623db343469db6418d37e527d. Another suggestion is change isn't that worthwhile. Revert and revisit. But this suggestion could include, you know, unifying the small mess of releases I made -- do it for a bunch of architectures at once, including the C bootstraps. Or primarily the C bootstraps (since they are so easy to cross build :) ) - Jay ________________________________ From: Jay K Sent: Monday, March 1, 2021 4:24 AM To: Peter McKinna ; m3devel Subject: etimer Sorry to be against change but I think the ETimer change broke the usual bootstrap order. That is, m3quake requires newer libm3. Maybe no big deal. You can update libm3 first. Maybe cause to make more releases. I was going to suggest a new bootstrap procedure, where we just build the world, but that violates another feature, that of which, which language features can be used where. The current procedure enables m3core/libm3 to use new language features, because they do not have to be built with old compiler. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 1 07:56:11 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 06:56:11 +0000 Subject: [M3devel] quake directory walk to replace scripts? Message-ID: Instead of the scripts, I propose the following: m3-sys/m3makefile: include_dir("m3quake") % maybe a different name, like package_dir("m3quake") ditto("m3front") % possibly an actual command, or include_dir, or other, like package_dir etc. m3-libs/m3makefile: include_dir("m3core") % ditto include_dir("libm3") gitroot/m3makefile: include_dir("m3-sys") % ditto-ish -- not "package" but something else? include_dir("m3-libs") % ditto-ish -- not "package" but something else? etc. As part of this, a few details: - The Program() and Library() directives, if they don't already, should act on and flush and reset state. - This is somewhat/completely predicated on them being last. - If you look at e.g. m3quake\src\m3makefile you see that program/library are not always last. Maybe that is ok. Maybe some quake functions are fairly stateless, e.g. ManpPage and this proposal remains easy. If we make up new names for include_dir, the notion of flush+act+reset can be easier. Specifically, you would flush+act+reset up on the next something_dir() -- end of previous implied by start of next, rather than library/program. use-cases would include: cd gitroot cm3 => build entire tree, in dependency order cd gitroot cm3 -dir:m3-sys/cm3 => build entire compiler, in dependency order (starting with m3core) cd gitroot cm3 -dir:m3-sys/cm3;m3-gui/tetris => build entire compiler and tetris, in dependency order (starting with m3core) cd gitroot cm3 -build -ship -destdir $TEMP/cm3boot1 create $TEMP/cm3boot1 and build and ship entire system into it $TEMP/cm3boot1 is preferably?required empty? Even if not shipping, this install root should be a command line option. Maybe destdir not the right name. We have CM3_INSTALL_PREFIX but environment variables maybe do not suffice if considering Windows. Similarly, a command line option to replace M3CONFIG would be good. -D might suffice for both though. Probably. cd gitroot cm3 -build -ship -testdir $TEMP/cm3boot1 -dir m3-sys/cm3 create $TEMP/cm3boot1 and build and ship up to cm3 cd gitroot cm3 -build -ship -testdir $TEMP/cm3boot1 -dironly m3-sys/cm3;m3-sys/m3front create $TEMP/cm3boot1 and build and ship only cm3 and m3front This is risky but can be useful. cd gitroot cm3 -build -ship -destdir $TEMP/cm3boot1 -dir m3-sys/cm3 -skipdir m3-sys/m3front;m3-sys/m3middle create $TEMP/cm3boot1 and build and ship up to cm3, skipping m3front and m3middle This is risky but can be useful, maybe. cd gitroot/m3-libs/m3core cm3 -build -ship -destdir $TEMP/cm3boot1 cd gitroot/m3-libs/libm3 cm3 -build -ship -destdir $TEMP/cm3boot1 cd gitroot/m3-libs/cm3 cm3 -build -ship -destdir $TEMP/cm3boot1 => No implied dependency walk? Risky, but partially simulates old scripts. But still useful to have the destdir switch. Should -ship imply -build? This is part of a larger proposal but I wanted to split it up. Ideally compiler could run in parallel across packages, but that isn't likely to be implemented ever. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 1 08:04:33 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 07:04:33 +0000 Subject: [M3devel] new build/bootstrap/setup design? Message-ID: A new vision for cm3 initial build? Here is a proposal. The goals are: - Users do not require prior cm3. - Reduce redistribution of generated files to a minimum (i.e. not binaries usually). - Resemble the 3.6 release, somewhat. The 3.6 release worked like this, roughly, as I recall. - Download target-specific assembly. - From the assembly, build cm3 (er, m3build & m3ship) - From the assembly output, build the rest of the tree, perhaps, I don't remember, rebuilding the assembly output. - Specifically, no prior cm3 required (ignoring how the assembly was produced). The plan would be: Distribute C archives only. Likely just a few: cm3-unix64le.tgz (almost everything current -- for amd64, arm64, riscv64, ppc64le) cm3-win64.tgz (little endian, for amd64 and arm64) -- Maybe stop here; the rest can work, but might be unnecessary cost/confusing. -- cm3-unix32le.tgz (almost everything old) cm3-win32.tgz (little endian; Windows big endian is very obscure/obsolete, i.e. CE and 360). cm3-unix64be.tgz (sparc64, hppa64, macppc64 -- not particularly used stuff) cm3-unix32be.tgz (sparc32, hppa32, macppc32 -- not particularly used stuff) They shall include a generated CMakeLists.txt. This is new and undeveloped. Along with a full source tree, granted, or separate. The approximate steps..which are long enough to require automation.. possibly related to another email I sent. # extract and build C bootstrap, providing cm3, in path tar xf cm3-unix64le.tgz cd cm3-unix64le cmake . cmake install destir=$(pwd)/boot1 PATH=$(pwd)/boot1/bin:$PATH # build cm3 with self, put it in path, remove previous, etc. # Near end of this, also, the .c files should exist that match cm3-unixle.tgz, # satisfying *some* requirements to get into *some* Linux distributions. tar xf ../cm3-src-all.tgz cd cm3-src-all cm3 -x -build -ship -destdir $(realpath $(pwd)/../boot2) -dir m3-sys/cm3 # This implies a directory walk w/o scripts on the side. Maybe those are ok, maybe not. PATH=$(pwd)/../boot2/bin:$PATH rm -rf * # scary, maybe cm3 -clean or -realclean instead rm -rf ../boot1 # just in case, still in path # build entire system with cm3-built-cm3; possibly skipping what already built, possibly tar xf ../cm3-src.tgz cm3 -build -x cm3 -ship -destdir /usr/local/cm3 cm3 should be willing to ship itself, i.e. when to a different directory or something, maybe always The first thing to get done is unifying the C code across the targets. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 1 08:09:15 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 07:09:15 +0000 Subject: [M3devel] output directory names Message-ID: Presently we do things like: - Produce not-exactly target-specific output, in target-specifid named directories. - This is both the .c files and the instantiated generics. Should we consider a structure like: src\foo.m3 out\i386_linux\foo.o out\unix32le\foo.c out\ArrayInteger.m3 # generic instantation The point, esp. of the last, is two part: - You could reuse identical files across targets; hopefully they really are identical. You could produce them in i386_linux and then compare with existing, including putting an input hash in the output, and if the input hash matches and the output does ont, eror - The target-specific paths I mentioned earlier in generic instantiations would go away. It is risky. I've seen other systems try to put non-target-specific outputs in non-target-specific directories. The risks include: - inability to build them in parallel - no verification that the files are in fact the same. My real goal is to make sure "unix64le.tar.gz" is the same across all targets, so the build/redistribution matrix is cut down. Perhaps another way to do this, is actually to introduce such targets? The then, if you actually compile and link the C, what directory to put it in? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 1 09:44:32 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 01 Mar 2021 11:44:32 +0300 Subject: [M3devel] Modula-3 on ReactOs ( "free Windows") and(or) OpenNT Re: new build/bootstrap/setup design? In-Reply-To: References: Message-ID: <71411614587740@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 1 11:01:39 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 10:01:39 +0000 Subject: [M3devel] Modula-3 on ReactOs ( "free Windows") and(or) OpenNT Re: new build/bootstrap/setup design? In-Reply-To: <71411614587740@mail.yandex.by> References: , <71411614587740@mail.yandex.by> Message-ID: I'm just trying to limit release forms, to reduce confusing, not lingering internal support. The system is quite portable and is likely to remain so. Alpha was big endian. Nobody has ported to NT/Alpha, or CE yet, nobody probably ever will, but it is probably easy. > PATH=$(pwd)/boot1/bin:$PATH I was just speaking approximately. I think the C bootstrap should use cmake, and really, probably, quake should result in generating cmake (which then generates...) and all that platform specific Quake/config code could go away. It'd be nice. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 1, 2021 8:44 AM To: Jay K ; m3devel Subject: Modula-3 on ReactOs ( "free Windows") and(or) OpenNT Re: [M3devel] new build/bootstrap/setup design? Hi! This may be problematic for "usual Windows user": == PATH=$(pwd)/boot1/bin:$PATH == > Windows big endian is very obscure/obsolete, i.e. CE and 360). I'm not sure: Windows CE steal ( potentially) useful on devices with small amount of RAM. And, please, don't forgot about OpenNT and(or) ReactOs ( "free Windows"). > cm3-unix64le.tgz (almost everything current -- for amd64, arm64, riscv64, ppc64le) This is include DEC Alpha? > cm3-unix32be.tgz (sparc32, hppa32, macppc32 -- not particularly used stuff) This steal ( potentially) useful for conversion source from Modula-2 to Modula-3. Best regards, Victor Miasnikov 01.03.2021, 10:05, "Jay K" : A new vision for cm3 initial build? Here is a proposal. The goals are: - Users do not require prior cm3. - Reduce redistribution of generated files to a minimum (i.e. not binaries usually). - Resemble the 3.6 release, somewhat. The 3.6 release worked like this, roughly, as I recall. - Download target-specific assembly. - From the assembly, build cm3 (er, m3build & m3ship) - From the assembly output, build the rest of the tree, perhaps, I don't remember, rebuilding the assembly output. - Specifically, no prior cm3 required (ignoring how the assembly was produced). The plan would be: Distribute C archives only. Likely just a few: cm3-unix64le.tgz (almost everything current -- for amd64, arm64, riscv64, ppc64le) cm3-win64.tgz (little endian, for amd64 and arm64) -- Maybe stop here; the rest can work, but might be unnecessary cost/confusing. -- cm3-unix32le.tgz (almost everything old) cm3-win32.tgz (little endian; Windows big endian is very obscure/obsolete, i.e. CE and 360). cm3-unix64be.tgz (sparc64, hppa64, macppc64 -- not particularly used stuff) cm3-unix32be.tgz (sparc32, hppa32, macppc32 -- not particularly used stuff) They shall include a generated CMakeLists.txt. This is new and undeveloped. Along with a full source tree, granted, or separate. The approximate steps..which are long enough to require automation.. possibly related to another email I sent. # extract and build C bootstrap, providing cm3, in path tar xf cm3-unix64le.tgz cd cm3-unix64le cmake . cmake install destir=$(pwd)/boot1 PATH=$(pwd)/boot1/bin:$PATH # build cm3 with self, put it in path, remove previous, etc. # Near end of this, also, the .c files should exist that match cm3-unixle.tgz, # satisfying *some* requirements to get into *some* Linux distributions. tar xf ../cm3-src-all.tgz cd cm3-src-all cm3 -x -build -ship -destdir $(realpath $(pwd)/../boot2) -dir m3-sys/cm3 # This implies a directory walk w/o scripts on the side. Maybe those are ok, maybe not. PATH=$(pwd)/../boot2/bin:$PATH rm -rf * # scary, maybe cm3 -clean or -realclean instead rm -rf ../boot1 # just in case, still in path # build entire system with cm3-built-cm3; possibly skipping what already built, possibly tar xf ../cm3-src.tgz cm3 -build -x cm3 -ship -destdir /usr/local/cm3 cm3 should be willing to ship itself, i.e. when to a different directory or something, maybe always The first thing to get done is unifying the C code across the targets. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 1 16:13:59 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 01 Mar 2021 18:13:59 +0300 Subject: [M3devel] Modula-3 on ReactOs ( "free Windows") and(or) OpenNT Re: new build/bootstrap/setup design? In-Reply-To: References: , <71411614587740@mail.yandex.by> Message-ID: <173741614602286@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 2 00:08:38 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 23:08:38 +0000 Subject: [M3devel] increment version? Message-ID: I acknowledge that versions are meaningless. I acknowledge that many people subscribe to "semantic versioning". I assert that "semantic versioning" misses the point, that all changes can be described as "breaking", more or less. Illusions to the contrary tend to come from working on smaller systems. "better" and "worse", "bugs" and "features" are just a continuum and there is just change and subjective judgement. I acknowledge we have no plan around version. I acknowledge that git hashes are more meaningful than versions, and hard to read. Nevertheless I suggest we increment the version. Increase compiler version number to d5.11.0. by jaykrell ? Pull Request #89 ? modula3/cm3 (github.com) And I suggest it might be worth some discussion on how version will be revised going forward. Maybe something date based. Which is good and bad. It erases some discussion. It pretends there is linearity. Or whenever there is a "bootstrap bump", as recently occured. But let's not spend much time on it, please. cm3 --version should have git hash, if it doesn't already, and remote/url/branch, if it doesn't already -- these start to be meaningful. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemming at henning-thielemann.de Tue Mar 2 00:13:07 2021 From: lemming at henning-thielemann.de (Henning Thielemann) Date: Tue, 2 Mar 2021 00:13:07 +0100 (CET) Subject: [M3devel] increment version? In-Reply-To: References: Message-ID: <1a7397db-4ec6-1f12-3fd1-993a62369c40@henning-thielemann.de> On Mon, 1 Mar 2021, Jay K wrote: > I acknowledge that versions are meaningless. > I acknowledge that many people subscribe to "semantic versioning". > I assert that "semantic versioning" misses the point, that all changes can be described as "breaking", more or less. In Modula-3 the addition of a function or a type to a module cannot break anything. From jayk123 at hotmail.com Tue Mar 2 00:20:40 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 1 Mar 2021 23:20:40 +0000 Subject: [M3devel] increment version? In-Reply-To: <1a7397db-4ec6-1f12-3fd1-993a62369c40@henning-thielemann.de> References: , <1a7397db-4ec6-1f12-3fd1-993a62369c40@henning-thielemann.de> Message-ID: Sure it can. Expand what you consider semantics. Consider systems with fixed sized buffers. Growing anything risks exceeding buffers. I have heard and experienced multiple such. Adding an export to a .dll broke something that enumerated exports and had a limit. M3quake had a string length limit that I hit. It got later removed. Consider poorly implemented real-time systems. Slowing anything down at all risks missing deadlines. Nevertheless we must draw some subjective lines and accept change. Behavior and performance are part of interface. Not just the names of modules and compile-time signatures therein. - Jay ________________________________ From: Henning Thielemann Sent: Monday, March 1, 2021 3:13 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] increment version? On Mon, 1 Mar 2021, Jay K wrote: > I acknowledge that versions are meaningless. > I acknowledge that many people subscribe to "semantic versioning". > I assert that "semantic versioning" misses the point, that all changes can be described as "breaking", more or less. In Modula-3 the addition of a function or a type to a module cannot break anything. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 2 08:44:12 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 2 Mar 2021 07:44:12 +0000 Subject: [M3devel] nested function ABI vs. build_dir Message-ID: The m3cc backend uses a "hidden" (from C) ABI feature, "static link", for calling nested functions. It is nice, but it is not accessible from C. The C backend does something else. As I recall, always passing an extra, last, parameter, which is not always used. So the two systems are not interoperable, when nested functions are involved. Therefore questions: - Should build_dir vary? It seems like yes, but maybe not worth the ongoing nuisance? - What does LLVM backend do? (Is it based on C backend? ? ) I think so. I looked a little. - Does LLVM offer the static link the way gcc does? Hard to search for..ok, gcc calls it static_chain and I think LLVM does support it. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 2 11:35:50 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 2 Mar 2021 10:35:50 +0000 Subject: [M3devel] Target.Structure_size_boundary Message-ID: Overall goal: Reduce target-specificity in the system, and Target.i3 variables. This is *kinda* unreasonable to me. Gcc has an option called "structure size boundary". You can say 8, 16, or 64 (bits). It mimiced in Target.i3 as Structure_size_boundary. In gcc it is only valid for non-Netbsd arm32. For all other systems, it is 8 (bits) and not changable. The size of a struct containing just a char is 1. The size of a struct containingly only chars, is not rounded up. Among arm32 targets, there are several ABIs. Depending on the arm32 ABI, the default is 8 or 32 (bits). $ cat 1.c typedef struct { char a; } A; int b = sizeof(A); $ arm-linux-gnueabihf-gcc -S -mabi=? arm-linux-gnueabihf-gcc: note: valid arguments to ?-mabi=? are: aapcs aapcs-linux apcs-gnu atpcs iwmmxt The default is 8: $ arm-linux-gnueabihf-gcc -S 1.c && grep word 1.s .word 1 $ arm-linux-gnueabi-gcc -S 1.c && grep word 1.s .word 1 $ /usr/bin/arm-none-eabi-gcc -S 1.c && grep word 1.s .word 1 But other ABIs not: $ arm-linux-gnueabihf-gcc -S -mabi=atpcs -mfloat-abi=soft 1.c && grep word 1.s .word 4 $ /usr/bin/arm-none-eabi-gcc -mabi=apcs-gnu -S 1.c && grep word 1.s .word 4 $ arm-linux-gnueabihf-gcc -S -mabi=aapcs -mfloat-abi=soft 1.c && grep word 1.s .word 1 Depending on which ABI is used, the option is deprecated: $ arm-linux-gnueabihf-gcc -S -mstructure-size-boundary=32 1.c && grep word 1.s cc1: warning: option ?-mstructure-size-boundary? is deprecated .word 4 Which isn't really a useful statement -- the default still varies. It is just that it should flow with ABI. cm3 does not really expose this notion of ABI. We have ARM_LINUX, ARMEL_LINUX, ARM_DARWIN and no other ARM32. I might add ARM32_NT. I expect Modula-3 is not really in use on ARM32. At most, it is used on Linux. ARM32_{NET,FREE,OPEN}BSD presumably would match Linux. Exposing this list: aapcs aapcs-linux apcs-gnu atpcs iwmmxt would seem not great, but it could be done. Or some sort of "pass through" to the backends. Interoperation with C structs for which this matters, I expect, is rare. That is, if your struct has any ints, it tends to not matter (unless you change packing). Comments suggest this might also impact an m68k system, which I doubt at this point Modula-3 will ever run on. My inclination is to make this const in Target.i3, but leave the rest of the code unchanged. So if someone wants to expose these fine grained ARM32 ABIs, they might get it right. We could also do all relevant computations with 8 and 32 and error if they vary. Keep in mind, it is not only interop with C to consider, but interop with other architectures writing the same structs to persistant store or over the network. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 2 19:39:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 2 Mar 2021 18:39:21 +0000 Subject: [M3devel] Compiler tests In-Reply-To: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> References: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> Message-ID: No. You are right. I should. I will try to soon. I don't remember if I ran them years ago when I wrote the C backend, but I have not been running them recently. That said, it does all work pretty well. The system does build itself completely, and debuggability exceeds that of the integrated backend and I think Darwin. ? - Jay ________________________________ From: Rodney M. Bates Sent: Friday, February 26, 2021 6:40 PM To: m3devel ; Jay Subject: Compiler tests Jay, you have mentioned a couple of times suspecting a bug in the C backend, as well as others. Are you periodically running the scripted compiler tests? They can help a lot. -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Mar 2 22:56:18 2021 From: lists at darko.org (Darko Volaric) Date: Tue, 2 Mar 2021 22:56:18 +0100 Subject: [M3devel] Compiler tests In-Reply-To: References: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> Message-ID: It definitely still has problems, I get *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/M3C.m3", line 2523 *** when compiling an interface at the declaration TYPE ExAtmX = { Nil, Sort, Inst, Bin, Str, Bool, Int, Real, Any, LocalLink, FeatLink, SortLink, RowLink, Stream, FeatAdr, Result }; with the last C file output being /*enum_define*/typedef UINT8 T2DE88987; /*declare_enum*/ #define T2DE88987_Nil ((UINT8)0) /*declare_enum_elt*/ #define T2DE88987_Sort ((UINT8)1) /*declare_enum_elt*/ #define T2DE88987_Inst ((UINT8)2) /*declare_enum_elt*/ #define T2DE88987_Bin ((UINT8)3) /*declare_enum_elt*/ #define T2DE88987_Str ((UINT8)4) /*declare_enum_elt*/ #define T2DE88987_Bool ((UINT8)5) /*declare_enum_elt*/ #define T2DE88987_Int ((UINT8)6) /*declare_enum_elt*/ #define T2DE88987_Real ((UINT8)7) /*declare_enum_elt*/ #define T2DE88987_Any ((UINT8)8) /*declare_enum_elt*/ #define T2DE88987_LocalLink ((UINT8)9) /*declare_enum_elt*/ #define T2DE88987_FeatLink ((UINT8)10) /*declare_enum_elt*/ #define T2DE88987_SortLink ((UINT8)11) /*declare_enum_elt*/ with Critical Mass Modula-3 version d5.10.0 last updated: 2016-10-05 compiled: 2021-02-26 09:11:48 configuration: /Users/darko/Apps/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Tue, Mar 2, 2021 at 7:39 PM Jay K wrote: > No. You are right. I should. I will try to soon. > I don't remember if I ran them years ago when I wrote the C backend, but I > have not been running them recently. > That said, it does all work pretty well. The system does build itself > completely, and debuggability exceeds that of the integrated backend and I > think Darwin. ? > > - Jay > > ------------------------------ > *From:* Rodney M. Bates > *Sent:* Friday, February 26, 2021 6:40 PM > *To:* m3devel ; Jay > *Subject:* Compiler tests > > Jay, you have mentioned a couple of times suspecting a bug in the > C backend, as well as others. Are you periodically running the > scripted compiler tests? They can help a lot. > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Mar 3 04:39:55 2021 From: jay.krell at cornell.edu (Jay) Date: Tue, 2 Mar 2021 19:39:55 -0800 Subject: [M3devel] Compiler tests In-Reply-To: References: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> Message-ID: Oh wow, thank you. Please add this to the tree if you can. I will look soon, and I will improve the diagnostics at least a little. Again, it does compile the entire tree. Something is ?unique? here. Maybe reusing an enum member?? - Jay On Tue, Mar 2, 2021 at 1:56 PM Darko Volaric wrote: > It definitely still has problems, I get > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file "../src/M3C.m3", line 2523 > > *** > > > when compiling an interface at the declaration > > > TYPE ExAtmX = { > Nil, > Sort, > Inst, > Bin, > Str, > Bool, > Int, > Real, > Any, > > LocalLink, > FeatLink, > SortLink, > RowLink, > > Stream, > FeatAdr, > > Result > }; > > > with the last C file output being > > /*enum_define*/typedef UINT8 T2DE88987; /*declare_enum*/ > #define T2DE88987_Nil ((UINT8)0) /*declare_enum_elt*/ > #define T2DE88987_Sort ((UINT8)1) /*declare_enum_elt*/ > #define T2DE88987_Inst ((UINT8)2) /*declare_enum_elt*/ > #define T2DE88987_Bin ((UINT8)3) /*declare_enum_elt*/ > #define T2DE88987_Str ((UINT8)4) /*declare_enum_elt*/ > #define T2DE88987_Bool ((UINT8)5) /*declare_enum_elt*/ > #define T2DE88987_Int ((UINT8)6) /*declare_enum_elt*/ > #define T2DE88987_Real ((UINT8)7) /*declare_enum_elt*/ > #define T2DE88987_Any ((UINT8)8) /*declare_enum_elt*/ > #define T2DE88987_LocalLink ((UINT8)9) /*declare_enum_elt*/ > #define T2DE88987_FeatLink ((UINT8)10) /*declare_enum_elt*/ > #define T2DE88987_SortLink ((UINT8)11) /*declare_enum_elt*/ > > > with > > > Critical Mass Modula-3 version d5.10.0 > > last updated: 2016-10-05 > > compiled: 2021-02-26 09:11:48 > > configuration: /Users/darko/Apps/cm3/bin/cm3.cfg > > host: AMD64_DARWIN > > target: AMD64_DARWIN > > > > > On Tue, Mar 2, 2021 at 7:39 PM Jay K wrote: > >> No. You are right. I should. I will try to soon. >> I don't remember if I ran them years ago when I wrote the C backend, but >> I have not been running them recently. >> That said, it does all work pretty well. The system does build itself >> completely, and debuggability exceeds that of the integrated backend and I >> think Darwin. ? >> >> - Jay >> >> ------------------------------ >> *From:* Rodney M. Bates >> *Sent:* Friday, February 26, 2021 6:40 PM >> *To:* m3devel ; Jay >> *Subject:* Compiler tests >> >> Jay, you have mentioned a couple of times suspecting a bug in the >> C backend, as well as others. Are you periodically running the >> scripted compiler tests? They can help a lot. >> >> >> -- >> Rodney Bates >> rodney.m.bates at acm.org >> > _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 3 07:55:08 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 3 Mar 2021 06:55:08 +0000 Subject: [M3devel] Compiler tests In-Reply-To: References: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> , Message-ID: Just pasting that enum into code, works. The line of the assert..cannot easily be matched up. I'll add more printing before some asserts. Maybe try with current? Or in debugger? Or provide a more complete source? Certainly it looks trivial. Thank you, - Jay ________________________________ From: Jay Sent: Wednesday, March 3, 2021 3:39 AM To: Darko Volaric Cc: Jay K ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Compiler tests Oh wow, thank you. Please add this to the tree if you can. I will look soon, and I will improve the diagnostics at least a little. Again, it does compile the entire tree. Something is ?unique? here. Maybe reusing an enum member?? - Jay On Tue, Mar 2, 2021 at 1:56 PM Darko Volaric > wrote: It definitely still has problems, I get *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/M3C.m3", line 2523 *** when compiling an interface at the declaration TYPE ExAtmX = { Nil, Sort, Inst, Bin, Str, Bool, Int, Real, Any, LocalLink, FeatLink, SortLink, RowLink, Stream, FeatAdr, Result }; with the last C file output being /*enum_define*/typedef UINT8 T2DE88987; /*declare_enum*/ #define T2DE88987_Nil ((UINT8)0) /*declare_enum_elt*/ #define T2DE88987_Sort ((UINT8)1) /*declare_enum_elt*/ #define T2DE88987_Inst ((UINT8)2) /*declare_enum_elt*/ #define T2DE88987_Bin ((UINT8)3) /*declare_enum_elt*/ #define T2DE88987_Str ((UINT8)4) /*declare_enum_elt*/ #define T2DE88987_Bool ((UINT8)5) /*declare_enum_elt*/ #define T2DE88987_Int ((UINT8)6) /*declare_enum_elt*/ #define T2DE88987_Real ((UINT8)7) /*declare_enum_elt*/ #define T2DE88987_Any ((UINT8)8) /*declare_enum_elt*/ #define T2DE88987_LocalLink ((UINT8)9) /*declare_enum_elt*/ #define T2DE88987_FeatLink ((UINT8)10) /*declare_enum_elt*/ #define T2DE88987_SortLink ((UINT8)11) /*declare_enum_elt*/ with Critical Mass Modula-3 version d5.10.0 last updated: 2016-10-05 compiled: 2021-02-26 09:11:48 configuration: /Users/darko/Apps/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Tue, Mar 2, 2021 at 7:39 PM Jay K > wrote: No. You are right. I should. I will try to soon. I don't remember if I ran them years ago when I wrote the C backend, but I have not been running them recently. That said, it does all work pretty well. The system does build itself completely, and debuggability exceeds that of the integrated backend and I think Darwin. ? - Jay ________________________________ From: Rodney M. Bates > Sent: Friday, February 26, 2021 6:40 PM To: m3devel >; Jay > Subject: Compiler tests Jay, you have mentioned a couple of times suspecting a bug in the C backend, as well as others. Are you periodically running the scripted compiler tests? They can help a lot. -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 3 09:20:01 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 3 Mar 2021 08:20:01 +0000 Subject: [M3devel] growable array? Message-ID: I'd like a growable array. I guess I will use Sequence(char). It would be nice if Sequence had a function .Linearize() or some other name, that if remlo has been ever called, resets the start to be at offset 0, via a memcpy or two. Or if the underlying array was accessible, i.e. for sequences for which remlo is never called. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 3 09:51:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 3 Mar 2021 08:51:31 +0000 Subject: [M3devel] messages for assertion failures? Message-ID: Currently when assertions fail, we get just file and line reported. This is not great. Granted, it is small. I would the expression printed, at least optionally. I'd really like to be able to report additional text besides the expression. It looks like additional/replacement text is implemented in cm3, but not documented: <* ASSERT expression WITH message *> or <* ASSERT expression , message *> or such, but that doesn't satisfy in general, because I want the expression shown, absent a message. I have it just about working..always on. We should consider a command line option to compiler, or a pragma on a source by source basis, and/or a way to remove the messages, or store them nearby but "with" the program (source is too large and too far away). For example we could hash the messages, to shorter messages and append to a nearby file "hash fullmessage" leaving you to "grep hash asserts.txt" or somthing. I would also like stacks to be reported for assertion failures. Also not just asserts but runtime errors: *** *** runtime error: *** An explicit or implicit NARROW() operation failed. *** file "..\src\misc\Scanner.m3", line 1082 *** *** execution of [, ] failed *** What was the expression/statement? (This is from me working on this change. ? ) Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 3 10:04:56 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 3 Mar 2021 09:04:56 +0000 Subject: [M3devel] m3front/scanner to mmap? Message-ID: Does anyone object if m3front/Scanner.m3 is switched to use mmap/MapViewOfFile? I understand they aren't as portable as buffered I/O but they are quite portable. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 3 10:07:55 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 3 Mar 2021 09:07:55 +0000 Subject: [M3devel] m3front/scanner globals? Message-ID: Is anyone bothered by m3front/Scanner's use of globals? Or is scanning not parallelizable? (I'm about to make it worse.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Mar 2 17:55:56 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 2 Mar 2021 10:55:56 -0600 Subject: [M3devel] nested function ABI vs. build_dir In-Reply-To: References: Message-ID: <2e8fce3a-4040-4bcc-d8d1-a0b74a6b11f0@lcwb.coop> In LLVM itself, functions are never nested. But M3CG_LLVM is adding an explicit extra parameter as the static link. It also gives it an LLVM attribute called "nest", which, according to comments in M3CG_LLVM, makes for ABI compatibility between llvm- and gcc-generated code. This probably only means ABI compatibility with respect to static links and only how they passed at the point of a call. I think the only thing LLVM does with this attribute is pass this parameter in a specific but target-dependent way (e.g., a dedicated register), and keeps this the same as gcc/m3cc does it. M3CG_LLVM is making it the first parameter, but that might be a bit confusing in light of the sometimes RtoL, sometimes LtoR variation in calling conventions. M3CG_LLVM constructs static link values at call sites and uses them in nonlocal references in its own way, which is quite possibly not the same as m3cc. Gcc has long supported nested functions as a C language extension. I know it groups all locals that are referred-to nonlocally (i.e., from within a deeper-nested function) together in the activation record, and uses this to simplify inlining. I am quite sure M3CG_LLVM & LLVM are not doing this, but maybe it doesn't matter, as long as displacements off static links are used. I have a vague recollection that there are other reasons why mixtures of code generated by different back ends would not work, but no specifics. On 3/2/21 1:44 AM, Jay K wrote: > The m3cc backend uses a "hidden" (from C) ABI feature, "static link", for calling nested functions. > It is nice, but it is not accessible from C. > > The C backend does something else. > As I recall, always passing an extra, last, parameter, which is not always used. > > So the two systems are not interoperable, when nested functions are involved. > > Therefore questions: > ?- Should build_dir vary? It seems like yes, but maybe not worth the ongoing nuisance? > ?- What does LLVM backend do? (Is it based on C backend? ? ) I think so. I looked a little. > ?- Does LLVM offer the static link the way gcc does? Hard to search for..ok, gcc calls it static_chain and I think LLVM does support it. > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Mon Mar 1 22:03:44 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 1 Mar 2021 15:03:44 -0600 Subject: [M3devel] output directory names In-Reply-To: References: Message-ID: I have long wished for a separate directory for target-independent, mechanically generated files. Here, I include generated source code, to be compiled. quake-generated instantiations are examples, but I also have a number lots of my own source code generators too. In Schutz, I have subdirectory 'derived', beside 'src', when needed. This is all done with m3makefiles and build scripts, but it would be nice to have something like it integrated into the M3 build system. On 3/1/21 1:09 AM, Jay K wrote: > Presently we do things like: > ? - Produce not-exactly target-specific output, in target-specifid named directories. > ?- This is both the .c files and the instantiated generics. > > Should we consider a structure like: > > ? ?src\foo.m3 > ? ?out\i386_linux\foo.o > ? ?out\unix32le\foo.c > ? ?out\ArrayInteger.m3 # generic instantation > The point, esp. of the last, is two part: > > ?- You could reuse identical files across targets; hopefully they really are identical. > ? ?You could produce them in i386_linux and then compare with existing, including > ? ?putting an input hash in the output, and if the input hash matches and the output does ont, eror > > ?- The target-specific paths I mentioned earlier in generic instantiations would go away. > > It is risky. I've seen other systems try to put non-target-specific outputs in non-target-specific directories. > The risks include: > ?- inability to build them in parallel > ?- no verification that the files are in fact the same. > > My real goal is to make sure "unix64le.tar.gz" is the same across all targets, so the build/redistribution matrix > is cut down. > > Perhaps another way to do this, is actually to introduce such targets? > The then, if you actually compile and link the C, what directory to put it in? > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Mar 3 16:15:21 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 3 Mar 2021 09:15:21 -0600 Subject: [M3devel] growable array? In-Reply-To: References: Message-ID: Look at m3-libs/vararray/src/VarArray.[im]g. Quite thorough, can be used so as to self-expand as side effect of accesses. A little heavy-weight, but more convenient to use. Generic on element type. I got tired of rewriting the same ol' pattern, so did this a couple of years ago. On 3/3/21 2:20 AM, Jay K wrote: > I'd like a growable array. > I guess I will use Sequence(char). > It would be nice if Sequence had a function .Linearize() or some other name, > that if remlo has been ever called, resets the start to be at offset 0, via a memcpy or two. > > Or if the underlying array was accessible, i.e. for sequences for which remlo is never called. > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Mar 3 18:08:47 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 3 Mar 2021 11:08:47 -0600 Subject: [M3devel] m3front/scanner to mmap? In-Reply-To: References: Message-ID: <67e6bb92-cf93-b0e0-d9af-2e365a42a74a@lcwb.coop> What/where is mmap/MapViewOfFile? On 3/3/21 3:04 AM, Jay K wrote: > Does anyone object if m3front/Scanner.m3 is switched to use mmap/MapViewOfFile? > I understand they aren't as portable as buffered I/O but they are quite portable. > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Mar 3 19:04:23 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 3 Mar 2021 12:04:23 -0600 Subject: [M3devel] growable array? In-Reply-To: References: Message-ID: P.S. Subscripts can be full range signed integer, both lower and upper bound. On 3/3/21 9:15 AM, Rodney M. Bates wrote: > Look at m3-libs/vararray/src/VarArray.[im]g.? Quite thorough, can be used > so as to self-expand as side effect of accesses.? A little heavy-weight, but > more convenient to use.? Generic on element type. > > I got tired of rewriting the same ol' pattern, so did this a couple of years ago. > > > On 3/3/21 2:20 AM, Jay K wrote: >> I'd like a growable array. >> I guess I will use Sequence(char). >> It would be nice if Sequence had a function .Linearize() or some other name, >> that if remlo has been ever called, resets the start to be at offset 0, via a memcpy or two. >> >> Or if the underlying array was accessible, i.e. for sequences for which remlo is never called. >> >> ??- Jay >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Thu Mar 4 02:46:28 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 4 Mar 2021 01:46:28 +0000 Subject: [M3devel] growable array? In-Reply-To: References: , Message-ID: Sorry but this seems a bit too complicated to me. Sequence is close to what I usually want, with the slight extra complexity around add/remove from either end. What I ?like? is usually std::vector. Sequence is ?like? std::deque. It is about what I think std::deque should be, except std::deque has surprisingly strong iterator validity guarantees (when the container is mutated) forcing it into a slightly more complicated implementation. Though the implementation I envisioned is simpler: start in the middle, expand if either end is hit, and not the ?circular? behavior. I did use Sequence. We can consider adding the others, perhaps. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 3, 2021 7:15:21 AM To: Jay K ; m3devel Subject: Re: [M3devel] growable array? Look at m3-libs/vararray/src/VarArray.[im]g. Quite thorough, can be used so as to self-expand as side effect of accesses. A little heavy-weight, but more convenient to use. Generic on element type. I got tired of rewriting the same ol' pattern, so did this a couple of years ago. On 3/3/21 2:20 AM, Jay K wrote: > I'd like a growable array. > I guess I will use Sequence(char). > It would be nice if Sequence had a function .Linearize() or some other name, > that if remlo has been ever called, resets the start to be at offset 0, via a memcpy or two. > > Or if the underlying array was accessible, i.e. for sequences for which remlo is never called. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C08712deb97c24eddf21c08d8de573184%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637503813357502374%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=es3qGXIZBBdIE3weulWBw%2FNp8o%2BgUUzSMNVIAKFegO0%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 4 07:21:34 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 4 Mar 2021 06:21:34 +0000 Subject: [M3devel] uin-common vs. uin-len Message-ID: Fyi: C:\s\cm3\m3-libs\m3core\src\unix>more m3makefile readonly _UnixPieces = { % AIX? "CYGWIN" : [ "uin-common" ], "HPUX" : [ "uin-common" ], "INTERIX" : [ "uin-common" ], % IRIX? "LINUX" : [ "uin-common" ], "MINGW" : [ "uin-common" ], "NT" : [ "uin-common" ], "SOLgnu" : [ "solaris-2-x", "uin-common" ], "SOLsun" : [ "solaris-2-x", "uin-common" ], "SOLARIS" : [ "uin-common" ], "VMS" : [ "uin-common" ], % both are available, depending on #if % both are available, depending on #if % We use a define for a more important purpose (socklen_t), % and it gives us this too. "ALPHA_OSF" : [ "uin-len" ], % BSD/Darwin have an 8 bit family and an 8 bit length field added "DARWIN" : [ "uin-len" ], "NETBSD" : [ "uin-len" ], "FREEBSD" : [ "uin-len" ], "OPENBSD" : [ "uin-len" ], uin-len is broken I'll see about fixing..I'd really like to remove these platform-dependent structs from the Modula-3 side.. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 4 08:02:09 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 4 Mar 2021 07:02:09 +0000 Subject: [M3devel] output directory names In-Reply-To: References: , Message-ID: These strings are usually: ../src/foo.m3 occasionally: ../src/abc/def/foo.m3 and the "bad" ones: ../I386_LINUX/foo.m3. I suggest: 1. Might as well trim to just foo.m3? 2. Or worst case, if generating C, remove "target" or "/target/" or "/targetc"/ or at least specifically *leading* forms "../target/" and "../targetc/". It is conceiveable Compiler.ThisFile is used to find nearby data, in the source tree?? So going too overboard here would break things?? I think not. Though, something more aggressive, splitting target-specific derived files from target-independent deriveds would be good. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, March 1, 2021 9:03 PM To: Jay K ; m3devel Subject: Re: [M3devel] output directory names I have long wished for a separate directory for target-independent, mechanically generated files. Here, I include generated source code, to be compiled. quake-generated instantiations are examples, but I also have a number lots of my own source code generators too. In Schutz, I have subdirectory 'derived', beside 'src', when needed. This is all done with m3makefiles and build scripts, but it would be nice to have something like it integrated into the M3 build system. On 3/1/21 1:09 AM, Jay K wrote: > Presently we do things like: > - Produce not-exactly target-specific output, in target-specifid named directories. > - This is both the .c files and the instantiated generics. > > Should we consider a structure like: > > src\foo.m3 > out\i386_linux\foo.o > out\unix32le\foo.c > out\ArrayInteger.m3 # generic instantation > The point, esp. of the last, is two part: > > - You could reuse identical files across targets; hopefully they really are identical. > You could produce them in i386_linux and then compare with existing, including > putting an input hash in the output, and if the input hash matches and the output does ont, eror > > - The target-specific paths I mentioned earlier in generic instantiations would go away. > > It is risky. I've seen other systems try to put non-target-specific outputs in non-target-specific directories. > The risks include: > - inability to build them in parallel > - no verification that the files are in fact the same. > > My real goal is to make sure "unix64le.tar.gz" is the same across all targets, so the build/redistribution matrix > is cut down. > > Perhaps another way to do this, is actually to introduce such targets? > The then, if you actually compile and link the C, what directory to put it in? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C91c4aecba778450af0bd08d8dcf59e4b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637502294774642065%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Zb76v2IEn87cj5EbLON6QDQPJkT4BGV8yFJqiru4coU%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 4 08:04:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 4 Mar 2021 07:04:10 +0000 Subject: [M3devel] m3front/scanner to mmap? In-Reply-To: <67e6bb92-cf93-b0e0-d9af-2e365a42a74a@lcwb.coop> References: , <67e6bb92-cf93-b0e0-d9af-2e365a42a74a@lcwb.coop> Message-ID: mmap is Posix. CreateFile + CreateFileMapping + MapViewOfFile is Win32. Granted there is nothing probably wrapping these in libm3/m3core, and they present some safety concerns -- how to check bounds of access, and what to do if I/O fails. But they are also generally very efficient. The kernel buffers end up shared into user space and copying is reduced. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 3, 2021 5:08 PM To: Jay K ; m3devel Subject: Re: [M3devel] m3front/scanner to mmap? What/where is mmap/MapViewOfFile? On 3/3/21 3:04 AM, Jay K wrote: > Does anyone object if m3front/Scanner.m3 is switched to use mmap/MapViewOfFile? > I understand they aren't as portable as buffered I/O but they are quite portable. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cf10c675a6d7c4a839e7108d8de6709f4%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637503881414052190%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=aLDtsIgspqmUSXkYV4HmKJCFpEi4sbB9EWQxmKuUIgg%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 4 09:38:43 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 4 Mar 2021 08:38:43 +0000 Subject: [M3devel] unetdb Message-ID: In the case of not returning the first item in the list, this does the wrong free. err := Unetdb.getaddrinfo(node,port,NIL,res); IF err # 0 THEN IPError.Raise (LookupFailure, err); ELSE WHILE res # NIL DO IF res.ai_family = Usocket.AF_INET THEN sa4 := LOOPHOLE(res.ai_addr,RefSockaddr4); a4 := LOOPHOLE(sa4.sin_addr,Address4); ep4 := NEW(Endpoint4); ep4.adr := a4; ep4.port := sa4.sin_port; Unetdb.freeaddrinfo(res); RETURN ep4; ELSIF res.ai_family = Usocket.AF_INET6 THEN sa16 := LOOPHOLE(res.ai_addr, RefSockaddr16); a16 := LOOPHOLE(sa16.sin6_addr, Address16); ep16 := NEW(Endpoint16); ep16.adr := a16; ep16.port := sa16.sin6_port; Unetdb.freeaddrinfo(res); RETURN ep16; END; res := res.ai_next; END; END; -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 03:34:17 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 02:34:17 +0000 Subject: [M3devel] OSF/Ultrix support, TCP hacks? Message-ID: We have some small hacks for OSF/1 and Ultrix. To get errors in networking code. There are at least two copies of it. Maybe time to remove them? Anyone really running these systems and can confirm they are working and/or desired? OSF/1 probably does work but I doubt anyone is using these systems, really. I would keep the m3core.h OSF/1 support and config files, but I'd remove the TCP hacks. I ran Modula-3 on OSF/1 briefly around 2010 at this list's request to make it work (see m3core.h). I ran Ultrix but not Modula-3 on it over 20 years ago. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 03:36:03 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 02:36:03 +0000 Subject: [M3devel] C99 dependency? Message-ID: Is anyone for or against a C99 dependency? - declaring locals "late"; maybe no big deal? - fenv stuff; we have this optionally in the system already, but maybe time to make it unconditional. Maybe a bigger deal? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 7 06:53:42 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 07 Mar 2021 08:53:42 +0300 Subject: [M3devel] C99 dependency? In-Reply-To: References: Message-ID: <271631615096268@mail.yandex.by> An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 7 07:14:28 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 07 Mar 2021 09:14:28 +0300 Subject: [M3devel] XDS Modula-2/Oberon-2 DEC Alpha mBURG / lBURG / iBURGRe: OSF/Ultrix support, TCP hacks? In-Reply-To: References: Message-ID: <259641615096477@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 07:14:30 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 06:14:30 +0000 Subject: [M3devel] C99 dependency? In-Reply-To: <271631615096268@mail.yandex.by> References: , <271631615096268@mail.yandex.by> Message-ID: It does have a bunch of C99 support actually. Win32 can compile as C++ also. I am mainly interested in fenv, which can probably be achieved in nonstandard ways also. - Jay ________________________________ From: vvm at tut.by Sent: Saturday, March 6, 2021 9:53:42 PM To: Jay K ; m3devel Subject: Re: [M3devel] C99 dependency? Hi! As I known, cl.exe from MS Visual Studio doesn't support C99 ( only relative modern C++ standarts) Best regards, Victor Miasnikov 07.03.2021, 05:36, "Jay K" : Is anyone for or against a C99 dependency? - declaring locals "late"; maybe no big deal? - fenv stuff; we have this optionally in the system already, but maybe time to make it unconditional. Maybe a bigger deal? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 7 07:20:32 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 07 Mar 2021 09:20:32 +0300 Subject: [M3devel] C99 dependency? In-Reply-To: References: , <271631615096268@mail.yandex.by> Message-ID: <243741615097716@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 08:19:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 07:19:10 +0000 Subject: [M3devel] C99 dependency? In-Reply-To: <243741615097716@mail.yandex.by> References: , <271631615096268@mail.yandex.by> , <243741615097716@mail.yandex.by> Message-ID: Yes, I386_NT / NT386 should work with the C backend. While I am more keen on dropping the gcc backend, the integrated backend perhaps should not be used any longer either. Well, I don't know. The C backend is a way of declaring the system very finished and working and portable, and can leave it alone, it'll just keep working. The integrated backend then becomes a highly optional fun project that will never be finished, either because of porting to all the systems (CPU and object file format), or because of maybe optimizing more, though its goal is to be run fast, not optimize much. As it stands though, only supporting NT/x86 while useful is also limiting. - Jay ________________________________ From: vvm at tut.by Sent: Sunday, March 7, 2021 6:20 AM To: Jay K ; m3devel Subject: Re: [M3devel] C99 dependency? Hi! If we can use cl.exe for both Modula-3 for Win 32bit ( as etalon) and AMD64_NT -- why not? Main advantage is using .pdb based debugging in x64dbg and(or) IDA Pro. Best regards, Victor Miasnikov 07.03.2021, 09:14, "Jay K" : It does have a bunch of C99 support actually. Win32 can compile as C++ also. I am mainly interested in fenv, which can probably be achieved in nonstandard ways also. - Jay ________________________________ From: vvm at tut.by > Sent: Saturday, March 6, 2021 9:53:42 PM To: Jay K >; m3devel > Subject: Re: [M3devel] C99 dependency? Hi! As I known, cl.exe from MS Visual Studio doesn't support C99 ( only relative modern C++ standarts) Best regards, Victor Miasnikov 07.03.2021, 05:36, "Jay K" >: Is anyone for or against a C99 dependency? - declaring locals "late"; maybe no big deal? - fenv stuff; we have this optionally in the system already, but maybe time to make it unconditional. Maybe a bigger deal? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 08:58:22 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 07:58:22 +0000 Subject: [M3devel] XDS Modula-2/Oberon-2 DEC Alpha mBURG / lBURG / iBURGRe: OSF/Ultrix support, TCP hacks? In-Reply-To: References: <259641615096477@mail.yandex.by>, <267431615098072@mail.yandex.by>, Message-ID: oops, adding back m3devel ________________________________ From: Jay K Sent: Sunday, March 7, 2021 7:16 AM To: vvm at tut.by Subject: Re: [M3devel] XDS Modula-2/Oberon-2 DEC Alpha mBURG / lBURG / iBURGRe: OSF/Ultrix support, TCP hacks? OSF/1 is much "better" than Ultrix. In particular, OSF/1 has I believe dynamic linking and pthreads. Ultrix lacks dynamic linking and I'm not sure about pthreads. Unless you need a 32bit system for some reasons. Sure I might try an emulator someday. Keep in mind, I didn't say drop Alpha (or MIPS) support. There are much newer systems that can run on those, Linux, OpenBSD, NetBSD, NT4. Heck, I have Alpha and m68k running now in a Debian chroot in Ubuntu in WSL2. Seriously. M68k_LINUX here we come shortly. I always kinda felt we were missing a modern M68K option (i.e. not counting old Sun). :) The specific code that bugs me is: https://github.com/modula3/cm3/blob/master/m3-comm/tcp/src/POSIX/TCPHack.m3 and a copy of almost identical here: https://github.com/modula3/cm3/commit/0e9817d2d5dd6a42741819982578eebf76a72974 and I wouldn't be surprised if this applies to VMS also, which is actually a system under active development (AMD64_VMS when possible!) Keep in mind, the system is heavily dependent on Win32 and Posix, and then C or gcc. There isn't much platform specific anywhere now, except in the gcc backend. And the integrated backend. And maybe m3gdb. Ideally the system is only as platform specific as hello, world. Seriously. The endian and word size and Win32 vs. Posix dependencies might never go away though. But maybe we can reduce it to just that. These little OSF1/Ultrix hacks are a minor nuisance en route to that. (Can we drop user threads? I don't know. I wonder if they will be useful on WebAsssembly.) - Jay ________________________________ From: vvm at tut.by Sent: Sunday, March 7, 2021 6:21 AM To: Jay K Subject: Re: [M3devel] XDS Modula-2/Oberon-2 DEC Alpha mBURG / lBURG / iBURGRe: OSF/Ultrix support, TCP hacks? - ??? In free version of one... 07.03.2021, 09:14, "vvm at tut.by" : Hi! At first: there are exist 3 or 4 excelent working emulators of DEC Alpha. They can run OS Ultrix. ( Personally I have installed Ultrix in free version in one of these emulator). As I known Ultrix is more modern version of OSF/1. I.e. it's simple marketing renaming. Why we need support DEC Alpha in Modula-3: CPU DEC Alpha is only 64bit CPU for this we have BURG grammar. We have lcc C langage compiler based BURG. We have in pm3 some support for lcc. We have mBURG writed on Modula-2 ( and worked m2tom3). We have public source code XDS Modula-2/Oberon-2 use iBURG grammar. I hope that creating "XDS Modula-3 for DEC Alpha" is relative practical way to create 64bit optimising compiler based on Modula-2 or Modula-3. Best regards, Victor Miasnikov 07.03.2021, 05:34, "Jay K" >: We have some small hacks for OSF/1 and Ultrix. To get errors in networking code. There are at least two copies of it. Maybe time to remove them? Anyone really running these systems and can confirm they are working and/or desired? OSF/1 probably does work but I doubt anyone is using these systems, really. I would keep the m3core.h OSF/1 support and config files, but I'd remove the TCP hacks. I ran Modula-3 on OSF/1 briefly around 2010 at this list's request to make it work (see m3core.h). I ran Ultrix but not Modula-3 on it over 20 years ago. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 7 09:09:56 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 07 Mar 2021 11:09:56 +0300 Subject: [M3devel] XDS Modula-2/Oberon-2 DEC Alpha mBURG / lBURG / iBURGRe: OSF/Ultrix support, TCP hacks? In-Reply-To: References: <259641615096477@mail.yandex.by>, <267431615098072@mail.yandex.by>, Message-ID: <258831615104565@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 09:54:44 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 08:54:44 +0000 Subject: [M3devel] socket SO_LINGER Win32 vs. Posix.. Message-ID: We have opposite looking code on Posix and Win32: C:\s\cm3\m3-comm\tcp\src\POSIX\TCP.m3 PROCEDURE InitFD(fd: CARDINAL) = (* We assume that the runtime ignores SIGPIPE signals *) VAR one: int := 1; linger := Usocket.struct_linger{1, 1}; <== Specifically this. BEGIN EVAL Usocket.setsockopt(fd, Usocket.SOL_SOCKET, Usocket.SO_LINGER, ADR(linger), BYTESIZE(linger)); EVAL Usocket.setsockopt( fd, Uin.IPPROTO_TCP, TCP_NODELAY, ADR(one), BYTESIZE(one)); MakeNonBlocking (fd); END InitFD; C:\s\cm3\m3-comm\tcp\src\WIN32\TCP.m3 PROCEDURE InitSock(sock: WinSock.SOCKET) = (* We assume that the runtime ignores SIGPIPE signals *) VAR one : BOOL := 1; linger := WinSock.struct_linger{0, 0}; <== Specifically this. BEGIN EVAL WinSock.setsockopt( sock, WinSock.SOL_SOCKET, WinSock.SO_LINGER, ADR(linger), BYTESIZE(linger)); EVAL WinSock.setsockopt( sock, WinSock.IPPROTO_TCP, WinSock.TCP_NODELAY, ADR(one), BYTESIZE(one)); IF WinSock.ioctlsocket(sock, WinSock.FIONBIO, ADR(one)) = SockErr THEN IPError.Die(); END; END InitSock; Ignoring the FIONBIO / MakeNonBlocking part, I think the right thing here is to delete both SO_LINGER settings, and probably TCP_NODELAY. The principles would be: - Accept defaults. - Let higher levels decide. - And do not have opposite behavior on Windows and Unix. Agreed? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 15:59:23 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 14:59:23 +0000 Subject: [M3devel] WinSock.FIONBIO as -2147195266 Re: Modula-3 subranges vs. opaque arbitrary 32bit constants? In-Reply-To: <40311614081470@mail.yandex.by> References: , <1523101613997774@mail.yandex.by> <1135121614015614@mail.yandex.by>,<40311614081470@mail.yandex.by> Message-ID: Win64: Fix socket ioctl codes. ? modula3/cm3 at 24873d4 (github.com) ________________________________ From: vvm at tut.by Sent: Tuesday, February 23, 2021 12:02 PM To: Jay K Subject: WinSock.FIONBIO as -2147195266 Re: [M3devel] Modula-3 subranges vs. opaque arbitrary 32bit constants? Hi! Some information dated 2019-11-XX: "TCP.m3.diff m3-comm--tcp--src--WIN32 {--- Revers {= WinSock.i3 is better place.txt" == - IF WinSock.ioctlsocket(sock, WinSock.FIONBIO, ADR(one)) = SockErr THEN + IF WinSock.ioctlsocket(sock, -2147195266, ADR(one)) = SockErr THEN == 2147772030 1000110011001111110 -2147195266 {= ... ioctlsocket's cmd parameter is a long, so the actual value of FIONBIO should be printed with %ld and is -2147195266 ? tmm1 Mar 25 '18 at 7:46 ... ... FASYNC : constant := FIOASYNC; FIOASYNC : constant := 16#8004667d#; FIONBIO : constant := -2147195266; FIONREAD : constant := 16#4004667f#; ... ... /// /// Use to determine the amount of data pending in the network's input buffer that can be read from socket s. /// FIONREAD = 1074030207, /// /// The *argp parameter is a pointer to an unsigned long value. /// Set *argp to a nonzero value if the nonblocking mode should be enabled, /// or zero if the nonblocking mode should be disabled. /// FIONBIO = -2147195266, FIOASYNC = 2147195267, SIOCSHIWAT = 2147192064, SIOCGHIWAT = 1074033409, SIOCSLOWAT = 2147192062, SIOCGLOWAT = 1074033411, /// /// Use to determine if all out of band (OOB) data has been read. /// SIOCATMARK = 1074033415 ... Best regards, Victor Miasnikov P.S. More info in attached files 22.02.2021, 20:46, "vvm at tut.by" : Hi! At 21 and 22 February I read 6 or 12 messages in M3 mail list about problem with TCP.m3 I think that this: -FIONBIO = Or (Or (W, INT), Or (FC, 126)); (* Set/clear non-bl.i/o *) + FIONBIO = -2147195266; should solve it. Please, try: today I don't have MS VS on my computer. Best regards, Victor Miasnikov 22.02.2021, 20:37, "Jay K" >: Well. I did not apply your patches. I encountered the errors, and then I fixed them. In some cases possibly the same as you. Please look at the history in master. It isn't much to read. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, February 22, 2021 12:48 PM To: Jay K >; m3devel > Subject: Re: [M3devel] Modula-3 subranges vs. opaque arbitrary 32bit constants? Hi! > m3tcp!TCP__InitSock+0xb8 [C:\s\cm3\m3-comm\tcp\src\WIN32\TCP.m3 @ 310]: Error's happened even with this patch? https://github.com/modula3/cm3/issues/47#issuecomment-501227088 ( Related patch: --- WinSock.i3 000 Sun Jun 02 21:24:39 2019 +++ WinSock.i3 Wed Jun 12 11:11:53 2019 @@ -80,7 +80,7 @@ (* file i/o controls *) FIONREAD = Or (Or (R, INT), Or (FC, 127)); (* Get # bytes to read *) - FIONBIO = Or (Or (W, INT), Or (FC, 126)); (* Set/clear non-bl.i/o *) + FIONBIO = -2147195266; (* Set/clear non-bl.i/o *) (* Ok only to 32 bit OS FIONBIO = Or (Or (W, INT), Or (FC, 126)); *) FIOASYNC = Or (Or (W, INT), Or (FC, 125)); (* Set/clear async i/o *) (* Socket i/o controls *) ) Best regards, Victor Miasnikov 22.02.2021, 01:09, "Jay K" >: Maybe it is a bug in the Modula-3 Windows headers cloning, or in the C backend. I'm not sure. Please help me to understand? The AMD64_NT port works well enough, again, that it can build the runtime and compiler, and the compiler can build everything. But networking fails, here: 0:000> k # Child-SP RetAddr Call Site 00 (Inline Function) --------`-------- KERNELBASE!DebugBreak [minkernel\kernelbase\debug.c @ 143] 01 00000053`b8cff188 00007fff`b8a35e93 KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] 02 00000053`b8cff190 00007fff`b8a2682f m3core!RTOS__Crash+0x43 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 26] 03 00000053`b8cff1e0 00007fff`b8a23bef m3core!RTProcess__Crash+0x6f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66] 04 00000053`b8cff220 00007fff`b8a23820 m3core!RTError__EndError+0x3f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119] 05 00000053`b8cff260 00007fff`b8a24257 m3core!RTError__MsgS+0xa0 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41] 06 00000053`b8cff2a0 00007fff`b8a23e5e m3core!RTException__Crash+0x317 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81] 07 00000053`b8cff330 00007fff`b8a23d4b m3core!RTException__DefaultBackstop+0xce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39] 08 00000053`b8cff3a0 00007fff`b8a37d7a m3core!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 09 00000053`b8cff3e0 00007fff`b8a23f2a m3core!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0a 00000053`b8cff440 00007fff`b8a23d4b m3core!RTException__DefaultBackstop+0x19a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49] 0b 00000053`b8cff4b0 00007fff`b8a37d7a m3core!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 0c 00000053`b8cff4f0 00007fff`b8a06cce m3core!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0d 00000053`b8cff550 00007ff8`2d57626a m3core!RTHooks__ReportFault+0xfe [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 112] 0e 00000053`b8cff5f0 00007ff8`2d5747f2 m3tcp!M_TCP_L_100_CRASH+0x1a [C:\s\cm3\m3-comm\tcp\AMD64_NT\TCP.m3.c @ 2759] 0f 00000053`b8cff620 00007ff8`2d57371c m3tcp!TCP__InitSock+0xc2 [C:\s\cm3\m3-comm\tcp\src\WIN32\TCP.m3 @ 310] 10 00000053`b8cff680 00007fff`fe8302b3 m3tcp!TCP__NewConnector+0xdc [C:\s\cm3\m3-comm\tcp\src\WIN32\TCP.m3 @ 72] 11 00000053`b8cff720 00007fff`fe82d814 sharedobj!LocalObjectSpace__SpaceInit+0x283 [C:\s\cm3\m3-comm\sharedobj\src\LocalObjectSpace.m3 @ 391] 12 00000053`b8cff930 00007fff`b8a21e17 sharedobj!LocalObjectSpace_M3+0x94 [C:\s\cm3\m3-comm\sharedobj\src\LocalObjectSpace.m3 @ 805] 13 00000053`b8cff990 00007fff`b8a21c9a m3core!RTLinker__RunMainBody+0x3e7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 408] 14 00000053`b8cffa30 00007fff`b8a21c9a m3core!RTLinker__RunMainBody+0x26a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 389] 15 00000053`b8cffad0 00007fff`b8a21c9a m3core!RTLinker__RunMainBody+0x26a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 389] 16 00000053`b8cffb70 00007fff`b8a21c9a m3core!RTLinker__RunMainBody+0x26a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 389] 17 00000053`b8cffc10 00007fff`b8a2209b m3core!RTLinker__RunMainBody+0x26a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 389] 18 00000053`b8cffcb0 00007fff`b8a21818 m3core!RTLinker__AddUnitI+0x11b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 116] 19 00000053`b8cffcf0 00007ff6`f5b31038 m3core!RTLinker__AddUnit+0x118 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 125] 1a 00000053`b8cffd50 00007ff6`f5db4d38 mentor!main+0x38 [C:\s\cm3\m3-demo\mentor\AMD64_NT\_m3main.c @ 23] 1b (Inline Function) --------`-------- mentor!invoke_main+0x22 [d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 1c 00000053`b8cffd80 00007ff8`49a67034 mentor!__scrt_common_main_seh+0x10c [d:\agent\_work\5\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 1d 00000053`b8cffdc0 00007ff8`4a8fd241 KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] 1e 00000053`b8cffdf0 00000000`00000000 ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] *** *** runtime error: *** An enumeration or subrange value was out of range. *** file "..\src\WIN32\TCP.m3", line 310 *** it is "simple" subrange problem. IF WinSock.ioctlsocket(sock, WinSock.FIONBIO, ADR(one)) = SockErr THEN IPError.Die(); END; It is *not* calling that IPError.Die(). ioctlsocket is declared as: PROCEDURE ioctlsocket (s: SOCKET; cmd: INT32; argp: UNTRACED REF UINT32): INT32; INT32 = Cstdint.int32_t; int32_t = Cstdint.int32_t; int32_t = Ctypes.int; int = [-16_7fffffff-1 .. 16_7fffffff]; It is range checking ioctlsocket parameter WinSock.FIONBIO against that and rejecting it. The code is: 307 00007ff8`2d5747ca e847620000 call m3tcp!setsockopt (00007ff8`2d57aa16) previous line 307 00007ff8`2d5747cf 89442430 mov dword ptr [rsp+30h],eax 310 00007ff8`2d5747d3 b87e660880 mov eax,8008667Eh WinSock.FIONBIO 310 00007ff8`2d5747d8 4889442440 mov qword ptr [rsp+40h],rax 310 00007ff8`2d5747dd 48817c2440ffffff7f cmp qword ptr [rsp+40h],7FFFFFFFh 310 00007ff8`2d5747e6 7e0a jle m3tcp!TCP__InitSock+0xc2 (00007ff8`2d5747f2) Branch m3tcp!TCP__InitSock+0xb8 [C:\s\cm3\m3-comm\tcp\src\WIN32\TCP.m3 @ 310]: 310 00007ff8`2d5747e8 b9c1260000 mov ecx,26C1h 310 00007ff8`2d5747ed e85e1a0000 call m3tcp!M_TCP_L_100_CRASH (00007ff8`2d576250) dv /V 00000053`b8cff660 @rsp+0x0040 L_345_L_346 = 0n2148034174 mov eax, constant is zero extending and to confirm: C:\s>type fio.c #include #include int main() { printf("%d %x\n", FIONBIO, FIONBIO); return 0; } C:\s>cl fio.c Microsoft (R) C/C++ Optimizing Compiler Version 19.24.28316 for x64 Copyright (C) Microsoft Corporation. All rights reserved. fio.c Microsoft (R) Incremental Linker Version 14.24.28316.0 Copyright (C) Microsoft Corporation. All rights reserved. /out:fio.exe fio.obj C:\s>fio.exe -2147195266 8004667e What is the right way to do this? Just change to UINT32? Even it works, can you explain it to me? I agree integers should not silently overflow, subranges are good, but this doesn't seem to me like it should be so difficult. I am inclined to further drain the Modula-3 "headers" and make it an "extern constant variable", really, though I'm not sure that will help, though it serves another purpose -- I like to minimally duplicate C headers. That is one way Posix ports became much easier. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 7 16:17:33 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 07 Mar 2021 18:17:33 +0300 Subject: [M3devel] WinSock.FIONBIO as -2147195266 Re: Modula-3 subranges vs. opaque arbitrary 32bit constants? In-Reply-To: References: , <1523101613997774@mail.yandex.by> <1135121614015614@mail.yandex.by>, <40311614081470@mail.yandex.by> Message-ID: <275011615129953@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 7 23:24:53 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 7 Mar 2021 22:24:53 +0000 Subject: [M3devel] writable strings? Message-ID: I believe the way TEXT works in Modula-3 is actually insufficient. In particular, between readonly strings and mutable strings, by some name. You need both. In C# they have readonly strings and writable stringbuffers. C++ just has writable writable std::string. The advantage of elevating some type to "StringBuffer" is significant optimizations, i.e. around buffer doubling. I guess SeqChar comes close. Except the addlo/remolo functions are usually overkill, and worse, being an object, gives it an extra level of indirection. That is too bad, that Modula3 "objects" and "methods" require the outer thing be traced. :( - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 09:16:35 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 08:16:35 +0000 Subject: [M3devel] conditional locking in IPv4 paths? Message-ID: We have approximately: (* TODO Not all systems need this lock, e.g. Linux *) VAR mu := NEW(MUTEX); PROCEDURE GetHostByName(nm: TEXT; VAR (*out*) res: Address): BOOLEAN RAISES {Error} = VAR s := M3toC.SharedTtoS(nm); err: int := 0; h : ADDRESS := NIL; (* hostent, null or not *) BEGIN LOCK mu DO err := IPInternal.GetHostByName(s, res, h); END; M3toC.FreeSharedS(nm, s); IF h = NIL THEN IPInternal.InterpretError(err); RETURN FALSE; END; RETURN TRUE; END GetHostByName; question then, what idiom to conditionally use the lock? - Derive ConditionalMutex from MutexRep var mu := NEW(ConditionalMutex)... It makes sense, but I'm not sure MUTEX is to ingrained in the language. i.e. what will LOCK work with? - Wrap up MUTEX or other and use TRY/FINALLY instead of LOCK and have try/finally either have an if, or call a function that has an if? - Could use mu null-ness for the if. - Could have LOCK accept null but probably not. - Assume contention is low and just use the lock. The lock *really* is not always needed. It is not needed on the better systems, like NT Linux Solaris AIX probably-Darwin but is "needed" on the usual suspects FreeBSD, OpenBSD, NetBSD, but does not actually suffice for them. - Give up on the IPv4 interfaces and layer them over IPv6 instead? - Give up on the IPv4 interfaces and change all the callers? 'cause you know the IPv6 interfaces fixed this mess on all sytems that implement them. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 09:21:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 08:21:31 +0000 Subject: [M3devel] require IPv6 interfaces? Message-ID: Proposal: Drop gethostbyname, gethostbyaddr, and migrate everyone in-tree to getaddrinfo, getnameinfo? The IPv6 interfaces do go back over 20 years. There was a preview for Windows 2000 and the work on Windows XP. They surely work on Darwin, Open/Free/NetBSD, AIX, Solaris, Linux, HP-UX, VMS. Perhaps this is the stack in the ground and drop all systems that lack them? Maybe. The system would still mostly work on other systems, just not networking. It'd cleanup the code a bit, and fix some race conditions even in current systems (which really should be fixed in those systems). - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 10:34:55 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 09:34:55 +0000 Subject: [M3devel] eliminating lazy/strict align Message-ID: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 10:37:56 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 09:37:56 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: Message-ID: How about this as a compromise: Make the change conditional on using the C backend? I can try that. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 8, 2021 9:34 AM To: m3devel Subject: [M3devel] eliminating lazy/strict align I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 8 11:44:50 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 08 Mar 2021 13:44:50 +0300 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: Message-ID: <318111615200054@mail.yandex.by> An HTML attachment was scrubbed... URL: From lists at darko.org Mon Mar 8 14:30:13 2021 From: lists at darko.org (Darko Volaric) Date: Mon, 8 Mar 2021 14:30:13 +0100 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: Message-ID: Those pragmas can be removed because they are not necessary. The only important thing is the Allow_packed_byte_aligned flag in Target.i3 which should be true for any arch for which it works such as x86 or PPC. That flag doesn't need to be turned off and on in source. The compiler will always pack structures in the same way regardless of that flag, the difference is that if you construct a packed structure (using BITS FOR) that forces a byte alignment then the compiler will allow it. On Mon, Mar 8, 2021 at 10:35 AM Jay K wrote: > I'm looking at target-specificness in m3front/m3middle again, and > lazyalign shows up. > > > I would really really really like targets to have far less difference. > I believe it is mostly counter productive. > I understand x86 can pack things more tightly and this can save memory > with certain structures. > And sharing data with C is made easier sometimes. > But I want the target matrix to be: (NT or Posix) and (32bit or > 64bit) and (endian) > and then 32bit and big endian can be considered unusual, so > people can just chose to download Windows or Unix archives, > and I don't think x86/amd64 merit a further expansion of this matrix. > > The system will be easier to setup, as long as it remains "self hosted". > (Rewriting m3front in C or C++ is another alternative.) > > This will break people's Pickles? > Or the Pickle code will adapt, just as if tranferring across architecture? > I think it will adapt. > > The lazyalign and strictalign pragmas would be ignored, or warn. > > The way they are implemented seems dubious, in that when you see lazyalign, > it alters Target, and I don't know if the ordering of that is good -- does > adding > a lazyalign type to a library change the code or layout of the entire > library? > > Thoughts please? > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Mon Mar 8 15:53:44 2021 From: dmuysers at hotmail.com (Dirk Muysers) Date: Mon, 08 Mar 2021 15:53:44 +0100 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: Message-ID: The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be:?(NT or Posix) and?(32bit or 64bit)?and?(endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". ? (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 8 18:33:52 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 08 Mar 2021 20:33:52 +0300 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: Message-ID: <323031615223310@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 18:48:29 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 17:48:29 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: <323031615223310@mail.yandex.by> References: , <323031615223310@mail.yandex.by> Message-ID: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers ; Jay K ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align Hi! I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) In addition, it's good idea convert all source code of OS from C language to Modula-3. May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > own linker It's looks like "Oberon way". > free C tools (e.g. Pelles C) Is it has free source code? It's based on lcc, that's good. > latest Visual Studio Why not? It's have native support of utf-8. > Windows 7 Pro As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" : The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K > wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Mon Mar 8 19:15:53 2021 From: dmuysers at hotmail.com (Dirk Muysers) Date: Mon, 08 Mar 2021 19:15:53 +0100 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <,323031615223310@mail.yandex.by> Message-ID: Welcome to the Microsoft universe (/hell). The very reason of Modula-3's existence was to forget C++ in favour of a better, safer, and simpler alternative. I don't want it to become yet another, hidden way to program in... C++! >> (if you don't like C++) turn your computer off. If you prefer C++, why do you waste your time on Modula-3? On 08/03/2021 18:48:33, Jay K wrote: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: ?amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). ?- Jay From: vvm at tut.by Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers ; Jay K ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align ? Hi! ?? ?I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) ?In addition, it's good idea convert all source code of OS from C language to Modula-3. ?May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. >?own linker ? It's looks like "Oberon way". >?free C tools (e.g. Pelles C) ?Is it has free source code? It's based on lcc, that's good. >?latest Visual Studio ?Why not? It's have native support of utf-8. >?Windows 7 Pro ?As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" : The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be:?(NT or Posix) and?(32bit or 64bit)?and?(endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". ? (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? ?- Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com [mailto:M3devel at elegosoft.com] https://m3lists.elegosoft.com/mailman/listinfo/m3devel [https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C7341e3aba9f347cebfe308d8e2585835%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637508216352212030%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HiG8FGj5U111E2FiqOZaIV1iK3mISc4WVZ6CB2uki1c%3D&reserved=0] -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 8 19:37:39 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 08 Mar 2021 21:37:39 +0300 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <323031615223310@mail.yandex.by> Message-ID: <211751615228368@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 19:57:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 18:57:10 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: <211751615228368@mail.yandex.by> References: <323031615223310@mail.yandex.by> , <211751615228368@mail.yandex.by> Message-ID: Yes and no. My interest isn't so much language interop, as reuse of language backends and execution environments. For example in WebAssembly you get portable executable and possibly run in browser (node.js/deno, etc.). In C++ you get efficient exception handling. Generate C++ and then declare the system quite portable and done. Language interop is another thing, true. Every few years another similar solution rolls out recieving similar mixed success as previous. There are countless examples. All programming is modular, more or less, with varying definition of modular. It is the only way to implement larger systems. To me, modularity includes mainly data hiding. Program through functions. There should be a minimum/zero public data in records/structs or globals, Many systems and Modula-3 violate this. Modula-3 isn't modular though. Where Modula-3 is unique vs. historical C and C++ is the language/compiler assumes a certain form of modularity. Where names have local scoped meaning, enabling fast compilation. C++ has this "now", but it is quite new, decades late. (modules, like in C++20 and not yet in widespread use). C and C++ practise have been there for decades, ignoring occasional abuse, and the language/compiler have to be pessimistic and slow because of occasional abuse. This is why C and C++ are far and away the slowest languages to compile. Not because of their complexity or their highly optimizing backends. What initially attracted me to Modula-3 was: - smaller easier to work on system - optionally safe native code; this was fairly unique circa 1995, but now is redundant with, just a few systems, .NET Native, Mono AOT, Rust, Go. - fast compilation to native code; again this was unique at the time Only later did I come to appreciate the "expressiveness" of C++ and realize how all the other languages fall short. Only later, granted, did I develop a distaste for JITs and garbage collectors. (Esp. "conservative" garbage collectors like Modula-3's and Mono's -- they leak.) And only recently do we have in Rust safety w/o garbage collection, very compelling. (if you ignore Cyclone). The language interop of Modula-3 and C is actually problematic. You cannot pass the address of nested functions to C. You cannot work with traced references in C. Just recently I stopped myself from rewriting some parts of cm3 in C because I realized I might write tracedRef->field and miss out on using barriers or such. The language interop of everything with C++ is problematic because C++ has so much better support for user defined types than the rest. For trivial example: class Person { virtual void Talk() = 0; }; void f1() { Person p; f2(&p); } void f2(Person* p) { p->Talk(); } does not translate to Modula-3. Modula-3 has the unfortunate limitation, I think, that "member functions" required traced references. I think. So you have to say: void f1() { f2(new Person); } wasting a heap allocation and adding memory pressure for the garbage collector to deal with. I understand the point about "dogfooding" but on the other side, I am concerned with ease of build. It is not lost on my that the fastest Java and C# compilers are the ones not written in Java and C#. People are accustomed to install C compiler binaries as if by magic: apt-get install gcc And the C compilers have passed whatever repeatable auditable source build requirements that Debian and Redhat have. They don't actually just accept binaries. And mixed for the test: apt-get install helloworld or wget whatever.com/helloworld.tgz tar xf helloworld.tgz cd helloworld ./configure # or cmake . make make install It'd be nice to get Modula-3 into the last form. Writing the compiler in Modula-3 fights that. Generating C sort of helps, but does not on its own meet the requirements to be in a Linux distribution. Generated C is still "binary" or more generally "derived". But it still helps people with looser ideals. Rewriting the frontend in C or C++ does solve that. It is a tradeoff, I agree. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 8, 2021 6:37 PM To: Mika Nystrom ; Jay K Cc: Dirk Muysers ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align Hi! I agree with all ideas/theses. ( > (maybe Oberon or some dialect of M2 is simpler, I dunno?) They are simply "more than need". ) Beat regards, Victor Miasnikov 08.03.2021, 21:13, "Mika Nystrom" : I sort of agree with Jay here. It's super important for M3 code to be able to co-exist well with C, and ideally with C++ as well. There is so much software out there that's written with C interfaces that we really want to be able to use. The fact that M3 interfaces poorly with C++ is starting to be an issue. M3 on the other hand interfaces amazingly well with C, if the C code is written well (i.e., if the C code is written in an "interface-based style" because that is what M3 itself lends itself to well). Most modern C code I have seen is indeed written in an interface-based style. The C folks finally learned something from the Modula world (ok the rest of the world calls it "Pascal" but we know that's a bit inaccurate), even if it has been slow going! In short, being able to interface to C/C++ is absolutely essential to being able to write useful M3 applications today. Although, Jay, I do disagree about rewriting m3front in C. I think m3front should be in M3 and you can bootstrap (or for you, cross-compile) as needed. Really think you should minimize the amount of C in the compiler and system in general, BUT maintain good general compatibity with C/C++. The reason I think it's important that the tools be written in M3 is a bit philosophical. I write a lot of code generators. And code "understanders" [not a real word I know but it's probably clear what I mean]. In my opinion, this is one of the strongest areas of M3: it's among the simplest of the practical imperative programming languages (maybe Oberon or some dialect of M2 is simpler, I dunno?) to parse and generate. These are also extremely strong and effective general-purpose programming techniques. If we can't use the code understanders and generators from M3 (eating our own dogfood) we are really not making a good show of programming. In fact this I think is one of the real holes in the Java ecosystem: that the Java system itself is written in C++. If M3 can't even implement its own compiler, I think it has no claim on the title "systems programming language". And, in fact, I think it's quite a good "systems programming language" (doesn't everyone on this list???? you'd assume so...) Mika P.S. Design example. I had to write some code to deal with obnoxiously large polynomials a few months ago. They had coefficients that were very large integers and high non-integer exponents (so they were actually so-called generalized polynomials and not normal polynomials). I did not want to implement a big math package myself and wasn't entirely happy with what was in the m3 tree, so I used MPRF (multiple-precision floating point library) from GNU and INRIA. Adding this to M3 was really easy because MPFR already uses interfaces in the sense that the type mpfr_t is opaque. A opaque C pointer INTERFACE MpfrP; TYPE T = ADDRESS; <*EXTERNAL MpfrC__alloc*> (* implemented in C glue *) PROCEDURE alloc() : MpfrPtrT; <*EXTERNAL MpfrC__free*> (* implemented in C glue *) PROCEDURE free(x : MpfrPtrT); <*EXTERNAL mpfr_init2*> PROCEDURE init2(t : T; prec : Prec); <*EXTERNAL mpfr_add*> PROCEDURE add(tgt, a, b : T; rnd : Rnd): int; <*EXTERNAL mpfr_sub*> PROCEDURE sub(tgt, a, b : T; rnd : Rnd): int; etc .... giving an interface to C code from the MPFR library plus a few glue bits to do alloc and free. Then on top of this build a very thin layer of M3 that looks like pure M3 and has the same procedures available but on an M3 type. INTERFACE Mpfr; TYPE T <: REFANY; RM = MpfrRoundingMode.T; Ternary = [ -1..1 ]; PROCEDURE New(prec : CARDINAL) : T; PROCEDURE Add(tgt, a, b : T; rnd := RM.N) : Ternary; WeakRef on the M3 handles the freeing of the underlying C type. Then using the "understanders" I talked about above, I do the following one-liner in an m3makefile somewhere: SchemeStubs ("Mpfr") I can now immediately use all this from the scheme interpreter ("mscheme" library). So my scheme immediately has access to a very powerful floating-point library after that, with GC and everything working on this fairy low-level C code, right away. It was all done in a few hours, very easy. But being able to pull these sorts of things off easily depends on being able to co-exist well with C. On Mon, Mar 8, 2021 at 9:48 AM Jay K > wrote: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers >; Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align Hi! I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) In addition, it's good idea convert all source code of OS from C language to Modula-3. May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > own linker It's looks like "Oberon way". > free C tools (e.g. Pelles C) Is it has free source code? It's based on lcc, that's good. > latest Visual Studio Why not? It's have native support of utf-8. > Windows 7 Pro As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" >: The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K > wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 8 20:03:51 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 08 Mar 2021 22:03:51 +0300 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: , <323031615223310@mail.yandex.by> Message-ID: <329211615229470@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 20:44:08 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 19:44:08 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: <340931615230499@mail.yandex.by> References: , <323031615223310@mail.yandex.by> <329211615229470@mail.yandex.by>,<340931615230499@mail.yandex.by> Message-ID: Right, the stack scanning, at least. It causes leaks. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 8, 2021 11:14:38 AM To: Jay K ; Dirk Muysers ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align Hi! Small question: > . "conservative" garbage collectors like Modula-3's Conservative GC in early M3 versions? I hope today we have "non-conservative GC in modern Modula-3? ( may be except stack traversal?) I red some comments in M3 GC source codes, but as I understand that M3 has different GC in his history. Best regards, Victor Miasnikov 08.03.2021, 22:03, "vvm at tut.by" : Hi! >Windows 7 is good. >Windows 10 is good. >The Microsoft C++ toolchain is good. It's good that M3 can continue support of "VMS for x86". > converting Modula-x to Modula-y I didn't write about this in my message about support of Windows, but... Utility m2tom3 is big advantage of Modula-3 world. The real alternative of C language based OSes is A2 writed on Active Oberon. I.e. Mac/*nix aren't real alternative of Windows. We have example of successful using of Modula-2 instead C language in XDS Modula-2/Oberon-2 compiler. > Writing a linker is a lot of work and not likely worthwhile. > Are you prepared to write symbols for debugging? Yes, I am not shue that "Oberon way" is always exelent. Support of debuggers very often simply doesn't implement in Oberon compilers. ( XDS can generate .sym files. It's early variant of ..pdb) Best regards, Victor Miasnikov 08.03.2021, 20:48, "Jay K" >: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers >; Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align Hi! I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) In addition, it's good idea convert all source code of OS from C language to Modula-3. May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > own linker It's looks like "Oberon way". > free C tools (e.g. Pelles C) Is it has free source code? It's based on lcc, that's good. > latest Visual Studio Why not? It's have native support of utf-8. > Windows 7 Pro As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" >: The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K > wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 20:56:09 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 19:56:09 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <,323031615223310@mail.yandex.by> , Message-ID: To me and I think many people, the point of Rust is safety with garbage collection. Maybe the data race analysis also, I understand that less. Safety without garbage collection is fairly unique. The rest of the language is nothing special or worse. If you can invent something without GC or JIT and closer to C++ you will do very well. - Jay ________________________________ From: Mika Nystrom Sent: Monday, March 8, 2021 11:22:05 AM To: Dirk Muysers Cc: Jay K ; vvm at tut.by ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align Dirk I agree with you too, without necessarily saying Jay is wrong. I think people should not write C++, ever---that language is really an unmitigated disaster from the points of view that informed the M3 design (safety, simplicity, etc). I think the failings of C++ are much of the reason Rust is taking off. And I understand that the more informed members of the Rust community (which seems to be a fairly informed community in the first place) are well aware of what the Modula family has done. In particular I think M3 was trying to solve some of the same problems that Rust tries today to solve. Personally I think there are more programs that would benefit from being written in M3 than there are programs that would benefit from being written in Rust, but I can see the arguments for both paths away from C++, and am even willing to say that there are situations where Rust makes for a better solution than M3. Not the types of programs I write, though. At the same time, I use C interfaces because it lets me interface to other people's code. I think it's a must. Now if C++ can coexist somehow... hidden... internally... no objection to that. There is so much code out there nowadays that it really helps to be able to use other people's code. Some C++ and C code out there is very well written. yes they had to work at lot harder at it, maybe, than if they had coded in M3 in the first place, but still, they didn't know that, and life is too short to reinvent absolutely every wheel... Mika On Mon, Mar 8, 2021 at 10:16 AM Dirk Muysers > wrote: Welcome to the Microsoft universe (/hell). The very reason of Modula-3's existence was to forget C++ in favour of a better, safer, and simpler alternative. I don't want it to become yet another, hidden way to program in... C++! >> (if you don't like C++) turn your computer off. If you prefer C++, why do you waste your time on Modula-3? On 08/03/2021 18:48:33, Jay K > wrote: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers >; Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align Hi! I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) In addition, it's good idea convert all source code of OS from C language to Modula-3. May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > own linker It's looks like "Oberon way". > free C tools (e.g. Pelles C) Is it has free source code? It's based on lcc, that's good. > latest Visual Studio Why not? It's have native support of utf-8. > Windows 7 Pro As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" >: The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K > wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 8 22:37:57 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 8 Mar 2021 21:37:57 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <,323031615223310@mail.yandex.by> , , Message-ID: m3front: Remove <*LazyAlign*> and <*StrictAlign*> by jaykrell ? Pull Request #130 ? modula3/cm3 (github.com) Ok? - Jay ________________________________ From: Jay K Sent: Monday, March 8, 2021 7:56 PM To: Mika Nystrom ; Dirk Muysers Cc: vvm at tut.by ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align To me and I think many people, the point of Rust is safety with garbage collection. Maybe the data race analysis also, I understand that less. Safety without garbage collection is fairly unique. The rest of the language is nothing special or worse. If you can invent something without GC or JIT and closer to C++ you will do very well. - Jay ________________________________ From: Mika Nystrom Sent: Monday, March 8, 2021 11:22:05 AM To: Dirk Muysers Cc: Jay K ; vvm at tut.by ; m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align Dirk I agree with you too, without necessarily saying Jay is wrong. I think people should not write C++, ever---that language is really an unmitigated disaster from the points of view that informed the M3 design (safety, simplicity, etc). I think the failings of C++ are much of the reason Rust is taking off. And I understand that the more informed members of the Rust community (which seems to be a fairly informed community in the first place) are well aware of what the Modula family has done. In particular I think M3 was trying to solve some of the same problems that Rust tries today to solve. Personally I think there are more programs that would benefit from being written in M3 than there are programs that would benefit from being written in Rust, but I can see the arguments for both paths away from C++, and am even willing to say that there are situations where Rust makes for a better solution than M3. Not the types of programs I write, though. At the same time, I use C interfaces because it lets me interface to other people's code. I think it's a must. Now if C++ can coexist somehow... hidden... internally... no objection to that. There is so much code out there nowadays that it really helps to be able to use other people's code. Some C++ and C code out there is very well written. yes they had to work at lot harder at it, maybe, than if they had coded in M3 in the first place, but still, they didn't know that, and life is too short to reinvent absolutely every wheel... Mika On Mon, Mar 8, 2021 at 10:16 AM Dirk Muysers > wrote: Welcome to the Microsoft universe (/hell). The very reason of Modula-3's existence was to forget C++ in favour of a better, safer, and simpler alternative. I don't want it to become yet another, hidden way to program in... C++! >> (if you don't like C++) turn your computer off. If you prefer C++, why do you waste your time on Modula-3? On 08/03/2021 18:48:33, Jay K > wrote: No. Windows 7 is good. Windows 10 is good. The Microsoft C++ toolchain is good. If you don't like them, use Linux or Mac, etc. or turn off your computer. I will not be converting Modula-x to Modula-y. There are source to source transforms that are useful, but not that. Consider instead C++ or JavaScript or TypeScript or Rust or LLVM IR or Java, as a means to connecting to other good backends. Writing a linker is a lot of work and not likely worthwhile. Are you prepared to write symbols for debugging? NT on Mips/PowerPC/Alpha might be nice, but OpenNT is apparently ancient unlicensed code. Mips/PowerPC/Alpha are plenty usable otherwise, for example Linux in Qemu, or real hardware. Though I find it difficult to care about any more than, already a long target list: amd64, arm64, riscv64, WebAssembly (32 now, 64 soon), NT, Darwin, Linux, Fuschia. And generating C++, really, gets you extremely broad reach. Though even then, closures, threads, exception handling, can be problems (esp. in wasm presently). - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 8, 2021 5:33 PM To: Dirk Muysers >; Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align Hi! I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. ( *nix can't. Main reason: W N T := V++ M++ S++) In addition, it's good idea convert all source code of OS from C language to Modula-3. May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > own linker It's looks like "Oberon way". > free C tools (e.g. Pelles C) Is it has free source code? It's based on lcc, that's good. > latest Visual Studio Why not? It's have native support of utf-8. > Windows 7 Pro As I known, Win 7 has equal telemetry level as Win 10. ( It's instaled by monthly hotfixes. 3-5 years ago) Best regards, Victor Miasnikov 08.03.2021, 17:54, "Dirk Muysers" >: The less M3 depends on Windows, the better. Don't oblige me to download the latest Visual Studio for Windows 10. For the moment I stick to AMD64 Windows 7 Pro (and a good Norton protection) because I want to own my computer instead of Microsoft owning it, spying on me, and force-install new versions that sometimes are buggy and prevent you from using your box as it happened lately. I think M3 should have at least its own linker or use free C tools (e.g. Pelles C) rather than MS's monstruous infrastructure. On 08/03/2021 10:35:19, Jay K > wrote: I'm looking at target-specificness in m3front/m3middle again, and lazyalign shows up. I would really really really like targets to have far less difference. I believe it is mostly counter productive. I understand x86 can pack things more tightly and this can save memory with certain structures. And sharing data with C is made easier sometimes. But I want the target matrix to be: (NT or Posix) and (32bit or 64bit) and (endian) and then 32bit and big endian can be considered unusual, so people can just chose to download Windows or Unix archives, and I don't think x86/amd64 merit a further expansion of this matrix. The system will be easier to setup, as long as it remains "self hosted". (Rewriting m3front in C or C++ is another alternative.) This will break people's Pickles? Or the Pickle code will adapt, just as if tranferring across architecture? I think it will adapt. The lazyalign and strictalign pragmas would be ignored, or warn. The way they are implemented seems dubious, in that when you see lazyalign, it alters Target, and I don't know if the ordering of that is good -- does adding a lazyalign type to a library change the code or layout of the entire library? Thoughts please? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Mar 9 01:32:14 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 8 Mar 2021 19:32:14 -0500 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <323031615223310@mail.yandex.by> Message-ID: <20210309003214.xewhqxneigpay2gk@topoi.pooq.com> On Monday, March 8, 2021 5:33 PM, vvm at tut.by wrote: > > I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. > > ( *nix can't. Main reason: W N T := V++ M++ S++) > > In addition, it's good idea convert all source code of OS from C language to Modula-3. > > May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > > > own linker > > It's looks like "Oberon way". The problem with having our own linker is that it makes interoperation with other languages more difficult. Especially other language that have *their* own linkers. -- hendrik From jayk123 at hotmail.com Tue Mar 9 02:10:09 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 01:10:09 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: <20210309003214.xewhqxneigpay2gk@topoi.pooq.com> References: <323031615223310@mail.yandex.by> , <20210309003214.xewhqxneigpay2gk@topoi.pooq.com> Message-ID: Linkers do not implement languages. Any that works is fine and does not hurt interop. But having a linker won?t help anything either except provide the fun and learning of writing it. - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Monday, March 8, 2021 4:32:14 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align On Monday, March 8, 2021 5:33 PM, vvm at tut.by wrote: > > I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. > > ( *nix can't. Main reason: W N T := V++ M++ S++) > > In addition, it's good idea convert all source code of OS from C language to Modula-3. > > May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > > > own linker > > It's looks like "Oberon way". The problem with having our own linker is that it makes interoperation with other languages more difficult. Especially other language that have *their* own linkers. -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C06502b1df95c497cf46408d8e292e448%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637508467822449721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iXzQIDk7BAjFJxNSTD8lqwUaPWTHpvAU%2FU%2F4YHePlnk%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Mar 9 02:25:59 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 8 Mar 2021 20:25:59 -0500 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: References: <323031615223310@mail.yandex.by> <20210309003214.xewhqxneigpay2gk@topoi.pooq.com> Message-ID: <20210309012559.7rzsrzarnevyn3qz@topoi.pooq.com> On Tue, Mar 09, 2021 at 01:10:09AM +0000, Jay K wrote: > Linkers do not implement languages. Any that works is fine and does not hurt interop. But > having a linker won?t help anything either except provide the fun and learning of writing it. > Linkers shouldn't implement languages. But: Whe C++ got serious, the linker ended up calling the compiler to instantiate unresolved template invocations. That was back in the 90's. It's an example of what we do *not* want. That's the compiler's job. -- hendrik > - Jay > ________________________________ > From: M3devel on behalf of Hendrik Boom > Sent: Monday, March 8, 2021 4:32:14 PM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align > > On Monday, March 8, 2021 5:33 PM, vvm at tut.by wrote: > > > > I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. > > > > ( *nix can't. Main reason: W N T := V++ M++ S++) > > > > In addition, it's good idea convert all source code of OS from C language to Modula-3. > > > > May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > > > > > own linker > > > > It's looks like "Oberon way". > > The problem with having our own linker is that it makes interoperation with other languages more > difficult. Especially other language that have *their* own linkers. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C06502b1df95c497cf46408d8e292e448%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637508467822449721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iXzQIDk7BAjFJxNSTD8lqwUaPWTHpvAU%2FU%2F4YHePlnk%3D&reserved=0 From jayk123 at hotmail.com Tue Mar 9 02:41:37 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 01:41:37 +0000 Subject: [M3devel] eliminating lazy/strict align In-Reply-To: <20210309012559.7rzsrzarnevyn3qz@topoi.pooq.com> References: <323031615223310@mail.yandex.by> <20210309003214.xewhqxneigpay2gk@topoi.pooq.com> , <20210309012559.7rzsrzarnevyn3qz@topoi.pooq.com> Message-ID: > When C++ got serious, the linker ended up calling the compiler to instantiate unresolved template I understand and had forgotten but this approach is rare or nonexistent these days. It is a legitimate problem, difficult to implement well, and C++ and Modula3 both do it poorly. C++ implementation require templates be textually visible, like in .h files. Modula3 requires manual instantiation in the build system. C++ is slower to compile as a result. Modula3 is more tedious and requires bad level of cross cutting cooperation: I recently broke cm3 build because I added a generic instantiation. C++ modules will help. And C++ already split up the checking between ?generic? and instantiated, but checking instantiated must remain some. Modula3 I think didn?t split the checking ? it is difficult to specify ? so remains slow. Go is like Modula3. Generics in C# have problems too. They are just not trivial. In particular C# static compilers cannot find all the instantiations so must have bad fallbacks of either JIT or a generalize slow but space-efficient version. The old C++ way with linker help was arguably better. Generics in all languages do increase load and expectations on backend and linker. More code, that is same or similar. More demand for inlining and desire to combine identical functions (unless maybe addesss-taken). Visual C++ linker has been combining identical functions for decades. Symbol names also get longer. Code search gets harder (in Modula3 you get the derived sources to mess up code search). - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Monday, March 8, 2021 5:25:59 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] eliminating lazy/strict align On Tue, Mar 09, 2021 at 01:10:09AM +0000, Jay K wrote: > Linkers do not implement languages. Any that works is fine and does not hurt interop. But > having a linker won?t help anything either except provide the fun and learning of writing it. > Linkers shouldn't implement languages. But: Whe C++ got serious, the linker ended up calling the compiler to instantiate unresolved template invocations. That was back in the 90's. It's an example of what we do *not* want. That's the compiler's job. -- hendrik > - Jay > ________________________________ > From: M3devel on behalf of Hendrik Boom > Sent: Monday, March 8, 2021 4:32:14 PM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] eliminating lazy/strict align > > On Monday, March 8, 2021 5:33 PM, vvm at tut.by wrote: > > > > I hope, that ReactOs as "free Windows" and(or) projects like OpenNT can be solution. > > > > ( *nix can't. Main reason: W N T := V++ M++ S++) > > > > In addition, it's good idea convert all source code of OS from C language to Modula-3. > > > > May be except 16bit part ( boot loader, MS DOS and Win16 emulation). It's can be convert to Modula-2. > > > > > own linker > > > > It's looks like "Oberon way". > > The problem with having our own linker is that it makes interoperation with other languages more > difficult. Especially other language that have *their* own linkers. > > -- hendrik > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C68e26e356dce4d1499c608d8e29a5808%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637508499817414794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=aBGvPcBqn0N0NsV6E%2FRv%2FmSJ4WPuPeq0k%2FNRijT4d%2Bw%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C68e26e356dce4d1499c608d8e29a5808%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637508499817424791%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=chFW6C0EOypsZ4vzXSFkpPEboXok0yl31%2Bgsx37Zf88%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 9 08:11:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 07:11:06 +0000 Subject: [M3devel] closures are a problem Message-ID: Closures are a problem. Specifically the ability to take the address of a nested function and pass that as a function pointer, and that it has the same type as a pointer to a non-nested or C function, and it can access the locals of the lexical enclosure (arbitrary levels deep, with recursion, but this doesn't make it worse). The problems: - The assumption that -1 is not valid code. (This value could be changed per system). - The assumption that code pages are readable. Some systems have execute-only memory. Reading works on wasm but I don't know what it does, wierd. - The assumption that reading code pages are meaningful. (wasm?) - The inability to pass these pointers to C code as function pointers. - I believe...sorry, I should know better, but I believe in the C backend I play lose with the signatures of functions, passing an extra parameter always, to functions that might not expect it. This likely plays poorly with some environments, such as authenticated ARM function pointers and WebAssembly. WebAssembly might eventually get a feature to help here. So, really, I wish this feature was not in the language. Barring that, maybe we can remove its use from the base system? Maybe have an optional flag or two: - prohibit taking address of nested function - assume function pointers are not closures I realize these are different. In that, the second idea in a lower level library, breaks use with a higher level library. I know about several approaches to solve this problem, but none of them are good. Traditional gcc generates code on the stack. Apple removed this from their compilers years ago. In some systems you can stamp out identical copies of code, with associated data, and manage that like a cheap heap. But I think that still is not worth while. It still is not cheap, and it is platform specific. Libffi and Mono use this on iPhone for example. Libffi will be using it for all systems, at least all Linux systems. Some systems depend on JIT for this (e.g. CLR on non-iphone). In normal C code this is handled in a reasonable way, case by case. For example the qsort_s function (not regular qsort) takes function pointer and a void*. The void* is extra context agreed upon between the qsort caller and the function pointer. qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. This achieves the same result as a "closure". Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. I'll try to find where in the system we use closures and if they can be "fixed" somehow. I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. The C output already mostly can compile to WebAssembly, not having done anything. I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Tue Mar 9 08:32:52 2021 From: vvm at tut.by (vvm at tut.by) Date: Tue, 09 Mar 2021 10:32:52 +0300 Subject: [M3devel] closures are a problem In-Reply-To: References: Message-ID: <412531615274833@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 9 08:45:35 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 07:45:35 +0000 Subject: [M3devel] closures are a problem In-Reply-To: <412531615274833@mail.yandex.by> References: , <412531615274833@mail.yandex.by> Message-ID: That's kinda what I meant. The problem isn't nested functions, as a naming/hiding thing. The problem isn't even nested functions accessing their lexically enclosing locals. The problem is taking the address of a nested function. - Jay ________________________________ From: vvm at tut.by Sent: Tuesday, March 9, 2021 7:32 AM To: Jay K ; m3devel Subject: Re: [M3devel] closures are a problem Hi! As small remark: > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. Even Oberon-07 didn't remove nested function. It only can't access to ( read/write) local variable in "parent function". Others Oberon(s) have "usual" nested functions. Best regards, Victor Miasnikov 09.03.2021, 10:11, "Jay K" : Closures are a problem. Specifically the ability to take the address of a nested function and pass that as a function pointer, and that it has the same type as a pointer to a non-nested or C function, and it can access the locals of the lexical enclosure (arbitrary levels deep, with recursion, but this doesn't make it worse). The problems: - The assumption that -1 is not valid code. (This value could be changed per system). - The assumption that code pages are readable. Some systems have execute-only memory. Reading works on wasm but I don't know what it does, wierd. - The assumption that reading code pages are meaningful. (wasm?) - The inability to pass these pointers to C code as function pointers. - I believe...sorry, I should know better, but I believe in the C backend I play lose with the signatures of functions, passing an extra parameter always, to functions that might not expect it. This likely plays poorly with some environments, such as authenticated ARM function pointers and WebAssembly. WebAssembly might eventually get a feature to help here. So, really, I wish this feature was not in the language. Barring that, maybe we can remove its use from the base system? Maybe have an optional flag or two: - prohibit taking address of nested function - assume function pointers are not closures I realize these are different. In that, the second idea in a lower level library, breaks use with a higher level library. I know about several approaches to solve this problem, but none of them are good. Traditional gcc generates code on the stack. Apple removed this from their compilers years ago. In some systems you can stamp out identical copies of code, with associated data, and manage that like a cheap heap. But I think that still is not worth while. It still is not cheap, and it is platform specific. Libffi and Mono use this on iPhone for example. Libffi will be using it for all systems, at least all Linux systems. Some systems depend on JIT for this (e.g. CLR on non-iphone). In normal C code this is handled in a reasonable way, case by case. For example the qsort_s function (not regular qsort) takes function pointer and a void*. The void* is extra context agreed upon between the qsort caller and the function pointer. qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. This achieves the same result as a "closure". Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. I'll try to find where in the system we use closures and if they can be "fixed" somehow. I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. The C output already mostly can compile to WebAssembly, not having done anything. I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. Thoughts? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Tue Mar 9 08:54:20 2021 From: vvm at tut.by (vvm at tut.by) Date: Tue, 09 Mar 2021 10:54:20 +0300 Subject: [M3devel] closures are a problem In-Reply-To: References: , <412531615274833@mail.yandex.by> Message-ID: <390221615276071@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 9 12:28:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 11:28:10 +0000 Subject: [M3devel] closures are a problem In-Reply-To: <390221615276071@mail.yandex.by> References: , <412531615274833@mail.yandex.by> , <390221615276071@mail.yandex.by> Message-ID: I know how it works. Granted, maybe I forgot some. I wrote the C backend. I create and use those structs there, and got the "static link" to work. I have debugged, it wasn't super difficult, what happens when "aligned_procedures" is wrong, since I brought up, e.g. sparc64, where it matters. "aligned procedures", is that before calling any function pointer, the data it points to are looked it, and if they are the integer (pointer-sized) -1, then it is a closure, and is followed by a function pointer and data pointer. On systems that care about alignment and for which instructions and functions are often only 4-aligned, when checking for an 8-byte integer, this suffers an alignment fault. So for those targets, we first check the alignment and if it isn't 8-aligned, it is deemed not a closure. This is all not great, for many reasons. The value -1 is chosen arbitrarily to be, probably not code. It could be varied per target. The check for an integer, should maybe be a check for a 4 byte integer, since the systems that care about alignment, have 4-aligned functions. Typically. But this does not address the larger problems. "all" local variables in "every" function are put in a local struct. A pointer to that struct is passed to nested functions. This includes that very pointer, i.e. for multiple levels of testing. You don't have to put "all" locals of "every" functions, just the ones referenced by nested functions. Some ABIs have an extra register reserved for passing the static link and m3cc leverages that. The C backend cannot, sort of. We might be able to actually, at least on Unix systems or when using gcc. There might be an attribute or something, to indicate passing the parameter that way. We could even on non-Apple systems, output nested functions. Perhaps. I hadn't really considered that before. I was very aiming for standard portable C. Perhaps a mistake. Perhaps. - Jay ________________________________ From: vvm at tut.by Sent: Tuesday, March 9, 2021 7:54 AM To: Jay K ; m3devel Subject: Re: [M3devel] closures are a problem Hi! > That's kinda what I meant. Yes, I understand. But in any case, O7 isn't be classical Oberon. If you want, I can send article that describe early version of XDS Oberon-2/Modula-2. But, as I remember, all local variable in "parent function" joins to one RECORD. And this record be parameter of "nested function". Best regards, Victor Miasnikov 09.03.2021, 10:45, "Jay K" : That's kinda what I meant. The problem isn't nested functions, as a naming/hiding thing. The problem isn't even nested functions accessing their lexically enclosing locals. The problem is taking the address of a nested function. - Jay ________________________________ From: vvm at tut.by > Sent: Tuesday, March 9, 2021 7:32 AM To: Jay K >; m3devel > Subject: Re: [M3devel] closures are a problem Hi! As small remark: > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. Even Oberon-07 didn't remove nested function. It only can't access to ( read/write) local variable in "parent function". Others Oberon(s) have "usual" nested functions. Best regards, Victor Miasnikov 09.03.2021, 10:11, "Jay K" >: Closures are a problem. Specifically the ability to take the address of a nested function and pass that as a function pointer, and that it has the same type as a pointer to a non-nested or C function, and it can access the locals of the lexical enclosure (arbitrary levels deep, with recursion, but this doesn't make it worse). The problems: - The assumption that -1 is not valid code. (This value could be changed per system). - The assumption that code pages are readable. Some systems have execute-only memory. Reading works on wasm but I don't know what it does, wierd. - The assumption that reading code pages are meaningful. (wasm?) - The inability to pass these pointers to C code as function pointers. - I believe...sorry, I should know better, but I believe in the C backend I play lose with the signatures of functions, passing an extra parameter always, to functions that might not expect it. This likely plays poorly with some environments, such as authenticated ARM function pointers and WebAssembly. WebAssembly might eventually get a feature to help here. So, really, I wish this feature was not in the language. Barring that, maybe we can remove its use from the base system? Maybe have an optional flag or two: - prohibit taking address of nested function - assume function pointers are not closures I realize these are different. In that, the second idea in a lower level library, breaks use with a higher level library. I know about several approaches to solve this problem, but none of them are good. Traditional gcc generates code on the stack. Apple removed this from their compilers years ago. In some systems you can stamp out identical copies of code, with associated data, and manage that like a cheap heap. But I think that still is not worth while. It still is not cheap, and it is platform specific. Libffi and Mono use this on iPhone for example. Libffi will be using it for all systems, at least all Linux systems. Some systems depend on JIT for this (e.g. CLR on non-iphone). In normal C code this is handled in a reasonable way, case by case. For example the qsort_s function (not regular qsort) takes function pointer and a void*. The void* is extra context agreed upon between the qsort caller and the function pointer. qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. This achieves the same result as a "closure". Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. I'll try to find where in the system we use closures and if they can be "fixed" somehow. I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. The C output already mostly can compile to WebAssembly, not having done anything. I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. Thoughts? - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 9 20:30:16 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 19:30:16 +0000 Subject: [M3devel] more win64 subrange problems uint32 := -1. Message-ID: Searching for 'lockfile'... C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): IF WinBase.LockFile( C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): IF WinBase.UnlockFile( C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): IF WinBase.LockFile( C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): IF WinBase.UnlockFile( 4 occurrence(s) have been found. IF WinBase.LockFile( hFile := h.handle, dwFileOffsetLow := 0, dwFileOffsetHigh := 0, nNumberOfBytesToLockLow := -1, nNumberOfBytesToLockHigh := -1) = 0 THEN err := WinBase.GetLastError(); PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= BEGIN IF WinBase.UnlockFile( hFile := h.handle, dwFileOffsetLow := 0, dwFileOffsetHigh := 0, nNumberOfBytesToUnlockLow := -1, nNumberOfBytesToUnlockHigh := -1) = 0 THEN OSErrorWin32.Raise() END; END RegularFileUnlock; <*EXTERNAL LockFile:WINAPI*> PROCEDURE LockFile (hFile : HANDLE; dwFileOffsetLow : UINT32; dwFileOffsetHigh : UINT32; nNumberOfBytesToLockLow : UINT32; nNumberOfBytesToLockHigh: UINT32 ): BOOL; <*EXTERNAL UnlockFile:WINAPI*> PROCEDURE UnlockFile (hFile : HANDLE; dwFileOffsetLow : UINT32; dwFileOffsetHigh : UINT32; nNumberOfBytesToUnlockLow : UINT32; nNumberOfBytesToUnlockHigh: UINT32 ): BOOL; These get warnings when compiling. But only for Win64, not Win32. I'm not sure if they fail at runtime or not. It isn't terrible pattern, I think, to reserve a few values like this. Perhaps in this case, you might as well give up half the values though and use int32? Well, on "high", but not on low". Of course, taking an int64 and not two 32bit integers, would be preferable. I am inclined, as ever, to just move a few lines of Modula3 to C, because I very much dislike the pattern of rewriting system C headers in other languages. Though it perhaps it isn't too bad on Windows. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Mar 9 23:14:21 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 9 Mar 2021 16:14:21 -0600 Subject: [M3devel] closures are a problem In-Reply-To: References: Message-ID: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > ?- The assumption that -1 is not valid code. > ?(This value could be changed per system). > > ?- The assumption that code pages are readable. > ?Some systems have execute-only memory. > ?Reading works on wasm but I don't know what it does, wierd. > > ?- The assumption that reading code pages are meaningful. (wasm?) > > ?- The inability to pass these pointers to C code as function pointers. > > ?- I believe...sorry, I should know better, but I believe in the C backend > ? ?I play lose with the signatures of functions, passing an extra parameter always, > ? ?to functions that might not expect it. > ? ?This likely plays poorly with some environments, such as authenticated ARM function pointers > ? ?and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > ?- prohibit taking address of nested function > ?- assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > ? Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Wed Mar 10 00:30:16 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 23:30:16 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: > He's working hard on getting > the system running on platforms > where it's not working toda To clarify and confirm, it does already work. AMD64_NT works, from long ago. ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). RISCV64_LINUX works. It builds itself in qemu. Debugging is also ok not great. I can make it better. I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. To fix the leaky conservative GC. Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. AppleWatch is a tough one. WebAssembly I believe also will not work, due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? Thank you, - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 9, 2021 3:08:04 PM To: rodney.m.bates at acm.org Cc: Jay K ; m3devel Subject: Re: [M3devel] closures are a problem I wouldn't be so harsh on Jay. He's working hard on getting the system running on platforms where it's not working today. I do agree though, Modula-3 is what it is. Those of us that use it, use it because we like what it offers, and I think most of the "deficiencies" in the language are there because they simplify something (either their own implementations, their own semantics, or some other semantics). I would not go around dumbing down M3, or even changing it in any significant way. As you write, if you don't like M3, then don't use it. Modula-3 doesn't have lots of things that C++ does have. That is deliberate (the green book even talks about this specific issue in some detail). Those of us that use M3 know that it lacks those things, and we use the language in spite of these things missing----or maybe BECAUSE those things are missing. To quote the Modula-3 report: 'In the early days of the Ada project, a general in the Ada Program Office opined that "obviously the Department of Defense is not interested in an artificially simplified language such as Pascal". Modula-3 represents the opposite point of view. We used every artifice that we could find or invent to make the language simple.' Yes, there are simplifying artifices. Sometimes they seem clunky for someone coming from C++, or Python, or Java. They are mostly there for a reason. Trying to get rid of those things misses the whole point of what Modula-3 is about. I'm with Rodney there. At the same time, I think that implementing some things in C isn't always a bad idea. Let me try to explain what I mean. I wrote in an earlier email (and gave an example) of how to take some C code that was written in an "interface-based paradigm" (which is really a programming approach that historically came out of Modula-2, as far as I know) and interfacing to that in M3. It's easy, IF the C is written "properly". It is a fact of life that Unix and many other OSes use C for their system-call interface. It is another fact of life that a lot of old C interfaces are NOT written in an exemplary interface-based style. Those interfaces are therefore a lot more difficult to attach to from M3. It makes sense to me to write glue code in C that cleans up the C interface in such a way that M3 can more easily attach, for example without cloning the headers. (I share Jay's worries about manually cloning C headers in M3.) Mika On Tue, Mar 9, 2021 at 2:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 00:40:26 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 23:40:26 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: No. Running the backend is noticeably slower but tolerable. I might try running the compiler on multiple files at once to improve that. It should help but requires icky builder work, or maybe quake. Given what the M3 IR looks like, I expect it should be comparable. It is very low level. You know, we have a lot of address-taking that either defeat the compilers, or not. That reminds me though. I have some hacks to avoid warnings that cause function calls. I should improve that. - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 9, 2021 3:34:11 PM To: Jay K Cc: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] closures are a problem Have you done a performance test of your generated C code vs. the gcc back end? Tony had some concerns, but we haven't heard from him for a while... what was that about? On Tue, Mar 9, 2021 at 3:30 PM Jay K > wrote: > He's working hard on getting > the system running on platforms > where it's not working toda To clarify and confirm, it does already work. AMD64_NT works, from long ago. ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). RISCV64_LINUX works. It builds itself in qemu. Debugging is also ok not great. I can make it better. I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. To fix the leaky conservative GC. Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. AppleWatch is a tough one. WebAssembly I believe also will not work, due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? Thank you, - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, March 9, 2021 3:08:04 PM To: rodney.m.bates at acm.org > Cc: Jay K >; m3devel > Subject: Re: [M3devel] closures are a problem I wouldn't be so harsh on Jay. He's working hard on getting the system running on platforms where it's not working today. I do agree though, Modula-3 is what it is. Those of us that use it, use it because we like what it offers, and I think most of the "deficiencies" in the language are there because they simplify something (either their own implementations, their own semantics, or some other semantics). I would not go around dumbing down M3, or even changing it in any significant way. As you write, if you don't like M3, then don't use it. Modula-3 doesn't have lots of things that C++ does have. That is deliberate (the green book even talks about this specific issue in some detail). Those of us that use M3 know that it lacks those things, and we use the language in spite of these things missing----or maybe BECAUSE those things are missing. To quote the Modula-3 report: 'In the early days of the Ada project, a general in the Ada Program Office opined that "obviously the Department of Defense is not interested in an artificially simplified language such as Pascal". Modula-3 represents the opposite point of view. We used every artifice that we could find or invent to make the language simple.' Yes, there are simplifying artifices. Sometimes they seem clunky for someone coming from C++, or Python, or Java. They are mostly there for a reason. Trying to get rid of those things misses the whole point of what Modula-3 is about. I'm with Rodney there. At the same time, I think that implementing some things in C isn't always a bad idea. Let me try to explain what I mean. I wrote in an earlier email (and gave an example) of how to take some C code that was written in an "interface-based paradigm" (which is really a programming approach that historically came out of Modula-2, as far as I know) and interfacing to that in M3. It's easy, IF the C is written "properly". It is a fact of life that Unix and many other OSes use C for their system-call interface. It is another fact of life that a lot of old C interfaces are NOT written in an exemplary interface-based style. Those interfaces are therefore a lot more difficult to attach to from M3. It makes sense to me to write glue code in C that cleans up the C interface in such a way that M3 can more easily attach, for example without cloning the headers. (I share Jay's worries about manually cloning C headers in M3.) Mika On Tue, Mar 9, 2021 at 2:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 00:47:19 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 23:47:19 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: Probably that you want to record information, correlated to instruction pointer ranges, as to GC roots in registers. You cannot do that with C. You are stuck with addresses of locals, impeding optimization. Or leaky conservative GC. Present gcc backend is same as C backend in this respect, but hypothetically possible to fix. They both take the leaky conservative GC approach. That is arguably all m3front offers...though the IR is fairly typed, I guess m3front has done its part, but you?d need to connect the backend-produced data to the runtime. Right? Unless this stuff is in there and I haven?t noticed? - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 9, 2021 3:37:13 PM To: Jay K Cc: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] closures are a problem OK I found it, but I am not sure what he's talking about: I note that the C/C++ route prevents a tighter integration of compiler and garbage collector which impedes performance significantly. I have for many years been working on a framework for supporting managed dynamic languages (MicroVM.org). We are in the throes of marrying that to LLVM and LLVM?s GC support for ahead of time compilation. I now believe it is possible to do something similar with M3 but it requires closer integration with LLVM?s type system. On Tue, Mar 9, 2021 at 3:34 PM Mika Nystrom > wrote: Have you done a performance test of your generated C code vs. the gcc back end? Tony had some concerns, but we haven't heard from him for a while... what was that about? On Tue, Mar 9, 2021 at 3:30 PM Jay K > wrote: > He's working hard on getting > the system running on platforms > where it's not working toda To clarify and confirm, it does already work. AMD64_NT works, from long ago. ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). RISCV64_LINUX works. It builds itself in qemu. Debugging is also ok not great. I can make it better. I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. To fix the leaky conservative GC. Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. AppleWatch is a tough one. WebAssembly I believe also will not work, due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? Thank you, - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, March 9, 2021 3:08:04 PM To: rodney.m.bates at acm.org > Cc: Jay K >; m3devel > Subject: Re: [M3devel] closures are a problem I wouldn't be so harsh on Jay. He's working hard on getting the system running on platforms where it's not working today. I do agree though, Modula-3 is what it is. Those of us that use it, use it because we like what it offers, and I think most of the "deficiencies" in the language are there because they simplify something (either their own implementations, their own semantics, or some other semantics). I would not go around dumbing down M3, or even changing it in any significant way. As you write, if you don't like M3, then don't use it. Modula-3 doesn't have lots of things that C++ does have. That is deliberate (the green book even talks about this specific issue in some detail). Those of us that use M3 know that it lacks those things, and we use the language in spite of these things missing----or maybe BECAUSE those things are missing. To quote the Modula-3 report: 'In the early days of the Ada project, a general in the Ada Program Office opined that "obviously the Department of Defense is not interested in an artificially simplified language such as Pascal". Modula-3 represents the opposite point of view. We used every artifice that we could find or invent to make the language simple.' Yes, there are simplifying artifices. Sometimes they seem clunky for someone coming from C++, or Python, or Java. They are mostly there for a reason. Trying to get rid of those things misses the whole point of what Modula-3 is about. I'm with Rodney there. At the same time, I think that implementing some things in C isn't always a bad idea. Let me try to explain what I mean. I wrote in an earlier email (and gave an example) of how to take some C code that was written in an "interface-based paradigm" (which is really a programming approach that historically came out of Modula-2, as far as I know) and interfacing to that in M3. It's easy, IF the C is written "properly". It is a fact of life that Unix and many other OSes use C for their system-call interface. It is another fact of life that a lot of old C interfaces are NOT written in an exemplary interface-based style. Those interfaces are therefore a lot more difficult to attach to from M3. It makes sense to me to write glue code in C that cleans up the C interface in such a way that M3 can more easily attach, for example without cloning the headers. (I share Jay's worries about manually cloning C headers in M3.) Mika On Tue, Mar 9, 2021 at 2:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 00:59:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 9 Mar 2021 23:59:31 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: Parallel should work here too. I will double check. But it is noticeably slower. ?Probably comparable? right, yes, I was ?trying? to answer both possible parts of the question albeit not well. (Parallel is broken on NT but that is a quake NT trivial problem I will fix, not a C backend problem. The log file names collide, is all.) - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 9, 2021 3:54:39 PM To: Jay K Cc: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] closures are a problem Not sure I care how long it takes to run the backend! Can't you just use the parallelization code in cm3 that we have for cm3cg? It's run in parallel more or less by default these days. I can try to find some time to work on this if I can figure out how your code plugs into cm3, but I remember it was not many hours' work to get cm3cg to run in parallel. Why would your backend be significantly different? What I'm really asking about is how good is the object code that's generated? Have you benchmarked that? Or is that what your "I expect it should be comparable" is referring to? On Tue, Mar 9, 2021 at 3:40 PM Jay K > wrote: No. Running the backend is noticeably slower but tolerable. I might try running the compiler on multiple files at once to improve that. It should help but requires icky builder work, or maybe quake. Given what the M3 IR looks like, I expect it should be comparable. It is very low level. You know, we have a lot of address-taking that either defeat the compilers, or not. That reminds me though. I have some hacks to avoid warnings that cause function calls. I should improve that. - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, March 9, 2021 3:34:11 PM To: Jay K > Cc: rodney.m.bates at acm.org >; m3devel > Subject: Re: [M3devel] closures are a problem Have you done a performance test of your generated C code vs. the gcc back end? Tony had some concerns, but we haven't heard from him for a while... what was that about? On Tue, Mar 9, 2021 at 3:30 PM Jay K > wrote: > He's working hard on getting > the system running on platforms > where it's not working toda To clarify and confirm, it does already work. AMD64_NT works, from long ago. ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). RISCV64_LINUX works. It builds itself in qemu. Debugging is also ok not great. I can make it better. I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. To fix the leaky conservative GC. Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. AppleWatch is a tough one. WebAssembly I believe also will not work, due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? Thank you, - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, March 9, 2021 3:08:04 PM To: rodney.m.bates at acm.org > Cc: Jay K >; m3devel > Subject: Re: [M3devel] closures are a problem I wouldn't be so harsh on Jay. He's working hard on getting the system running on platforms where it's not working today. I do agree though, Modula-3 is what it is. Those of us that use it, use it because we like what it offers, and I think most of the "deficiencies" in the language are there because they simplify something (either their own implementations, their own semantics, or some other semantics). I would not go around dumbing down M3, or even changing it in any significant way. As you write, if you don't like M3, then don't use it. Modula-3 doesn't have lots of things that C++ does have. That is deliberate (the green book even talks about this specific issue in some detail). Those of us that use M3 know that it lacks those things, and we use the language in spite of these things missing----or maybe BECAUSE those things are missing. To quote the Modula-3 report: 'In the early days of the Ada project, a general in the Ada Program Office opined that "obviously the Department of Defense is not interested in an artificially simplified language such as Pascal". Modula-3 represents the opposite point of view. We used every artifice that we could find or invent to make the language simple.' Yes, there are simplifying artifices. Sometimes they seem clunky for someone coming from C++, or Python, or Java. They are mostly there for a reason. Trying to get rid of those things misses the whole point of what Modula-3 is about. I'm with Rodney there. At the same time, I think that implementing some things in C isn't always a bad idea. Let me try to explain what I mean. I wrote in an earlier email (and gave an example) of how to take some C code that was written in an "interface-based paradigm" (which is really a programming approach that historically came out of Modula-2, as far as I know) and interfacing to that in M3. It's easy, IF the C is written "properly". It is a fact of life that Unix and many other OSes use C for their system-call interface. It is another fact of life that a lot of old C interfaces are NOT written in an exemplary interface-based style. Those interfaces are therefore a lot more difficult to attach to from M3. It makes sense to me to write glue code in C that cleans up the C interface in such a way that M3 can more easily attach, for example without cloning the headers. (I share Jay's worries about manually cloning C headers in M3.) Mika On Tue, Mar 9, 2021 at 2:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 01:57:58 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 00:57:58 +0000 Subject: [M3devel] Compiler tests In-Reply-To: References: <6d34df9f-57a8-06c1-e3b9-1b34731267c9@lcwb.coop> , , Message-ID: Darko, Please try again with master? You have to update m3core and libm3, and then update.py. Assertion failures now include the text of the failed expression. So it should be a little clearer what is happening. The entire system compiles with the C backend, and has for many years, modulo some repaired bitrot, so this is surprising (maybe except for Grisu, I have to look at that). - Jay ________________________________ From: Jay K Sent: Tuesday, March 2, 2021 10:55:08 PM To: Jay ; Darko Volaric Cc: m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Compiler tests Just pasting that enum into code, works. The line of the assert..cannot easily be matched up. I'll add more printing before some asserts. Maybe try with current? Or in debugger? Or provide a more complete source? Certainly it looks trivial. Thank you, - Jay ________________________________ From: Jay Sent: Wednesday, March 3, 2021 3:39 AM To: Darko Volaric Cc: Jay K ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Compiler tests Oh wow, thank you. Please add this to the tree if you can. I will look soon, and I will improve the diagnostics at least a little. Again, it does compile the entire tree. Something is ?unique? here. Maybe reusing an enum member?? - Jay On Tue, Mar 2, 2021 at 1:56 PM Darko Volaric > wrote: It definitely still has problems, I get *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/M3C.m3", line 2523 *** when compiling an interface at the declaration TYPE ExAtmX = { Nil, Sort, Inst, Bin, Str, Bool, Int, Real, Any, LocalLink, FeatLink, SortLink, RowLink, Stream, FeatAdr, Result }; with the last C file output being /*enum_define*/typedef UINT8 T2DE88987; /*declare_enum*/ #define T2DE88987_Nil ((UINT8)0) /*declare_enum_elt*/ #define T2DE88987_Sort ((UINT8)1) /*declare_enum_elt*/ #define T2DE88987_Inst ((UINT8)2) /*declare_enum_elt*/ #define T2DE88987_Bin ((UINT8)3) /*declare_enum_elt*/ #define T2DE88987_Str ((UINT8)4) /*declare_enum_elt*/ #define T2DE88987_Bool ((UINT8)5) /*declare_enum_elt*/ #define T2DE88987_Int ((UINT8)6) /*declare_enum_elt*/ #define T2DE88987_Real ((UINT8)7) /*declare_enum_elt*/ #define T2DE88987_Any ((UINT8)8) /*declare_enum_elt*/ #define T2DE88987_LocalLink ((UINT8)9) /*declare_enum_elt*/ #define T2DE88987_FeatLink ((UINT8)10) /*declare_enum_elt*/ #define T2DE88987_SortLink ((UINT8)11) /*declare_enum_elt*/ with Critical Mass Modula-3 version d5.10.0 last updated: 2016-10-05 compiled: 2021-02-26 09:11:48 configuration: /Users/darko/Apps/cm3/bin/cm3.cfg host: AMD64_DARWIN target: AMD64_DARWIN On Tue, Mar 2, 2021 at 7:39 PM Jay K > wrote: No. You are right. I should. I will try to soon. I don't remember if I ran them years ago when I wrote the C backend, but I have not been running them recently. That said, it does all work pretty well. The system does build itself completely, and debuggability exceeds that of the integrated backend and I think Darwin. ? - Jay ________________________________ From: Rodney M. Bates > Sent: Friday, February 26, 2021 6:40 PM To: m3devel >; Jay > Subject: Compiler tests Jay, you have mentioned a couple of times suspecting a bug in the C backend, as well as others. Are you periodically running the scripted compiler tests? They can help a lot. -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Mar 10 03:57:53 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 9 Mar 2021 20:57:53 -0600 Subject: [M3devel] more win64 subrange problems uint32 := -1. In-Reply-To: References: Message-ID: <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop> On 3/9/21 1:30 PM, Jay K wrote: > Searching for 'lockfile'... > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): ? ?IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): ? ?IF WinBase.UnlockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): ? ?IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): ? ?IF WinBase.UnlockFile( > 4 occurrence(s) have been found. > > > ? ? IF WinBase.LockFile( > ? ? ? ? ? ?hFile := h.handle, > ? ? ? ? ? ?dwFileOffsetLow := 0, > ? ? ? ? ? ?dwFileOffsetHigh := 0, > ? ? ? ? ? ?nNumberOfBytesToLockLow := -1, > ? ? ? ? ? ?nNumberOfBytesToLockHigh := -1) = 0 THEN > ? ? ? err := WinBase.GetLastError(); > > > PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= > ? BEGIN > ? ? IF WinBase.UnlockFile( > ? ? ? ? ? ?hFile := h.handle, > ? ? ? ? ? ?dwFileOffsetLow := 0, > ? ? ? ? ? ?dwFileOffsetHigh := 0, > ? ? ? ? ? ?nNumberOfBytesToUnlockLow := -1, > ? ? ? ? ? ?nNumberOfBytesToUnlockHigh := -1) = 0 THEN > ? ? ? OSErrorWin32.Raise() > ? ? END; > ? END RegularFileUnlock; > > <*EXTERNAL LockFile:WINAPI*> > PROCEDURE LockFile (hFile ? ? ? ? ? ? ? ? ? : HANDLE; > ? ? ? ? ? ? ? ? ? ? dwFileOffsetLow ? ? ? ? : UINT32; > ? ? ? ? ? ? ? ? ? ? dwFileOffsetHigh ? ? ? ?: UINT32; > ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToLockLow : UINT32; > ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToLockHigh: UINT32 ? ): BOOL; > > <*EXTERNAL UnlockFile:WINAPI*> > PROCEDURE UnlockFile (hFile ? ? ? ? ? ? ? ? ? ? : HANDLE; > ? ? ? ? ? ? ? ? ? ? ? dwFileOffsetLow ? ? ? ? ? : UINT32; > ? ? ? ? ? ? ? ? ? ? ? dwFileOffsetHigh ? ? ? ? ?: UINT32; > ? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToUnlockLow : UINT32; > ? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToUnlockHigh: UINT32 ? ): BOOL; > > > These get warnings when compiling. But only for Win64, not Win32. > I'm not sure if they fail at runtime or not. Makes perfect sense. You appear to be trying to pass a negative number, but the parameter type is UINT32, an unsigned type that includes no negative values. On 64-bits, -1 is well within the range of INTEGER and really is a negative value, not in the range of UINT32 (whose lower bound is zero) and thus the warning. If you execute this call with -1, it will also get a runtime range error. On 32-bits, UINT32 (and INT32 too) are full-range native words. The difference is only in whether the code treats the bits as signed twos-complement, or unsigned binary. It can be either, but all places must be consistent, and in this case, you won't be changing the Windows system call, so that will control which it is. On 32-bits, the bit pattern -1 = 16_FFFFFFFF is a negative number if treated as signed, but it's the maximum 32-bit unsigned (positive) value, if treated as unsigned. I.e., it's 2^32-1. If you want a signed value in a 32-bit word, you can only get up to 2^31-1. No warning or error here, because the range covers all of a machine word and every bit pattern is in it. But I greatly doubt you will really want to lock 2^32-1 bytes--the entire maximum addressable RAM size. No error messages, but no doubt a big bug. The external LockFile:WINAPI will have to treat bit patterns in 2^31..2^32-1 as either negative, or large positive. They can't be ambiguous. So you need to look at the definition of LockFile:WINAPI (I presume it's a C prototype) and see whether it really treats the parameters as signed or unsigned, and, if the former, adjust the type of the parameters to INT32. If the latter, don't pass any negative values. If it won't tell you this somehow, you have grounds to faunch. Then consistently make all the values you pass lie within this range. Don't try to pass a negative number to a UINT32 and don't try to pass anything > 2^31-1 to an INT32. Either way, you'll have trouble, sometimes with warnings and errors, sometimes with just bad behavior. Just intuitively from the formal names, I would guess zero bytes is the fewest you would want to lock, so zero is probably what you want. > > It isn't terrible pattern, I think, to reserve a few values like this. > Perhaps in this case, you might as well give up half the values though and use int32? > Well, on "high", but not on low". > Of course, taking an int64 and not two 32bit integers, would be preferable. > > I am inclined, as ever, to just move a few lines of Modula3 to C, > because I very much dislike the pattern of rewriting system C headers in other languages. > Though it perhaps it isn't too bad on Windows. > > ?- Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Wed Mar 10 04:06:47 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 03:06:47 +0000 Subject: [M3devel] more win64 subrange problems uint32 := -1. In-Reply-To: <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop> References: , <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop> Message-ID: It isn't me. It is the code we have been sitting on forever. Just that we have only been targeting Win32, not Win64. The two 32bit numbers combine to form a 64bit number. The lower 32bits are meant to be unsigned -- not sign extended. The signedness of the upper 32bits does not matter much, as 31 bits is enough in practise (63 bits overall). The overall value -1 is surely special, it is not a magnitude but a magic value. You could likely think of this as a union of a boolean and an almost 64bit number. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 10, 2021 2:57 AM To: Jay K ; m3devel Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. On 3/9/21 1:30 PM, Jay K wrote: > Searching for 'lockfile'... > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): IF WinBase.UnlockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): IF WinBase.UnlockFile( > 4 occurrence(s) have been found. > > > IF WinBase.LockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToLockLow := -1, > nNumberOfBytesToLockHigh := -1) = 0 THEN > err := WinBase.GetLastError(); > > > PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= > BEGIN > IF WinBase.UnlockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToUnlockLow := -1, > nNumberOfBytesToUnlockHigh := -1) = 0 THEN > OSErrorWin32.Raise() > END; > END RegularFileUnlock; > > <*EXTERNAL LockFile:WINAPI*> > PROCEDURE LockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToLockLow : UINT32; > nNumberOfBytesToLockHigh: UINT32 ): BOOL; > > <*EXTERNAL UnlockFile:WINAPI*> > PROCEDURE UnlockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToUnlockLow : UINT32; > nNumberOfBytesToUnlockHigh: UINT32 ): BOOL; > > > These get warnings when compiling. But only for Win64, not Win32. > I'm not sure if they fail at runtime or not. Makes perfect sense. You appear to be trying to pass a negative number, but the parameter type is UINT32, an unsigned type that includes no negative values. On 64-bits, -1 is well within the range of INTEGER and really is a negative value, not in the range of UINT32 (whose lower bound is zero) and thus the warning. If you execute this call with -1, it will also get a runtime range error. On 32-bits, UINT32 (and INT32 too) are full-range native words. The difference is only in whether the code treats the bits as signed twos-complement, or unsigned binary. It can be either, but all places must be consistent, and in this case, you won't be changing the Windows system call, so that will control which it is. On 32-bits, the bit pattern -1 = 16_FFFFFFFF is a negative number if treated as signed, but it's the maximum 32-bit unsigned (positive) value, if treated as unsigned. I.e., it's 2^32-1. If you want a signed value in a 32-bit word, you can only get up to 2^31-1. No warning or error here, because the range covers all of a machine word and every bit pattern is in it. But I greatly doubt you will really want to lock 2^32-1 bytes--the entire maximum addressable RAM size. No error messages, but no doubt a big bug. The external LockFile:WINAPI will have to treat bit patterns in 2^31..2^32-1 as either negative, or large positive. They can't be ambiguous. So you need to look at the definition of LockFile:WINAPI (I presume it's a C prototype) and see whether it really treats the parameters as signed or unsigned, and, if the former, adjust the type of the parameters to INT32. If the latter, don't pass any negative values. If it won't tell you this somehow, you have grounds to faunch. Then consistently make all the values you pass lie within this range. Don't try to pass a negative number to a UINT32 and don't try to pass anything > 2^31-1 to an INT32. Either way, you'll have trouble, sometimes with warnings and errors, sometimes with just bad behavior. Just intuitively from the formal names, I would guess zero bytes is the fewest you would want to lock, so zero is probably what you want. > > It isn't terrible pattern, I think, to reserve a few values like this. > Perhaps in this case, you might as well give up half the values though and use int32? > Well, on "high", but not on low". > Of course, taking an int64 and not two 32bit integers, would be preferable. > > I am inclined, as ever, to just move a few lines of Modula3 to C, > because I very much dislike the pattern of rewriting system C headers in other languages. > Though it perhaps it isn't too bad on Windows. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C889fa88893b84f2234d608d8e37054c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637509418889721122%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=37cIsIlpH9vggIpls4yWTCwQrtkJFXU1EDbU3%2FZi7ZE%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 04:10:12 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 03:10:12 +0000 Subject: [M3devel] more win64 subrange problems uint32 := -1. In-Reply-To: References: , <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop>, Message-ID: Actually this could be a way of locking the entire file, no matter its present or future size. The documentation says both are unsigned, so the Modula-3 .i3 files match the C .h files. -1 is a common shorthand for "maximum", despite being, you know, a small negative number. ~0u can also be used. -1 is arguably better, because it probably expands to 64bit maximum in context. Despite the range violation. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Wednesday, March 10, 2021 3:06 AM To: m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. It isn't me. It is the code we have been sitting on forever. Just that we have only been targeting Win32, not Win64. The two 32bit numbers combine to form a 64bit number. The lower 32bits are meant to be unsigned -- not sign extended. The signedness of the upper 32bits does not matter much, as 31 bits is enough in practise (63 bits overall). The overall value -1 is surely special, it is not a magnitude but a magic value. You could likely think of this as a union of a boolean and an almost 64bit number. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 10, 2021 2:57 AM To: Jay K ; m3devel Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. On 3/9/21 1:30 PM, Jay K wrote: > Searching for 'lockfile'... > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): IF WinBase.UnlockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): IF WinBase.UnlockFile( > 4 occurrence(s) have been found. > > > IF WinBase.LockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToLockLow := -1, > nNumberOfBytesToLockHigh := -1) = 0 THEN > err := WinBase.GetLastError(); > > > PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= > BEGIN > IF WinBase.UnlockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToUnlockLow := -1, > nNumberOfBytesToUnlockHigh := -1) = 0 THEN > OSErrorWin32.Raise() > END; > END RegularFileUnlock; > > <*EXTERNAL LockFile:WINAPI*> > PROCEDURE LockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToLockLow : UINT32; > nNumberOfBytesToLockHigh: UINT32 ): BOOL; > > <*EXTERNAL UnlockFile:WINAPI*> > PROCEDURE UnlockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToUnlockLow : UINT32; > nNumberOfBytesToUnlockHigh: UINT32 ): BOOL; > > > These get warnings when compiling. But only for Win64, not Win32. > I'm not sure if they fail at runtime or not. Makes perfect sense. You appear to be trying to pass a negative number, but the parameter type is UINT32, an unsigned type that includes no negative values. On 64-bits, -1 is well within the range of INTEGER and really is a negative value, not in the range of UINT32 (whose lower bound is zero) and thus the warning. If you execute this call with -1, it will also get a runtime range error. On 32-bits, UINT32 (and INT32 too) are full-range native words. The difference is only in whether the code treats the bits as signed twos-complement, or unsigned binary. It can be either, but all places must be consistent, and in this case, you won't be changing the Windows system call, so that will control which it is. On 32-bits, the bit pattern -1 = 16_FFFFFFFF is a negative number if treated as signed, but it's the maximum 32-bit unsigned (positive) value, if treated as unsigned. I.e., it's 2^32-1. If you want a signed value in a 32-bit word, you can only get up to 2^31-1. No warning or error here, because the range covers all of a machine word and every bit pattern is in it. But I greatly doubt you will really want to lock 2^32-1 bytes--the entire maximum addressable RAM size. No error messages, but no doubt a big bug. The external LockFile:WINAPI will have to treat bit patterns in 2^31..2^32-1 as either negative, or large positive. They can't be ambiguous. So you need to look at the definition of LockFile:WINAPI (I presume it's a C prototype) and see whether it really treats the parameters as signed or unsigned, and, if the former, adjust the type of the parameters to INT32. If the latter, don't pass any negative values. If it won't tell you this somehow, you have grounds to faunch. Then consistently make all the values you pass lie within this range. Don't try to pass a negative number to a UINT32 and don't try to pass anything > 2^31-1 to an INT32. Either way, you'll have trouble, sometimes with warnings and errors, sometimes with just bad behavior. Just intuitively from the formal names, I would guess zero bytes is the fewest you would want to lock, so zero is probably what you want. > > It isn't terrible pattern, I think, to reserve a few values like this. > Perhaps in this case, you might as well give up half the values though and use int32? > Well, on "high", but not on low". > Of course, taking an int64 and not two 32bit integers, would be preferable. > > I am inclined, as ever, to just move a few lines of Modula3 to C, > because I very much dislike the pattern of rewriting system C headers in other languages. > Though it perhaps it isn't too bad on Windows. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C889fa88893b84f2234d608d8e37054c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637509418889721122%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=37cIsIlpH9vggIpls4yWTCwQrtkJFXU1EDbU3%2FZi7ZE%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 04:38:14 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 03:38:14 +0000 Subject: [M3devel] more win64 subrange problems uint32 := -1. In-Reply-To: References: , <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop>, , Message-ID: Ok docs: ?Locking a region that goes beyond the current end-of-file position is not an error.? So it should be 16_FFFFFFFF? Which is bit harder to read and write but ok. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Tuesday, March 9, 2021 7:10:12 PM To: m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. Actually this could be a way of locking the entire file, no matter its present or future size. The documentation says both are unsigned, so the Modula-3 .i3 files match the C .h files. -1 is a common shorthand for "maximum", despite being, you know, a small negative number. ~0u can also be used. -1 is arguably better, because it probably expands to 64bit maximum in context. Despite the range violation. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Wednesday, March 10, 2021 3:06 AM To: m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. It isn't me. It is the code we have been sitting on forever. Just that we have only been targeting Win32, not Win64. The two 32bit numbers combine to form a 64bit number. The lower 32bits are meant to be unsigned -- not sign extended. The signedness of the upper 32bits does not matter much, as 31 bits is enough in practise (63 bits overall). The overall value -1 is surely special, it is not a magnitude but a magic value. You could likely think of this as a union of a boolean and an almost 64bit number. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 10, 2021 2:57 AM To: Jay K ; m3devel Subject: Re: [M3devel] more win64 subrange problems uint32 := -1. On 3/9/21 1:30 PM, Jay K wrote: > Searching for 'lockfile'... > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): IF WinBase.UnlockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): IF WinBase.LockFile( > C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): IF WinBase.UnlockFile( > 4 occurrence(s) have been found. > > > IF WinBase.LockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToLockLow := -1, > nNumberOfBytesToLockHigh := -1) = 0 THEN > err := WinBase.GetLastError(); > > > PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= > BEGIN > IF WinBase.UnlockFile( > hFile := h.handle, > dwFileOffsetLow := 0, > dwFileOffsetHigh := 0, > nNumberOfBytesToUnlockLow := -1, > nNumberOfBytesToUnlockHigh := -1) = 0 THEN > OSErrorWin32.Raise() > END; > END RegularFileUnlock; > > <*EXTERNAL LockFile:WINAPI*> > PROCEDURE LockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToLockLow : UINT32; > nNumberOfBytesToLockHigh: UINT32 ): BOOL; > > <*EXTERNAL UnlockFile:WINAPI*> > PROCEDURE UnlockFile (hFile : HANDLE; > dwFileOffsetLow : UINT32; > dwFileOffsetHigh : UINT32; > nNumberOfBytesToUnlockLow : UINT32; > nNumberOfBytesToUnlockHigh: UINT32 ): BOOL; > > > These get warnings when compiling. But only for Win64, not Win32. > I'm not sure if they fail at runtime or not. Makes perfect sense. You appear to be trying to pass a negative number, but the parameter type is UINT32, an unsigned type that includes no negative values. On 64-bits, -1 is well within the range of INTEGER and really is a negative value, not in the range of UINT32 (whose lower bound is zero) and thus the warning. If you execute this call with -1, it will also get a runtime range error. On 32-bits, UINT32 (and INT32 too) are full-range native words. The difference is only in whether the code treats the bits as signed twos-complement, or unsigned binary. It can be either, but all places must be consistent, and in this case, you won't be changing the Windows system call, so that will control which it is. On 32-bits, the bit pattern -1 = 16_FFFFFFFF is a negative number if treated as signed, but it's the maximum 32-bit unsigned (positive) value, if treated as unsigned. I.e., it's 2^32-1. If you want a signed value in a 32-bit word, you can only get up to 2^31-1. No warning or error here, because the range covers all of a machine word and every bit pattern is in it. But I greatly doubt you will really want to lock 2^32-1 bytes--the entire maximum addressable RAM size. No error messages, but no doubt a big bug. The external LockFile:WINAPI will have to treat bit patterns in 2^31..2^32-1 as either negative, or large positive. They can't be ambiguous. So you need to look at the definition of LockFile:WINAPI (I presume it's a C prototype) and see whether it really treats the parameters as signed or unsigned, and, if the former, adjust the type of the parameters to INT32. If the latter, don't pass any negative values. If it won't tell you this somehow, you have grounds to faunch. Then consistently make all the values you pass lie within this range. Don't try to pass a negative number to a UINT32 and don't try to pass anything > 2^31-1 to an INT32. Either way, you'll have trouble, sometimes with warnings and errors, sometimes with just bad behavior. Just intuitively from the formal names, I would guess zero bytes is the fewest you would want to lock, so zero is probably what you want. > > It isn't terrible pattern, I think, to reserve a few values like this. > Perhaps in this case, you might as well give up half the values though and use int32? > Well, on "high", but not on low". > Of course, taking an int64 and not two 32bit integers, would be preferable. > > I am inclined, as ever, to just move a few lines of Modula3 to C, > because I very much dislike the pattern of rewriting system C headers in other languages. > Though it perhaps it isn't too bad on Windows. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C889fa88893b84f2234d608d8e37054c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637509418889721122%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=37cIsIlpH9vggIpls4yWTCwQrtkJFXU1EDbU3%2FZi7ZE%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Wed Mar 10 05:55:33 2021 From: lists at darko.org (Darko Volaric) Date: Wed, 10 Mar 2021 05:55:33 +0100 Subject: [M3devel] closures are a problem In-Reply-To: References: Message-ID: I couldn't agree more with this sentiment and the details. I don't know what your interest in M3 is Jay if you're constantly railing against the basic tenets of the language and lamenting it doesn't have the characteristics of C, or is somehow inconvenient in comparison at least. M3 is a great language (to those who like it) for most of the reasons you complain about it. No, it doesn't fit into the C style hegemony, it's from a different branch of language design that is in many ways ideologically opposed to C. I'm sure no one is opposed to you hacking or experimenting but you need to delineate those things from the existing design lest you do irreparable damage. There's nothing stopping you from limiting what your C back end accepts but changing the core of the compiler or language to suit your tastes is wrong. Elsewhere removing functionality because you want to "simplify" things and you don't see where it's used is also destructive. There is lots of code outside this source tree, people use this compiler for their work with substantial code bases. On Tue, Mar 9, 2021 at 11:14 PM Rodney M. Bates wrote: > Jay, I have become very fed up with your constant attempts and > sometimes success at undermining Modula-3. Passing nested procedures > as parameters is part of the language. It has been there from the > beginning. It is used, to good effect, in many places in existing > code. The other back ends all implement it, and it works. Please > stop proposing to dumb down the language, just because you find things > too difficult to implement or don' t want to. > > Many other languages provide this capability too, differing only in > the way they prevent the chaotic unsafety of references to > non-existing environments. Even extended GNU C provides the > capability, although its only method of addressing missing > environments is to tell the programmer not to do it, or chaos will > ensue. > > Aside from crippling the language, downgrading the implementation by > replacing Modula-3 code with C or C++ is just another variant of the > same thing. We should be doing the opposite. There is very very > little that needs to be coded in C/C++. Wrapping different > target-dependent sections of code in #ifdefs in the same file is, at > best, no cleaner or more elegant than putting them in separate files. > Calling the former a single code base while claiming the latter is not > is just verbal voodoo. There is different code for different targets > either way. > > If you don't like to implement Modula-3, chose another language to > work on. If you don't like to code in Modula-3, choose a project > coded in another language. If you are finding that using C as an > intermediate language makes things more difficult or more unpleasant > than you want, well, as I recall, you had ample warning before you > started, from some of us who have gotten run around that flag pole in > the past. > > If your method of dealing with the implementation problems that > confront you is to define away requirements, then it casts doubt on > the wisdom of the whole C backend approach, both from the perspective > of the implementation itself, or of just what you really want to do. > > As for values of procedure type, do it the same way it's already done. > If C is as fine for low-level code as you keep telling us, this will > be straightforward. > > > On 3/9/21 1:11 AM, Jay K wrote: > > Closures are a problem. > > > > Specifically the ability to take the address of a nested function > > and pass that as a function pointer, and that it has the same > > type as a pointer to a non-nested or C function, and it can access > > the locals of the lexical enclosure (arbitrary levels deep, with > recursion, > > but this doesn't make it worse). > > > > The problems: > > > > - The assumption that -1 is not valid code. > > (This value could be changed per system). > > > > - The assumption that code pages are readable. > > Some systems have execute-only memory. > > Reading works on wasm but I don't know what it does, wierd. > > > > - The assumption that reading code pages are meaningful. (wasm?) > > > > - The inability to pass these pointers to C code as function pointers. > > > > - I believe...sorry, I should know better, but I believe in the C > backend > > I play lose with the signatures of functions, passing an extra > parameter always, > > to functions that might not expect it. > > This likely plays poorly with some environments, such as > authenticated ARM function pointers > > and WebAssembly. > > > > WebAssembly might eventually get a feature to help here. > > > > So, really, I wish this feature was not in the language. > > Barring that, maybe we can remove its use from the base system? > > Maybe have an optional flag or two: > > - prohibit taking address of nested function > > - assume function pointers are not closures > > > > I realize these are different. In that, the second idea in a lower level > > library, breaks use with a higher level library. > > > > I know about several approaches to solve this problem, but none of them > are good. > > > > Traditional gcc generates code on the stack. > > Apple removed this from their compilers years ago. > > > > In some systems you can stamp out identical copies of code, with > associated data, and > > manage that like a cheap heap. But I think that still is not worth while. > > It still is not cheap, and it is platform specific. > > Libffi and Mono use this on iPhone for example. > > Libffi will be using it for all systems, at least all Linux systems. > > > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > > > In normal C code this is handled in a reasonable way, case by case. > > > > For example the qsort_s function (not regular qsort) takes function > pointer and a void*. > > The void* is extra context agreed upon between the qsort caller and the > function pointer. > > qsort does not know what it is. It can be a pointer to a struct. It can > be an integer. It can be ignored. > > There is casting require, but it is ok. It isn't "technically safe", but > it is easy to get correct. > > > > This achieves the same result as a "closure". > > > > Occasionally functions, like regular qsort, "forgot" to take the extra > void*, and this can be problem indeed. > > > > I'll try to find where in the system we use closures and if they can be > "fixed" somehow. > > I would like to soon-ish see if about compiling the system to > WebAssembly, via C, and running it in node.js. > > The C output already mostly can compile to WebAssembly, not having done > anything. > > > > I think Oberon removed this feature, for example. I just glanced at its > specification and it seemed to disallow it. > > > > Thoughts? > > > > - Jay > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 08:03:44 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 07:03:44 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: Let me ask a different way. Or different repetition/emphasis. Is anyone interested in running Modula-3 code on Android? Maybe, maybe not. It will not quite work, on current systems. The limitation is not the C backend. A current gcc or LLVM backend would have the same problem. m3front issues read through code pointers. See the function is_closure. Some Android devices have unreadable executable memory. If you issue such read something bad happens, like a SIGSEGV or the process is killed or something. It is for security. Android is not unique in having this feature. Maybe maybe there are workarounds. For example, Android mainly runs on ARM64. ARM64 instructions are 4-byte aligned. Modula-3 closures could have low bits set to indicate their presence. I think. https://source.android.com/devices/tech/debug/execute-only-memory Hm, well, they added it, and then they changed it back, and the explanation is unclear to me. Perhaps it will come back again later? Or on other devices? Perhaps it is running on some devices. But there are other systems with unreadable code pages and this would not work there. It is perhaps a sign of what future systems will look like. Cross the bridge earlier or later? Is anyone interested in running Modula-3 code as WebAssembly? Such as in a browser or in node.js/deno? With high performance JTs and such; and "portable executable". "high performance" I mean, accepting a surprisingly low level "bytecode" that is *not* limited to "like" Java and C#, but is actually meant to run C, C++, and Rust. Again, maybe, maybe not. Here I believe there is a different but related set of problems, and the blame is murkier. WebAssembly has certain kind of "security". Honestly I'm not sure of the details. It isn't "safe" in the ways we used to think. You can overflow local arrays, but you cannot overflow to hit the return address or to corrupt "host" data. Kinda like how kernels are safe from user space, perhaps. I believe code and data live in separate address spaces, essentially. So you can read a code pointer, landing you in random data. But if it works, I suppose the -1 marker remains rare, and if you actually see -1, that is actually your data pointer, the closure. So maybe that part "works". Though the -1 business needs more investigation. Maybe a larger marker and higher alignment factor should be used. WebAssembly also requires the caller and callee know the callee function signature accurately. You cannot do this: void f1(int); void f2() { f1(1,2); } You get a runtime error. While this structure seems wierd and useless, it also generally works in traditional systems, and I believe I used this in the C backend quite profitably. 'cause f1 might actually expect two integers, the caller is not sure. I need to review my own work here, granted. So, what if the answers were both yes? What if, there was a flag in the config file that said "no closures"? Upon which, you'd get a compiler error, if you took the address of a nested function. Nested functions still work, when called directly. Just not taking their address. And then, to whatever extent closures are used in the cm3 tree, we try to transform them away, hopefully producing reasonable idiomatic correct Modula-3? For example, one transform would be, I think, pass an enum and use a switch, and in the cases turnaround and call the same nested functions directoy. This works sometimes, when the pointer values are bounded. Does anyone want to run Modula-3 code on an Apple Watch? Maybe, maybe not. To run on an Apple Watch, you have submit high level LLVM IR. By "high level" I mean, you cannot just have an array of bytes and jump to it. Which on most systems, you can do -- possibly mprotect or VirtualProtect or compiler directive to make the bytes executable. A route to producing LLVM IR is, you gussed it, C. (Yeah sure, or directly, I understand.) However, it is more restrictive than this. With Apple Watch, Apple achieved a long term "holy grail" of software. You submit your LLVM IR to them. They compile it. For whatever processor they chose. The "low levelness" of the IR is just that it targets a 32bit little endian system. The current Apple Watch hardware is ARM64 and cannot run the 32bit instruction sets. The earlier generation ran only 32bit. They just compile for whatever, and the apps are limited to a 32bit address space. Or maybe it is 64bit. The theory works either way, really, and they put it into practise. They just slightly subsetted C++ and it is enough that "static layout" (struct offsets) and "internal static binding" (function calls do not require pointers) still work.. it is extremely close to a traditional system. But for this to work, C/C++ had to be constrained just a bit. There is no make/get/set/swapcontext. There is no setjmp/longjmp. So for example, a C++ backend could use try/throw/except and destructors (for finally) instead of setjmp/longjmp. This would benefit many platforms also, but is more required for Apple Watch. But Apple Watch would also present problems for the garbage collector. Honestly, I don't see me working on this, but it is interesting, and maybe the system can be "redesigned" or parameterized to work there. In particular, I guess, you also cannot get processor context from a signal handler. You cannot setjmp to home the registers. So it is difficult to fetch registers. I eventually read the name of a common expensive portable technique -- "shadow stack". WebAssembly maybe though. Now, about the current systems that work seemingly just fine with this language feature. Have you noticed what m3frontdoes? This read of the function pointer and looking for -1 marker? Doesn't this seem, really, kinda gross? Who is to say that a function cannot start with the bytes 0xffffffff or 0xffffffffffffffff? It is up to each porter I guess to learn the instruction encoding and decide if this works, and settle on some other constant that is somehow guaranteed not to start a function? Have we just been maybe a little negligent, but it has been ok? Or did everyone check the meaning of 0xff... on x86, amd64, Sparc, PowerPC, Alpha, ARM, etc.? Am I just too much of a perfectionist? On amd64 at least, 0xff can start an instruction, but the sequence 0xff 0xff seems to be presently invalid. Should we just trust Intel never to define this encoding, or recompile if it changes? You know they constantly are defining previously undefined sequences. It is their business. And we are monitoring the Intel manuals for this evolution? Intel does define an "undefined" opcode, kinda reserved and kinda defined at the same time. Maybe use that? You could do something like, insert a jmp to a "constant" marker function. (through a relocated pointer, the code could verify at runtime, a bit expensively, granted) This is kinda getting somewhere, but then does require deeper per-target work. Perhaps worthwhile. Certainly wouldn't make retargeting WebAssembly easier. Please keep in mind, I am not talking about nested functions in general, but taking their address and passing it as a "callback" to other functions. Like how qsort/bsearch work. Surely the taking of address, while an actual thing, implemented, problematic, is not always done for all nested functions. - Jay ________________________________ From: Darko Volaric Sent: Wednesday, March 10, 2021 4:55 AM To: rodney.m.bates at acm.org Cc: Jay K ; m3devel Subject: Re: [M3devel] closures are a problem I couldn't agree more with this sentiment and the details. I don't know what your interest in M3 is Jay if you're constantly railing against the basic tenets of the language and lamenting it doesn't have the characteristics of C, or is somehow inconvenient in comparison at least. M3 is a great language (to those who like it) for most of the reasons you complain about it. No, it doesn't fit into the C style hegemony, it's from a different branch of language design that is in many ways ideologically opposed to C. I'm sure no one is opposed to you hacking or experimenting but you need to delineate those things from the existing design lest you do irreparable damage. There's nothing stopping you from limiting what your C back end accepts but changing the core of the compiler or language to suit your tastes is wrong. Elsewhere removing functionality because you want to "simplify" things and you don't see where it's used is also destructive. There is lots of code outside this source tree, people use this compiler for their work with substantial code bases. On Tue, Mar 9, 2021 at 11:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 11:43:02 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 10:43:02 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: , Message-ID: You don't have to convince me to avoid globals or heap allocation. ? I was working on something else (factoring target out of C output) and noticed this too: PROCEDURE BuildGeneric (t: T; kind, file, name, generic: TEXT; args: QVSeq.T) RAISES {Quake.Error} = CONST tmp = ".generic.tmp"; VAR txts: TextVector; PROCEDURE Emit (wr: Wr.T) RAISES {Wr.Failure, Thread.Alerted} = VAR comma: TEXT := NIL; BEGIN Out (wr, "(*generated by m3build*)", Wr.EOL, Wr.EOL); Out (wr, kind, " ", name, " = "); Out (wr, generic, " ("); FOR i := 0 TO LAST (txts^) DO Out (wr, comma, txts[i]); comma := ", "; END; Out (wr, ") END ", name, ".", Wr.EOL); END Emit; BEGIN IF (t.mode = MM.Build) THEN txts := SeqToTexts (t, args); Utils.WriteFile (tmp, Emit, append := FALSE); t.cp_if (tmp, file); Utils.Remove (tmp); END; END BuildGeneric; which is passing Emit to WriteFile and Emit uses outer locals. So this is something I would probably like to "fix" somehow, once I confirm it is a problem for WebAssembly. At least subject to a flag and fix the in-tree code. You can still keep using the feature. - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, March 10, 2021 8:46 AM To: Jay K Cc: Darko Volaric ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] closures are a problem I'm certainly not running on Android or anything like that at the moment, but I think it could definitely be useful. The passing of a nested function as a parameter to another function is *definitely* a useful thing to do. It allows you to do things with stack-allocated memory that you can't do otherwise. In these cases the passing of the nested function isn't just a convenience to avoid global variables but is a way of avoiding heap allocations, because it sort of allows you to allocate a variable amount of memory on the stack without using alloca() or similar. I have an example that I call "upwards recursion": you have an ordinary recursive procedure that at every point in its recursion passes an auxiliary function downwards, that auxiliary function allows access to the parent stack frame, but the child in the recursion can call the auxiliary, and the auxiliary calls the auxiliary in its own stack frame: so that: a calls b calls c calls d calls e (decides to call e.f calls d.f calls c.f calls b.f calls a.f). This way the recursion can proceed without allocating any heap memory until you get to the one special case where you care, then you use the upward recursion to figure out what you need on the heap. If the few times you need to call the "upward chain" are few relative to the total number of recursions, the average amount of heap memory allocated is infinitesimal. Used for recursive searches of various kinds that have an uncommon base case that's moderately expensive. For example imagine searching through a large database for a specific match and then doing something when found, something that requires a basically static amount of information per recursion depth. It can be a very efficient approach. (Example use: searching for a node in a large hierarchical circuit description, then printing that node's fully qualified (dotted) name.) Mika P.S. code example PROCEDURE FindCanonicalNames(longNames: LongNames.T; rootType : CellRec.T; maxDepth : CARDINAL) : AtomTextTbl.T = (* when this procedure is run, MarkMinDepth must have been run, so that all the .aux fields contain the type's min depth *) VAR res := NEW(AtomTextTbl.Default).init(); PROCEDURE Recurse(type : CellRec.T; depth : CARDINAL; f : Formatter) = VAR dummy : TEXT; BEGIN IF depth = type.aux AND NOT res.get(type.nm, dummy) THEN (* <==== upward recursion triggered <====== *) (* ok we have a new canon. name. *) WITH canon = f() DO IF doDebug THEN Debug.Out(F("canonical name of %s is %s", Atom.ToText(type.nm), canon)) END; EVAL res.put(type.nm, canon) END END; IF depth # maxDepth THEN FOR i := FIRST(type.subcells^) TO LAST(type.subcells^) DO (* using this nested procedure approach defers printing to the last possible moment AND stores the necessary linkage on the stack! *) PROCEDURE Formatter() : TEXT = BEGIN RETURN f() & "." & LongNames.DecodeToText(longNames, instance) END Formatter; VAR instance := type.subcells[i].instance; BEGIN Recurse(type.subcells[i].type, depth + 1, Formatter) END END END END Recurse; On Tue, Mar 9, 2021 at 11:04 PM Jay K > wrote: Let me ask a different way. Or different repetition/emphasis. Is anyone interested in running Modula-3 code on Android? Maybe, maybe not. It will not quite work, on current systems. The limitation is not the C backend. A current gcc or LLVM backend would have the same problem. m3front issues read through code pointers. See the function is_closure. Some Android devices have unreadable executable memory. If you issue such read something bad happens, like a SIGSEGV or the process is killed or something. It is for security. Android is not unique in having this feature. Maybe maybe there are workarounds. For example, Android mainly runs on ARM64. ARM64 instructions are 4-byte aligned. Modula-3 closures could have low bits set to indicate their presence. I think. https://source.android.com/devices/tech/debug/execute-only-memory Hm, well, they added it, and then they changed it back, and the explanation is unclear to me. Perhaps it will come back again later? Or on other devices? Perhaps it is running on some devices. But there are other systems with unreadable code pages and this would not work there. It is perhaps a sign of what future systems will look like. Cross the bridge earlier or later? Is anyone interested in running Modula-3 code as WebAssembly? Such as in a browser or in node.js/deno? With high performance JTs and such; and "portable executable". "high performance" I mean, accepting a surprisingly low level "bytecode" that is *not* limited to "like" Java and C#, but is actually meant to run C, C++, and Rust. Again, maybe, maybe not. Here I believe there is a different but related set of problems, and the blame is murkier. WebAssembly has certain kind of "security". Honestly I'm not sure of the details. It isn't "safe" in the ways we used to think. You can overflow local arrays, but you cannot overflow to hit the return address or to corrupt "host" data. Kinda like how kernels are safe from user space, perhaps. I believe code and data live in separate address spaces, essentially. So you can read a code pointer, landing you in random data. But if it works, I suppose the -1 marker remains rare, and if you actually see -1, that is actually your data pointer, the closure. So maybe that part "works". Though the -1 business needs more investigation. Maybe a larger marker and higher alignment factor should be used. WebAssembly also requires the caller and callee know the callee function signature accurately. You cannot do this: void f1(int); void f2() { f1(1,2); } You get a runtime error. While this structure seems wierd and useless, it also generally works in traditional systems, and I believe I used this in the C backend quite profitably. 'cause f1 might actually expect two integers, the caller is not sure. I need to review my own work here, granted. So, what if the answers were both yes? What if, there was a flag in the config file that said "no closures"? Upon which, you'd get a compiler error, if you took the address of a nested function. Nested functions still work, when called directly. Just not taking their address. And then, to whatever extent closures are used in the cm3 tree, we try to transform them away, hopefully producing reasonable idiomatic correct Modula-3? For example, one transform would be, I think, pass an enum and use a switch, and in the cases turnaround and call the same nested functions directoy. This works sometimes, when the pointer values are bounded. Does anyone want to run Modula-3 code on an Apple Watch? Maybe, maybe not. To run on an Apple Watch, you have submit high level LLVM IR. By "high level" I mean, you cannot just have an array of bytes and jump to it. Which on most systems, you can do -- possibly mprotect or VirtualProtect or compiler directive to make the bytes executable. A route to producing LLVM IR is, you gussed it, C. (Yeah sure, or directly, I understand.) However, it is more restrictive than this. With Apple Watch, Apple achieved a long term "holy grail" of software. You submit your LLVM IR to them. They compile it. For whatever processor they chose. The "low levelness" of the IR is just that it targets a 32bit little endian system. The current Apple Watch hardware is ARM64 and cannot run the 32bit instruction sets. The earlier generation ran only 32bit. They just compile for whatever, and the apps are limited to a 32bit address space. Or maybe it is 64bit. The theory works either way, really, and they put it into practise. They just slightly subsetted C++ and it is enough that "static layout" (struct offsets) and "internal static binding" (function calls do not require pointers) still work.. it is extremely close to a traditional system. But for this to work, C/C++ had to be constrained just a bit. There is no make/get/set/swapcontext. There is no setjmp/longjmp. So for example, a C++ backend could use try/throw/except and destructors (for finally) instead of setjmp/longjmp. This would benefit many platforms also, but is more required for Apple Watch. But Apple Watch would also present problems for the garbage collector. Honestly, I don't see me working on this, but it is interesting, and maybe the system can be "redesigned" or parameterized to work there. In particular, I guess, you also cannot get processor context from a signal handler. You cannot setjmp to home the registers. So it is difficult to fetch registers. I eventually read the name of a common expensive portable technique -- "shadow stack". WebAssembly maybe though. Now, about the current systems that work seemingly just fine with this language feature. Have you noticed what m3frontdoes? This read of the function pointer and looking for -1 marker? Doesn't this seem, really, kinda gross? Who is to say that a function cannot start with the bytes 0xffffffff or 0xffffffffffffffff? It is up to each porter I guess to learn the instruction encoding and decide if this works, and settle on some other constant that is somehow guaranteed not to start a function? Have we just been maybe a little negligent, but it has been ok? Or did everyone check the meaning of 0xff... on x86, amd64, Sparc, PowerPC, Alpha, ARM, etc.? Am I just too much of a perfectionist? On amd64 at least, 0xff can start an instruction, but the sequence 0xff 0xff seems to be presently invalid. Should we just trust Intel never to define this encoding, or recompile if it changes? You know they constantly are defining previously undefined sequences. It is their business. And we are monitoring the Intel manuals for this evolution? Intel does define an "undefined" opcode, kinda reserved and kinda defined at the same time. Maybe use that? You could do something like, insert a jmp to a "constant" marker function. (through a relocated pointer, the code could verify at runtime, a bit expensively, granted) This is kinda getting somewhere, but then does require deeper per-target work. Perhaps worthwhile. Certainly wouldn't make retargeting WebAssembly easier. Please keep in mind, I am not talking about nested functions in general, but taking their address and passing it as a "callback" to other functions. Like how qsort/bsearch work. Surely the taking of address, while an actual thing, implemented, problematic, is not always done for all nested functions. - Jay ________________________________ From: Darko Volaric > Sent: Wednesday, March 10, 2021 4:55 AM To: rodney.m.bates at acm.org > Cc: Jay K >; m3devel > Subject: Re: [M3devel] closures are a problem I couldn't agree more with this sentiment and the details. I don't know what your interest in M3 is Jay if you're constantly railing against the basic tenets of the language and lamenting it doesn't have the characteristics of C, or is somehow inconvenient in comparison at least. M3 is a great language (to those who like it) for most of the reasons you complain about it. No, it doesn't fit into the C style hegemony, it's from a different branch of language design that is in many ways ideologically opposed to C. I'm sure no one is opposed to you hacking or experimenting but you need to delineate those things from the existing design lest you do irreparable damage. There's nothing stopping you from limiting what your C back end accepts but changing the core of the compiler or language to suit your tastes is wrong. Elsewhere removing functionality because you want to "simplify" things and you don't see where it's used is also destructive. There is lots of code outside this source tree, people use this compiler for their work with substantial code bases. On Tue, Mar 9, 2021 at 11:14 PM Rodney M. Bates > wrote: Jay, I have become very fed up with your constant attempts and sometimes success at undermining Modula-3. Passing nested procedures as parameters is part of the language. It has been there from the beginning. It is used, to good effect, in many places in existing code. The other back ends all implement it, and it works. Please stop proposing to dumb down the language, just because you find things too difficult to implement or don' t want to. Many other languages provide this capability too, differing only in the way they prevent the chaotic unsafety of references to non-existing environments. Even extended GNU C provides the capability, although its only method of addressing missing environments is to tell the programmer not to do it, or chaos will ensue. Aside from crippling the language, downgrading the implementation by replacing Modula-3 code with C or C++ is just another variant of the same thing. We should be doing the opposite. There is very very little that needs to be coded in C/C++. Wrapping different target-dependent sections of code in #ifdefs in the same file is, at best, no cleaner or more elegant than putting them in separate files. Calling the former a single code base while claiming the latter is not is just verbal voodoo. There is different code for different targets either way. If you don't like to implement Modula-3, chose another language to work on. If you don't like to code in Modula-3, choose a project coded in another language. If you are finding that using C as an intermediate language makes things more difficult or more unpleasant than you want, well, as I recall, you had ample warning before you started, from some of us who have gotten run around that flag pole in the past. If your method of dealing with the implementation problems that confront you is to define away requirements, then it casts doubt on the wisdom of the whole C backend approach, both from the perspective of the implementation itself, or of just what you really want to do. As for values of procedure type, do it the same way it's already done. If C is as fine for low-level code as you keep telling us, this will be straightforward. On 3/9/21 1:11 AM, Jay K wrote: > Closures are a problem. > > Specifically the ability to take the address of a nested function > and pass that as a function pointer, and that it has the same > type as a pointer to a non-nested or C function, and it can access > the locals of the lexical enclosure (arbitrary levels deep, with recursion, > but this doesn't make it worse). > > The problems: > > - The assumption that -1 is not valid code. > (This value could be changed per system). > > - The assumption that code pages are readable. > Some systems have execute-only memory. > Reading works on wasm but I don't know what it does, wierd. > > - The assumption that reading code pages are meaningful. (wasm?) > > - The inability to pass these pointers to C code as function pointers. > > - I believe...sorry, I should know better, but I believe in the C backend > I play lose with the signatures of functions, passing an extra parameter always, > to functions that might not expect it. > This likely plays poorly with some environments, such as authenticated ARM function pointers > and WebAssembly. > > WebAssembly might eventually get a feature to help here. > > So, really, I wish this feature was not in the language. > Barring that, maybe we can remove its use from the base system? > Maybe have an optional flag or two: > - prohibit taking address of nested function > - assume function pointers are not closures > > I realize these are different. In that, the second idea in a lower level > library, breaks use with a higher level library. > > I know about several approaches to solve this problem, but none of them are good. > > Traditional gcc generates code on the stack. > Apple removed this from their compilers years ago. > > In some systems you can stamp out identical copies of code, with associated data, and > manage that like a cheap heap. But I think that still is not worth while. > It still is not cheap, and it is platform specific. > Libffi and Mono use this on iPhone for example. > Libffi will be using it for all systems, at least all Linux systems. > > Some systems depend on JIT for this (e.g. CLR on non-iphone). > > In normal C code this is handled in a reasonable way, case by case. > > For example the qsort_s function (not regular qsort) takes function pointer and a void*. > The void* is extra context agreed upon between the qsort caller and the function pointer. > qsort does not know what it is. It can be a pointer to a struct. It can be an integer. It can be ignored. > There is casting require, but it is ok. It isn't "technically safe", but it is easy to get correct. > > This achieves the same result as a "closure". > > Occasionally functions, like regular qsort, "forgot" to take the extra void*, and this can be problem indeed. > > I'll try to find where in the system we use closures and if they can be "fixed" somehow. > I would like to soon-ish see if about compiling the system to WebAssembly, via C, and running it in node.js. > The C output already mostly can compile to WebAssembly, not having done anything. > > I think Oberon removed this feature, for example. I just glanced at its specification and it seemed to disallow it. > > Thoughts? > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Mar 10 17:09:08 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 10 Mar 2021 10:09:08 -0600 Subject: [M3devel] closures are a problem In-Reply-To: References: Message-ID: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> On 3/9/21 5:30 PM, Jay K wrote: > To clarify and confirm, it does already work. > Well, that makes me very confused. Why then did you propose to eliminate the the feature (passing nested procedures as parameters) from Modula-3 and follow it with a litany of reasons to do so and why you couldn't or didn't want to implement it? Were you rolling? If so, it certainly worked on me. > AMD64_NT works, from long ago. > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > RISCV64_LINUX works. It builds itself in qemu. > Debugging is also ok not great. > I can make it better. > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > To fix the leaky conservative GC. > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > AppleWatch is a tough one. > > WebAssembly I believe also will not work, > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > Thank you, > - Jay > - -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Mar 10 17:51:35 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 10 Mar 2021 10:51:35 -0600 Subject: [M3devel] more win64 subrange problems uint32 := -1. In-Reply-To: References: <6e9418a9-b71a-f188-221d-c32cd38ad871@lcwb.coop> Message-ID: <2e3b563e-c8c5-3453-78cf-f48d5aa4bec4@lcwb.coop> On 3/9/21 9:06 PM, Jay K wrote: > It isn't me. It is the code we have been sitting on forever. > Just that we have only been targeting Win32, not Win64. > > The two 32bit numbers combine to form a 64bit number. > The lower 32bits are meant to be unsigned -- not sign extended. > The signedness of the upper 32bits does not matter much, as 31 bits is enough in practise (63 bits overall). > > The overall value -1 is surely special, it is not a magnitude but a magic value. > You could likely think of this as a union of a boolean and an almost 64bit number. > > ?- Jay > > OK, new information. So to make this work on machines of both word sizes, you need to pass 16_FFFFFFFF, (which for clarity means 16_00000000FFFFFFFF on 64-bit). This is in range for UNIT32. On either word size. On 64-bit, -1 is 16_FFFFFFFFFFFFFFFF, out of range. If LockFile will treat the entire pair as 64-bit signed, and you want to pass the maximum representable positive value, then instead pass 16_7FFFFFFF for the upper UINT32 parameter. Yes, -1 is special, and part of its specialness is that its bit pattern meaning differs with word size, And that difference is the source of the all this trouble. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Rodney M. Bates > *Sent:* Wednesday, March 10, 2021 2:57 AM > *To:* Jay K ; m3devel > *Subject:* Re: [M3devel] more win64 subrange problems uint32 := -1. > > > On 3/9/21 1:30 PM, Jay K wrote: >> Searching for 'lockfile'... >> C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(236): ? ?IF WinBase.LockFile( >> C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(251): ? ?IF WinBase.UnlockFile( >> C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(267): ? ?IF WinBase.LockFile( >> C:\s\cm3\m3-libs\libm3\src\os\WIN32\FileWin32.m3(285): ? ?IF WinBase.UnlockFile( >> 4 occurrence(s) have been found. >> >> >>? ? ? IF WinBase.LockFile( >>? ? ? ? ? ? ?hFile := h.handle, >>? ? ? ? ? ? ?dwFileOffsetLow := 0, >>? ? ? ? ? ? ?dwFileOffsetHigh := 0, >>? ? ? ? ? ? ?nNumberOfBytesToLockLow := -1, >>? ? ? ? ? ? ?nNumberOfBytesToLockHigh := -1) = 0 THEN >>? ? ? ? err := WinBase.GetLastError(); >> >> >> PROCEDURE RegularFileUnlock(h: RegularFile.T) RAISES {OSError.E}= >>? ? BEGIN >>? ? ? IF WinBase.UnlockFile( >>? ? ? ? ? ? ?hFile := h.handle, >>? ? ? ? ? ? ?dwFileOffsetLow := 0, >>? ? ? ? ? ? ?dwFileOffsetHigh := 0, >>? ? ? ? ? ? ?nNumberOfBytesToUnlockLow := -1, >>? ? ? ? ? ? ?nNumberOfBytesToUnlockHigh := -1) = 0 THEN >>? ? ? ? OSErrorWin32.Raise() >>? ? ? END; >>? ? END RegularFileUnlock; >> >> <*EXTERNAL LockFile:WINAPI*> >> PROCEDURE LockFile (hFile ? ? ? ? ? ? ? ? ? : HANDLE; >>? ? ? ? ? ? ? ? ? ? ? dwFileOffsetLow ? ? ? ? : UINT32; >>? ? ? ? ? ? ? ? ? ? ? dwFileOffsetHigh ? ? ? ?: UINT32; >>? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToLockLow : UINT32; >>? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToLockHigh: UINT32 ? ): BOOL; >> >> <*EXTERNAL UnlockFile:WINAPI*> >> PROCEDURE UnlockFile (hFile ? ? ? ? ? ? ? ? ? ? : HANDLE; >>? ? ? ? ? ? ? ? ? ? ? ? dwFileOffsetLow ? ? ? ? ? : UINT32; >>? ? ? ? ? ? ? ? ? ? ? ? dwFileOffsetHigh ? ? ? ? ?: UINT32; >>? ? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToUnlockLow : UINT32; >>? ? ? ? ? ? ? ? ? ? ? ? nNumberOfBytesToUnlockHigh: UINT32 ? ): BOOL; >> >> >> These get warnings when compiling. But only for Win64, not Win32. >> I'm not sure if they fail at runtime or not. > > Makes perfect sense.? You appear to be trying to pass a negative number, but > the parameter type is UINT32, an unsigned type that includes no negative values. > > On 64-bits, -1 is well within the range of INTEGER and really is a negative > value, not in the range of UINT32 (whose lower bound is zero) and thus the warning. > If you execute this call with -1, it will also get a runtime range error. > > On 32-bits, UINT32 (and INT32 too) are full-range native words.? The difference > is only in whether the code treats the bits as signed twos-complement, or > unsigned binary.? It can be either, but all places must be consistent, and > in this case, you won't be changing the Windows system call, so that will > control which it is. > > On 32-bits, the bit pattern -1 = 16_FFFFFFFF is a negative number if > treated as signed, but it's the maximum 32-bit unsigned (positive) value, if treated > as unsigned.? I.e., it's 2^32-1.? If you want a signed value in a 32-bit word, you > can only get up to 2^31-1.? No warning or error here, because the range > covers all of a machine word and every bit pattern is in it.? But I greatly > doubt you will really want to lock 2^32-1 bytes--the entire maximum addressable > RAM size.? No error messages, but no doubt a big bug. > > The external LockFile:WINAPI will have to treat bit patterns in 2^31..2^32-1 > as either negative, or large positive.? They can't be ambiguous.? So you need > to look at the definition of LockFile:WINAPI (I presume it's a C prototype) > and see whether it really treats the parameters as signed or unsigned, and, > if the former, adjust the type of the parameters to INT32.? If the latter, > don't pass any negative values. > > If it won't tell you this somehow, you have grounds to faunch.? Then consistently > make all the values you pass lie within this range.? Don't try to pass a negative number > to a UINT32 and don't try to pass anything > 2^31-1 to an INT32.? Either way, > you'll have trouble, sometimes with warnings and errors, sometimes with just > bad behavior. > > Just intuitively from the formal names, I would guess zero bytes is the > fewest you would want to lock, so zero is probably what you want. > > >> >> It isn't terrible pattern, I think, to reserve a few values like this. >> Perhaps in this case, you might as well give up half the values though and use int32? >> Well, on "high", but not on low". >> Of course, taking an int64 and not two 32bit integers, would be preferable. >> >> I am inclined, as ever, to just move a few lines of Modula3 to C, >> because I very much dislike the pattern of rewriting system C headers in other languages. >> Though it perhaps it isn't too bad on Windows. >> >>? ?- Jay >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C889fa88893b84f2234d608d8e37054c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637509418889721122%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=37cIsIlpH9vggIpls4yWTCwQrtkJFXU1EDbU3%2FZi7ZE%3D&reserved=0 >> > > -- > Rodney Bates > rodney.m.bates at acm.org > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From mika at alum.mit.edu Wed Mar 10 19:38:28 2021 From: mika at alum.mit.edu (Mika Nystrom) Date: Wed, 10 Mar 2021 10:38:28 -0800 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: Message-ID: Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. On Tue, Mar 9, 2021 at 11:50 AM Jay Krell wrote: > ------------------------------ > You can view, comment on, or merge this pull request online at: > > https://github.com/modula3/cm3/pull/135 > Commit Summary > > - sysutils: Allow Lower and Upper of NIL, returning NIL. > > File Changes > > - *M* m3-libs/sysutils/src/pm3/TextUtils.m3 > > (8) > > Patch Links: > > - https://github.com/modula3/cm3/pull/135.patch > - https://github.com/modula3/cm3/pull/135.diff > > ? > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > , or unsubscribe > > . > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 21:05:39 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 20:05:39 +0000 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: , Message-ID: Why not? Caller can avoid nil check and get back same result they would have had. Smaller code. Infinitesimally slowing some paths. - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, March 10, 2021 10:38:28 AM To: modula3/cm3 ; Jay K ; m3devel Cc: modula3/cm3 ; Subscribed Subject: Re: [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. On Tue, Mar 9, 2021 at 11:50 AM Jay Krell > wrote: ________________________________ You can view, comment on, or merge this pull request online at: https://github.com/modula3/cm3/pull/135 Commit Summary * sysutils: Allow Lower and Upper of NIL, returning NIL. File Changes * M m3-libs/sysutils/src/pm3/TextUtils.m3 (8) Patch Links: * https://github.com/modula3/cm3/pull/135.patch * https://github.com/modula3/cm3/pull/135.diff ? You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 10 21:34:12 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 20:34:12 +0000 Subject: [M3devel] closures are a problem In-Reply-To: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> References: , <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> Message-ID: I thought I was repeatedly clear. It is a problem on systems that have execute-only memory. Such as Android..well, I thought, and it was, but not as much as I thought. And other systems. These systems "crash" in some form if you try to read code bytes. I believe it is problem on WebAssembly, but I will confirm soon. I suspect it is problem on ARM systems with the new "pointer authentication" feature. My M1 Mac I think does not have that enabled. It sounds like current iPhones work that way. I haven't tried. I don't know if Apple is going to require that in future or not. It is ugly to have this -1 marker and believe that means not-code. It is not always necessarily a problem. It would be nice, perhaps, to remove their use in-tree and then clearly reduce the problem. Though it would also be good to confirm the WebAssembly situation if the system becomes available there, so people know what they can do, and WebAssembly might gain support eventually. There might be other ways to deal with this. In particular, could you maybe always pass around two words? Or three words, the third being a type indicator? (maybe the nil-ness or lower bit of the second word could be the type indicator; the difference between 2 and 3 isn't important though). If the system was all Modula-3 then yes. Given interoperation with C, maybe. Question in my mind becomes, can the compiler (or perhaps libraries) see the boundaries between the systems? And when recieving a pointer from C, expand it. When passing a pointer to C, runtime check that it isn't a closure, which doesn't work today anyway (it'd crash), and shrink it. For example, and maybe this is worth implementing to prove it out.. You cannot pass a Modula-3 closure to stdlib.h/qsort. You can pass a Modula-3 non-closure to stdlib.h/qsort. If you then implement qsort in Modula-3, with the same apparent mistake of "missing context", ideally it would accept both context-free C and Modula-3 functions, and Modula-3 closures. And it'd just work seamlessly, except for possible compile-time or runtime errors for passing a closure to C. - Jay ________________________________ From: Rodney M. Bates Sent: Wednesday, March 10, 2021 4:09 PM To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] closures are a problem On 3/9/21 5:30 PM, Jay K wrote: > To clarify and confirm, it does already work. > Well, that makes me very confused. Why then did you propose to eliminate the the feature (passing nested procedures as parameters) from Modula-3 and follow it with a litany of reasons to do so and why you couldn't or didn't want to implement it? Were you rolling? If so, it certainly worked on me. > AMD64_NT works, from long ago. > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > RISCV64_LINUX works. It builds itself in qemu. > Debugging is also ok not great. > I can make it better. > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > To fix the leaky conservative GC. > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > AppleWatch is a tough one. > > WebAssembly I believe also will not work, > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > Thank you, > - Jay > - -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 11 00:41:51 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 10 Mar 2021 23:41:51 +0000 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: , Message-ID: I ended up with NIL check in caller, so it is ok either way, presently m3middle: Some uppercasing of target to aid case insensitivity. ? modula3/cm3 at 82f1dce (github.com) I suggest "target" can go away though, replaced by its parts: word size endian Unix or Win32 and even those should be rarely used. Most code is the same independent of them. And the concatenation, well, is still needed: output_dir. But has less meaning. It could just be "obj" or "testbuild1". It is just a string. Granted, it is a string for which you have partitioned according to those factors. You don't want to use the same string for different word size. But code should never have to do anything based on the combination wordsize+endian+os, just usually one of the pieces. Except again for placing output. Perhaps this is much about little. - Jay ________________________________ From: Mika Nystrom Sent: Wednesday, March 10, 2021 8:44 PM To: Jay K Cc: modula3/cm3 ; m3devel ; modula3/cm3 ; Subscribed Subject: Re: [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) yeah ok, but you're making the NIL check unconditional. Normally you'd have some invariant in the code that guarantees that the object is not NIL so you would not do the NIL check. The NIL check is implicit as an assertion (segfault) against the invariant in that case. In any case you wouldn't need an actual IF to check for NIL. Not that it's a big deal. I'm not deeply against the change, just saying that most of the time properties like that are inherent in the calling code. And if not, why not...? Ok I'm not sure what led to the change so I won't say more. Maybe it's the right thing to do. Especially if this isn't a widely used library. I think there must be several interfaces called TextUtils. Is this only used in cm3 itself? On Wed, Mar 10, 2021 at 12:05 PM Jay K > wrote: Why not? Caller can avoid nil check and get back same result they would have had. Smaller code. Infinitesimally slowing some paths. - Jay ________________________________ From: Mika Nystrom > Sent: Wednesday, March 10, 2021 10:38:28 AM To: modula3/cm3 >; Jay K >; m3devel > Cc: modula3/cm3 >; Subscribed > Subject: Re: [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. On Tue, Mar 9, 2021 at 11:50 AM Jay Krell > wrote: ________________________________ You can view, comment on, or merge this pull request online at: https://github.com/modula3/cm3/pull/135 Commit Summary * sysutils: Allow Lower and Upper of NIL, returning NIL. File Changes * M m3-libs/sysutils/src/pm3/TextUtils.m3 (8) Patch Links: * https://github.com/modula3/cm3/pull/135.patch * https://github.com/modula3/cm3/pull/135.diff ? You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Mar 11 01:29:35 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 10 Mar 2021 19:29:35 -0500 Subject: [M3devel] closures are a problem In-Reply-To: References: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> Message-ID: <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com> On Wed, Mar 10, 2021 at 08:34:12PM +0000, Jay K wrote: > I thought I was repeatedly clear. > > It is a problem on systems that have execute-only memory. > Such as Android..well, I thought, and it was, but not > as much as I thought. > And other systems. > These systems "crash" in some form if you try to read code bytes. > > I believe it is problem on WebAssembly, but I will confirm soon. > > I suspect it is problem on ARM systems with the new "pointer authentication" feature. > My M1 Mac I think does not have that enabled. > It sounds like current iPhones work that way. I haven't tried. > I don't know if Apple is going to require that in future or not. > > It is ugly to have this -1 marker and believe that means not-code. > > It is not always necessarily a problem. > > It would be nice, perhaps, to remove their use in-tree and then clearly > reduce the problem. Though it would also be good to confirm > the WebAssembly situation if the system becomes available there, > so people know what they can do, > and WebAssembly might gain support eventually. > > There might be other ways to deal with this. > > In particular, could you maybe always pass around two words? > Or three words, the third being a type indicator? > (maybe the nil-ness or lower bit of the second word could be the > type indicator; the difference between 2 and 3 isn't important though). This is the standard way to implement nested procedures, invented way back in the 60's for Algol 60. Of course if you don't need the "address" of the procedure, its environment will be a subset ouf yours, and you don't need to make the two-word "address" You just load up the right register and do the call. Andif you statically know that the procedure being called is a top-level procedure (this is known for almost all procedure calle) you don't need the environment pointer a all. Now if you're calling an external procedure using C calling conventions, you just call it. Is it even possible in Modula 3 to call a C procedure nonstatically? I'd say, if you need the "address" of a C procedure you just (at compile time) make a Modula 3 procedure that calls it without taking the address, and return the two=word "address" of the Modula 3 procedure instead. Is this hard? -- hendrik > > If the system was all Modula-3 then yes. > > Given interoperation with C, maybe. > > Question in my mind becomes, can the compiler (or perhaps libraries) > see the boundaries between the systems? > > And when recieving a pointer from C, expand it. > > When passing a pointer to C, runtime check that it isn't a closure, > which doesn't work today anyway (it'd crash), and shrink it. > > For example, and maybe this is worth implementing to prove it out.. > You cannot pass a Modula-3 closure to stdlib.h/qsort. > You can pass a Modula-3 non-closure to stdlib.h/qsort. > If you then implement qsort in Modula-3, with the same apparent > mistake of "missing context", ideally it would accept > both context-free C and Modula-3 functions, and Modula-3 closures. > And it'd just work seamlessly, except for possible compile-time > or runtime errors for passing a closure to C. > > - Jay > > ________________________________ > From: Rodney M. Bates > Sent: Wednesday, March 10, 2021 4:09 PM > To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org > Cc: m3devel > Subject: Re: [M3devel] closures are a problem > > > > On 3/9/21 5:30 PM, Jay K wrote: > > To clarify and confirm, it does already work. > > > > Well, that makes me very confused. Why then did you propose to eliminate the > the feature (passing nested procedures as parameters) from Modula-3 and > follow it with a litany of reasons to do so and why you couldn't or didn't > want to implement it? > > Were you rolling? If so, it certainly worked on me. > > > > AMD64_NT works, from long ago. > > > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > > > RISCV64_LINUX works. It builds itself in qemu. > > Debugging is also ok not great. > > I can make it better. > > > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > > To fix the leaky conservative GC. > > > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > > AppleWatch is a tough one. > > > > WebAssembly I believe also will not work, > > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > > > Thank you, > > - Jay > > - > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Thu Mar 11 02:03:21 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 10 Mar 2021 20:03:21 -0500 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: Message-ID: <20210311010321.4qsqqwjprjlnhx33@topoi.pooq.com> On Wed, Mar 10, 2021 at 10:38:28AM -0800, Mika Nystrom wrote: > Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? > > Usually passing NIL to a TEXT manipulation function suggests a bug or logic > error in higher level code... in my experience, anyhow. So the question is whether the compiler should be fixing programmer error behind his back. -- hendrik > > > On Tue, Mar 9, 2021 at 11:50 AM Jay Krell wrote: > > > ------------------------------ > > You can view, comment on, or merge this pull request online at: > > > > https://github.com/modula3/cm3/pull/135 > > Commit Summary > > > > - sysutils: Allow Lower and Upper of NIL, returning NIL. > > > > File Changes > > > > - *M* m3-libs/sysutils/src/pm3/TextUtils.m3 > > > > (8) > > > > Patch Links: > > > > - https://github.com/modula3/cm3/pull/135.patch > > - https://github.com/modula3/cm3/pull/135.diff > > > > ? > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub > > , or unsubscribe > > > > . > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Thu Mar 11 02:11:28 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 10 Mar 2021 19:11:28 -0600 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: Message-ID: On 3/10/21 12:38 PM, Mika Nystrom wrote: > Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? > > Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. > > From Text.i3: ---------------------------------------------------------------------- (* A non-nil "TEXT" represents an immutable, zero-based sequence of characters. "NIL" does not represent any sequence of characters, ^^^ it will not be returned from any procedure in this interface, and ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ it is a checked runtime error to pass "NIL" to any procedure in ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this interface. *) ^^^^^^^^^^^^^^^ INTERFACE Text; --------------------------------------------------------------------- It would be nice if Text and TextUtils were consistent with each other on this. Even nicer if TextUtils were consistent with itself, not just Lower and Upper. I have no strong position on which way. > On Tue, Mar 9, 2021 at 11:50 AM Jay Krell > wrote: > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > > You can view, comment on, or merge this pull request online at: > > https://github.com/modula3/cm3/pull/135 > > > Commit Summary > > * sysutils: Allow Lower and Upper of NIL, returning NIL. > > > File Changes > > * *M* m3-libs/sysutils/src/pm3/TextUtils.m3 (8) > > > Patch Links: > > * https://github.com/modula3/cm3/pull/135.patch > * https://github.com/modula3/cm3/pull/135.diff > > ? > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub , or unsubscribe . > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From hendrik at topoi.pooq.com Thu Mar 11 02:25:36 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Wed, 10 Mar 2021 20:25:36 -0500 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: References: Message-ID: <20210311012536.vdptuoouzl6k2xra@topoi.pooq.com> On Wed, Mar 10, 2021 at 07:11:28PM -0600, Rodney M. Bates wrote: > > > On 3/10/21 12:38 PM, Mika Nystrom wrote: > > Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? > > > > Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. Using NIL might be useful if one needed to have an optional string and distinguish between an empty string and an absence of a string. But, even then, don't pass it to a string function. -- hendrik > > > > > > From Text.i3: > ---------------------------------------------------------------------- > (* A non-nil "TEXT" represents an immutable, zero-based sequence of > characters. "NIL" does not represent any sequence of characters, > ^^^ > it will not be returned from any procedure in this interface, and > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > it is a checked runtime error to pass "NIL" to any procedure in > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > this interface. *) > ^^^^^^^^^^^^^^^ > > INTERFACE Text; > --------------------------------------------------------------------- > > It would be nice if Text and TextUtils were consistent with each other on this. > > Even nicer if TextUtils were consistent with itself, not just Lower and Upper. > > I have no strong position on which way. > > > > On Tue, Mar 9, 2021 at 11:50 AM Jay Krell > wrote: > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > > > > > You can view, comment on, or merge this pull request online at: > > > > https://github.com/modula3/cm3/pull/135 > > > > > > Commit Summary > > > > * sysutils: Allow Lower and Upper of NIL, returning NIL. > > > > > > File Changes > > > > * *M* m3-libs/sysutils/src/pm3/TextUtils.m3 (8) > > > > > > Patch Links: > > > > * https://github.com/modula3/cm3/pull/135.patch > > * https://github.com/modula3/cm3/pull/135.diff > > > > ? > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub , or unsubscribe . > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jayk123 at hotmail.com Thu Mar 11 06:43:58 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 05:43:58 +0000 Subject: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) In-Reply-To: <20210311012536.vdptuoouzl6k2xra@topoi.pooq.com> References: , <20210311012536.vdptuoouzl6k2xra@topoi.pooq.com> Message-ID: > using NIL might be useful if one needed to have an optional string an This is exactly the use case. In the wider world I have experienced plenty of code for which NULL indicates not passing an "optional" parameter. It is generally not considered a big deal, or problematic or anything. And definitely many but not all code bases consider a missing string different from an empty string. I have experienced other opposite approaches in smaller quantity. It isn't just n people have n opinions, but that the n opinions can be the opposite and completely uncomposable. Some project maintainers believe optional parameters are confusing and do stuff like this: void foo_full(int, char*) // real implementation { ... } void foo(int i) // thin wrapper { foo_full(i, NULL); } It does not help the implementor of foo_full at all, nor what it passes the data on to. It is a nice-ish size optimization at the foo call sites, perhaps. I don't really buy it but maybe. It foo_full's signature is changing a lot, and most callers just call foo, that can help maintenance. it also depends on if you can visit all the callers and update them or not. Sometimes you just cannot change signatures for compatibility across dynamic linking across multiple code bases. > But, even then, don't pass it to a string function. This can be tedious and error prone and just asking for bugs, when the alternative is trivial correct safe efficient, etc. At least to the extent that you can convince yourself that these are reasonable semantics. i.e. Almost any transform of a nil string is an identity? - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Thursday, March 11, 2021 1:25 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] [modula3/cm3] sysutils: Allow Lower and Upper of NIL, returning NIL. (#135) On Wed, Mar 10, 2021 at 07:11:28PM -0600, Rodney M. Bates wrote: > > > On 3/10/21 12:38 PM, Mika Nystrom wrote: > > Curious why you need Upper and Lower to support Upper-ing and Lower-ing NIL? > > > > Usually passing NIL to a TEXT manipulation function suggests a bug or logic error in higher level code... in my experience, anyhow. Using NIL might be useful if one needed to have an optional string and distinguish between an empty string and an absence of a string. But, even then, don't pass it to a string function. -- hendrik > > > > > > From Text.i3: > ---------------------------------------------------------------------- > (* A non-nil "TEXT" represents an immutable, zero-based sequence of > characters. "NIL" does not represent any sequence of characters, > ^^^ > it will not be returned from any procedure in this interface, and > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > it is a checked runtime error to pass "NIL" to any procedure in > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > this interface. *) > ^^^^^^^^^^^^^^^ > > INTERFACE Text; > --------------------------------------------------------------------- > > It would be nice if Text and TextUtils were consistent with each other on this. > > Even nicer if TextUtils were consistent with itself, not just Lower and Upper. > > I have no strong position on which way. > > > > On Tue, Mar 9, 2021 at 11:50 AM Jay Krell > wrote: > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > > > > > You can view, comment on, or merge this pull request online at: > > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fpull%2F135&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646580670%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=bRDrfwlY9zsZhOaiuboOhwKx2In12hQaT4gSZA26JTE%3D&reserved=0 > > > > > > Commit Summary > > > > * sysutils: Allow Lower and Upper of NIL, returning NIL. > > > > > > File Changes > > > > * *M* m3-libs/sysutils/src/pm3/TextUtils.m3 (8) > > > > > > Patch Links: > > > > * https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fpull%2F135.patch&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646580670%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wQjo9l3abzVPo8sxeqrAqzRCQezBIDVtW0pemb%2FDpS4%3D&reserved=0 > > * https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmodula3%2Fcm3%2Fpull%2F135.diff&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646580670%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IwOXq8KvdgpUxl99kLnMjIPKKjhKbUaGj03qCvMv1%2FU%3D&reserved=0 > > > > ? > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub , or unsubscribe . > > > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646580670%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pCUuzQAte7FrLgDtGXLQqztmb6yJ9zjuYvOxjjg%2Bea8%3D&reserved=0 > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646590663%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JrA4MU6VHbwSVTA3oI3N6eKs8kPwlRFkI9a%2BHpkhj5c%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C40de1cf354b246eef51e08d8e42ca1c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510227646590663%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JrA4MU6VHbwSVTA3oI3N6eKs8kPwlRFkI9a%2BHpkhj5c%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 11 07:10:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 06:10:06 +0000 Subject: [M3devel] closures are a problem In-Reply-To: <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com> References: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> , <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com> Message-ID: > Is it even possible in Modula 3 to call a C procedure nonstatically? Yes I believe so. I believe it just works. Perhaps it is not considered useful? > I'd say, if you need the "address" of a C procedure you just > (at compile time) make a Modula 3 procedure that calls it > without taking the address, and return the two=word "address" of > the Modula 3 procedure instead. > > Is this hard? I think this is the start of a solution. But I am not sure it is complete. Or rather, that part of the solution is convincing. But more generally, how do you pass function pointers e.g. in Trestle to Win32 or Xlib? There is function pointer interop between Modula-3 and C and most scenarios work and are seamless. One scenario does not work and crashes. That is, C and non-nested Modula-3 functions interop fine. Address of nested Modula-3 function is the problem, as I have been saying. You end up being "stuck" with the single word C representation. Which is why we have this if_closure thing. I guess. Question becomes, what amount of interop and what amount of features are a worthwhile tradeoff, and how statically understood is the system, such as to enable workarounds? I mean, do you really always know that a function is C and that address-of means to generate a wrapper function that throws out the "static link" and passes on the rest of the parameters? Maybe. Similarly, if I pass a Modula-3 function address to C, what does that look like? Truncate the two words to one? Assert the second word is nil? Maybe this can work. And then, the function if_closure just goes away. Perhaps all quite nice. I don't mind going through those thin thunks, my only concern is two part: - Are all the boundaries knowable where type translation has to occur under the covers? - Does anyone compare these things for equality? Does that work? Why did the original implementors go with the single word + if_closure approach? Thank you, - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Thursday, March 11, 2021 12:29 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem On Wed, Mar 10, 2021 at 08:34:12PM +0000, Jay K wrote: > I thought I was repeatedly clear. > > It is a problem on systems that have execute-only memory. > Such as Android..well, I thought, and it was, but not > as much as I thought. > And other systems. > These systems "crash" in some form if you try to read code bytes. > > I believe it is problem on WebAssembly, but I will confirm soon. > > I suspect it is problem on ARM systems with the new "pointer authentication" feature. > My M1 Mac I think does not have that enabled. > It sounds like current iPhones work that way. I haven't tried. > I don't know if Apple is going to require that in future or not. > > It is ugly to have this -1 marker and believe that means not-code. > > It is not always necessarily a problem. > > It would be nice, perhaps, to remove their use in-tree and then clearly > reduce the problem. Though it would also be good to confirm > the WebAssembly situation if the system becomes available there, > so people know what they can do, > and WebAssembly might gain support eventually. > > There might be other ways to deal with this. > > In particular, could you maybe always pass around two words? > Or three words, the third being a type indicator? > (maybe the nil-ness or lower bit of the second word could be the > type indicator; the difference between 2 and 3 isn't important though). This is the standard way to implement nested procedures, invented way back in the 60's for Algol 60. Of course if you don't need the "address" of the procedure, its environment will be a subset ouf yours, and you don't need to make the two-word "address" You just load up the right register and do the call. Andif you statically know that the procedure being called is a top-level procedure (this is known for almost all procedure calle) you don't need the environment pointer a all. Now if you're calling an external procedure using C calling conventions, you just call it. Is it even possible in Modula 3 to call a C procedure nonstatically? I'd say, if you need the "address" of a C procedure you just (at compile time) make a Modula 3 procedure that calls it without taking the address, and return the two=word "address" of the Modula 3 procedure instead. Is this hard? -- hendrik > > If the system was all Modula-3 then yes. > > Given interoperation with C, maybe. > > Question in my mind becomes, can the compiler (or perhaps libraries) > see the boundaries between the systems? > > And when recieving a pointer from C, expand it. > > When passing a pointer to C, runtime check that it isn't a closure, > which doesn't work today anyway (it'd crash), and shrink it. > > For example, and maybe this is worth implementing to prove it out.. > You cannot pass a Modula-3 closure to stdlib.h/qsort. > You can pass a Modula-3 non-closure to stdlib.h/qsort. > If you then implement qsort in Modula-3, with the same apparent > mistake of "missing context", ideally it would accept > both context-free C and Modula-3 functions, and Modula-3 closures. > And it'd just work seamlessly, except for possible compile-time > or runtime errors for passing a closure to C. > > - Jay > > ________________________________ > From: Rodney M. Bates > Sent: Wednesday, March 10, 2021 4:09 PM > To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org > Cc: m3devel > Subject: Re: [M3devel] closures are a problem > > > > On 3/9/21 5:30 PM, Jay K wrote: > > To clarify and confirm, it does already work. > > > > Well, that makes me very confused. Why then did you propose to eliminate the > the feature (passing nested procedures as parameters) from Modula-3 and > follow it with a litany of reasons to do so and why you couldn't or didn't > want to implement it? > > Were you rolling? If so, it certainly worked on me. > > > > AMD64_NT works, from long ago. > > > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > > > RISCV64_LINUX works. It builds itself in qemu. > > Debugging is also ok not great. > > I can make it better. > > > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > > To fix the leaky conservative GC. > > > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > > AppleWatch is a tough one. > > > > WebAssembly I believe also will not work, > > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > > > Thank you, > > - Jay > > - > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026662066%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SwDerQFNqm%2BSrzBcV3Pb4Qs0WmCogmYwICuGu%2BmxrME%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026672062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUmEr%2FVcDkuO0bw0D5Puwr5T67aRlhyFZvohoEWY2gc%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 11 07:34:19 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 06:34:19 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> , <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com>, Message-ID: Repeating myself perhaps.. > And if you statically know that the procedure being called is a > top-level procedure (this is known for almost all procedure calle) > you don't need the environment pointer a all. > > Now if you're calling an external procedure using C calling conventions, > you just call it. The problem in my mind, is knowing all this perfectly. The current system does work "perfectly", with the caveats noted. It does not need to distinguish Modula-3 code from C code. "external" functions have the same calling convention as not-external. Taking the address of a non-nested Modula-3 function or C function always just produces one word. Taking the address of a nested Modula-3 function always produces a closure, I think. Perhaps m3front checks if the static link is used or not. Sometimes nesting is just a namespace construct. Calling a function pointer always goes through the if_closure dynamic check, so always accepts any of the three types. The one never-working scenario is passing the address of a nested function to C. There remains no good solution for that. (bad solutions include JIT, in heap or stack) If all C/Modula-3 boundaries are knowable, that does seem to offer like what you say. It means a function pointer is now always two words when Modula-3 passes to Modula-3, and little adapters are needed between C and Modula-3, in various combinations, so each can call pointers to the other, and each can take the address of the other. A function pointer would generally always be two words. Pass the parameters and the static link. The static link could be first or last. Does it hold together? - Jay ________________________________ From: Jay K Sent: Thursday, March 11, 2021 6:10 AM To: Hendrik Boom ; m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem > Is it even possible in Modula 3 to call a C procedure nonstatically? Yes I believe so. I believe it just works. Perhaps it is not considered useful? > I'd say, if you need the "address" of a C procedure you just > (at compile time) make a Modula 3 procedure that calls it > without taking the address, and return the two=word "address" of > the Modula 3 procedure instead. > > Is this hard? I think this is the start of a solution. But I am not sure it is complete. Or rather, that part of the solution is convincing. But more generally, how do you pass function pointers e.g. in Trestle to Win32 or Xlib? There is function pointer interop between Modula-3 and C and most scenarios work and are seamless. One scenario does not work and crashes. That is, C and non-nested Modula-3 functions interop fine. Address of nested Modula-3 function is the problem, as I have been saying. You end up being "stuck" with the single word C representation. Which is why we have this if_closure thing. I guess. Question becomes, what amount of interop and what amount of features are a worthwhile tradeoff, and how statically understood is the system, such as to enable workarounds? I mean, do you really always know that a function is C and that address-of means to generate a wrapper function that throws out the "static link" and passes on the rest of the parameters? Maybe. Similarly, if I pass a Modula-3 function address to C, what does that look like? Truncate the two words to one? Assert the second word is nil? Maybe this can work. And then, the function if_closure just goes away. Perhaps all quite nice. I don't mind going through those thin thunks, my only concern is two part: - Are all the boundaries knowable where type translation has to occur under the covers? - Does anyone compare these things for equality? Does that work? Why did the original implementors go with the single word + if_closure approach? Thank you, - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Thursday, March 11, 2021 12:29 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem On Wed, Mar 10, 2021 at 08:34:12PM +0000, Jay K wrote: > I thought I was repeatedly clear. > > It is a problem on systems that have execute-only memory. > Such as Android..well, I thought, and it was, but not > as much as I thought. > And other systems. > These systems "crash" in some form if you try to read code bytes. > > I believe it is problem on WebAssembly, but I will confirm soon. > > I suspect it is problem on ARM systems with the new "pointer authentication" feature. > My M1 Mac I think does not have that enabled. > It sounds like current iPhones work that way. I haven't tried. > I don't know if Apple is going to require that in future or not. > > It is ugly to have this -1 marker and believe that means not-code. > > It is not always necessarily a problem. > > It would be nice, perhaps, to remove their use in-tree and then clearly > reduce the problem. Though it would also be good to confirm > the WebAssembly situation if the system becomes available there, > so people know what they can do, > and WebAssembly might gain support eventually. > > There might be other ways to deal with this. > > In particular, could you maybe always pass around two words? > Or three words, the third being a type indicator? > (maybe the nil-ness or lower bit of the second word could be the > type indicator; the difference between 2 and 3 isn't important though). This is the standard way to implement nested procedures, invented way back in the 60's for Algol 60. Of course if you don't need the "address" of the procedure, its environment will be a subset ouf yours, and you don't need to make the two-word "address" You just load up the right register and do the call. Andif you statically know that the procedure being called is a top-level procedure (this is known for almost all procedure calle) you don't need the environment pointer a all. Now if you're calling an external procedure using C calling conventions, you just call it. Is it even possible in Modula 3 to call a C procedure nonstatically? I'd say, if you need the "address" of a C procedure you just (at compile time) make a Modula 3 procedure that calls it without taking the address, and return the two=word "address" of the Modula 3 procedure instead. Is this hard? -- hendrik > > If the system was all Modula-3 then yes. > > Given interoperation with C, maybe. > > Question in my mind becomes, can the compiler (or perhaps libraries) > see the boundaries between the systems? > > And when recieving a pointer from C, expand it. > > When passing a pointer to C, runtime check that it isn't a closure, > which doesn't work today anyway (it'd crash), and shrink it. > > For example, and maybe this is worth implementing to prove it out.. > You cannot pass a Modula-3 closure to stdlib.h/qsort. > You can pass a Modula-3 non-closure to stdlib.h/qsort. > If you then implement qsort in Modula-3, with the same apparent > mistake of "missing context", ideally it would accept > both context-free C and Modula-3 functions, and Modula-3 closures. > And it'd just work seamlessly, except for possible compile-time > or runtime errors for passing a closure to C. > > - Jay > > ________________________________ > From: Rodney M. Bates > Sent: Wednesday, March 10, 2021 4:09 PM > To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org > Cc: m3devel > Subject: Re: [M3devel] closures are a problem > > > > On 3/9/21 5:30 PM, Jay K wrote: > > To clarify and confirm, it does already work. > > > > Well, that makes me very confused. Why then did you propose to eliminate the > the feature (passing nested procedures as parameters) from Modula-3 and > follow it with a litany of reasons to do so and why you couldn't or didn't > want to implement it? > > Were you rolling? If so, it certainly worked on me. > > > > AMD64_NT works, from long ago. > > > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > > > RISCV64_LINUX works. It builds itself in qemu. > > Debugging is also ok not great. > > I can make it better. > > > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > > To fix the leaky conservative GC. > > > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > > AppleWatch is a tough one. > > > > WebAssembly I believe also will not work, > > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > > > Thank you, > > - Jay > > - > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026662066%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SwDerQFNqm%2BSrzBcV3Pb4Qs0WmCogmYwICuGu%2BmxrME%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026672062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUmEr%2FVcDkuO0bw0D5Puwr5T67aRlhyFZvohoEWY2gc%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 11 08:15:57 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 07:15:57 +0000 Subject: [M3devel] closures are a problem In-Reply-To: References: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> , <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com>, , Message-ID: Also, does anyone worry about the loss of atomiticy of loading/storing what was previously one word, now two? Can anyone speak to the experience and history of PowerPC calling conventions? PowerPC function pointers used to be a pointer to code/data pair, like this. But eventually that was revised away and now the are the "normal" single word pointers. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 11, 2021 6:34 AM To: Hendrik Boom ; m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem Repeating myself perhaps.. > And if you statically know that the procedure being called is a > top-level procedure (this is known for almost all procedure calle) > you don't need the environment pointer a all. > > Now if you're calling an external procedure using C calling conventions, > you just call it. The problem in my mind, is knowing all this perfectly. The current system does work "perfectly", with the caveats noted. It does not need to distinguish Modula-3 code from C code. "external" functions have the same calling convention as not-external. Taking the address of a non-nested Modula-3 function or C function always just produces one word. Taking the address of a nested Modula-3 function always produces a closure, I think. Perhaps m3front checks if the static link is used or not. Sometimes nesting is just a namespace construct. Calling a function pointer always goes through the if_closure dynamic check, so always accepts any of the three types. The one never-working scenario is passing the address of a nested function to C. There remains no good solution for that. (bad solutions include JIT, in heap or stack) If all C/Modula-3 boundaries are knowable, that does seem to offer like what you say. It means a function pointer is now always two words when Modula-3 passes to Modula-3, and little adapters are needed between C and Modula-3, in various combinations, so each can call pointers to the other, and each can take the address of the other. A function pointer would generally always be two words. Pass the parameters and the static link. The static link could be first or last. Does it hold together? - Jay ________________________________ From: Jay K Sent: Thursday, March 11, 2021 6:10 AM To: Hendrik Boom ; m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem > Is it even possible in Modula 3 to call a C procedure nonstatically? Yes I believe so. I believe it just works. Perhaps it is not considered useful? > I'd say, if you need the "address" of a C procedure you just > (at compile time) make a Modula 3 procedure that calls it > without taking the address, and return the two=word "address" of > the Modula 3 procedure instead. > > Is this hard? I think this is the start of a solution. But I am not sure it is complete. Or rather, that part of the solution is convincing. But more generally, how do you pass function pointers e.g. in Trestle to Win32 or Xlib? There is function pointer interop between Modula-3 and C and most scenarios work and are seamless. One scenario does not work and crashes. That is, C and non-nested Modula-3 functions interop fine. Address of nested Modula-3 function is the problem, as I have been saying. You end up being "stuck" with the single word C representation. Which is why we have this if_closure thing. I guess. Question becomes, what amount of interop and what amount of features are a worthwhile tradeoff, and how statically understood is the system, such as to enable workarounds? I mean, do you really always know that a function is C and that address-of means to generate a wrapper function that throws out the "static link" and passes on the rest of the parameters? Maybe. Similarly, if I pass a Modula-3 function address to C, what does that look like? Truncate the two words to one? Assert the second word is nil? Maybe this can work. And then, the function if_closure just goes away. Perhaps all quite nice. I don't mind going through those thin thunks, my only concern is two part: - Are all the boundaries knowable where type translation has to occur under the covers? - Does anyone compare these things for equality? Does that work? Why did the original implementors go with the single word + if_closure approach? Thank you, - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Thursday, March 11, 2021 12:29 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] closures are a problem On Wed, Mar 10, 2021 at 08:34:12PM +0000, Jay K wrote: > I thought I was repeatedly clear. > > It is a problem on systems that have execute-only memory. > Such as Android..well, I thought, and it was, but not > as much as I thought. > And other systems. > These systems "crash" in some form if you try to read code bytes. > > I believe it is problem on WebAssembly, but I will confirm soon. > > I suspect it is problem on ARM systems with the new "pointer authentication" feature. > My M1 Mac I think does not have that enabled. > It sounds like current iPhones work that way. I haven't tried. > I don't know if Apple is going to require that in future or not. > > It is ugly to have this -1 marker and believe that means not-code. > > It is not always necessarily a problem. > > It would be nice, perhaps, to remove their use in-tree and then clearly > reduce the problem. Though it would also be good to confirm > the WebAssembly situation if the system becomes available there, > so people know what they can do, > and WebAssembly might gain support eventually. > > There might be other ways to deal with this. > > In particular, could you maybe always pass around two words? > Or three words, the third being a type indicator? > (maybe the nil-ness or lower bit of the second word could be the > type indicator; the difference between 2 and 3 isn't important though). This is the standard way to implement nested procedures, invented way back in the 60's for Algol 60. Of course if you don't need the "address" of the procedure, its environment will be a subset ouf yours, and you don't need to make the two-word "address" You just load up the right register and do the call. Andif you statically know that the procedure being called is a top-level procedure (this is known for almost all procedure calle) you don't need the environment pointer a all. Now if you're calling an external procedure using C calling conventions, you just call it. Is it even possible in Modula 3 to call a C procedure nonstatically? I'd say, if you need the "address" of a C procedure you just (at compile time) make a Modula 3 procedure that calls it without taking the address, and return the two=word "address" of the Modula 3 procedure instead. Is this hard? -- hendrik > > If the system was all Modula-3 then yes. > > Given interoperation with C, maybe. > > Question in my mind becomes, can the compiler (or perhaps libraries) > see the boundaries between the systems? > > And when recieving a pointer from C, expand it. > > When passing a pointer to C, runtime check that it isn't a closure, > which doesn't work today anyway (it'd crash), and shrink it. > > For example, and maybe this is worth implementing to prove it out.. > You cannot pass a Modula-3 closure to stdlib.h/qsort. > You can pass a Modula-3 non-closure to stdlib.h/qsort. > If you then implement qsort in Modula-3, with the same apparent > mistake of "missing context", ideally it would accept > both context-free C and Modula-3 functions, and Modula-3 closures. > And it'd just work seamlessly, except for possible compile-time > or runtime errors for passing a closure to C. > > - Jay > > ________________________________ > From: Rodney M. Bates > Sent: Wednesday, March 10, 2021 4:09 PM > To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org > Cc: m3devel > Subject: Re: [M3devel] closures are a problem > > > > On 3/9/21 5:30 PM, Jay K wrote: > > To clarify and confirm, it does already work. > > > > Well, that makes me very confused. Why then did you propose to eliminate the > the feature (passing nested procedures as parameters) from Modula-3 and > follow it with a litany of reasons to do so and why you couldn't or didn't > want to implement it? > > Were you rolling? If so, it certainly worked on me. > > > > AMD64_NT works, from long ago. > > > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > > > RISCV64_LINUX works. It builds itself in qemu. > > Debugging is also ok not great. > > I can make it better. > > > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > > To fix the leaky conservative GC. > > > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > > AppleWatch is a tough one. > > > > WebAssembly I believe also will not work, > > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > > > Thank you, > > - Jay > > - > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026662066%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SwDerQFNqm%2BSrzBcV3Pb4Qs0WmCogmYwICuGu%2BmxrME%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026672062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUmEr%2FVcDkuO0bw0D5Puwr5T67aRlhyFZvohoEWY2gc%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 11 09:18:57 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 08:18:57 +0000 Subject: [M3devel] LONGINT ok to use now in cm3? Message-ID: Is it ok to use LONGINT in cm3/m3middle/m3back etc.? stuff like this to start: Searching for 'LONGINT'... C:\s\cm3\m3-sys\m3middle\src\CoffTime.c(21):INTEGER /* should be LONGINT */ C:\s\cm3\m3-sys\m3middle\src\CoffTime.c(30):INTEGER /* should be LONGINT */ C:\s\cm3\m3-sys\m3middle\src\CoffTime.c(46):INTEGER /* should be LONGINT */ C:\s\cm3\m3-sys\m3middle\src\POSIX\CoffTime.c(11):INTEGER /* should be LONGINT */ C:\s\cm3\m3-sys\m3middle\src\POSIX\CoffTime.c(22):INTEGER /* should be LONGINT */ LONGINT support has been in for many years, including in NT386 (and even then, it got the syntactic support right away, just that they were initiallly 32bits, long ago.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Thu Mar 11 18:56:02 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 11 Mar 2021 11:56:02 -0600 Subject: [M3devel] SocketPosixC.c won't compile Message-ID: <040628c8-fc62-5457-d607-8a5452e8162c@lcwb.coop> SocketPosixC.c won't compile: === package /home/rodney/proj/m3/git-merge/cm3/m3-libs/libm3 === +++ cm3 -build -DROOT='/home/rodney/proj/m3/git-merge/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/rodney/proj/m3/git-merge/cm3' +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides new source -> compiling SocketPosixC.c ../src/os/POSIX/SocketPosixC.c:32:19: error: uninitialized const ?refetchError? [-fpermissive] static const char refetchError; ^ ../src/os/POSIX/SocketPosixC.c:635:1: error: expected declaration before ?}? token } /* extern C */ ^ compile_c => 1 C compiler failed compiling: ../src/os/POSIX/SocketPosixC.c compilation failed => not building library "libm3.a" Fatal Error: package build failed *** execution of cm3 -build -DROOT='/home/rodney/proj/m3/git-merge/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/rodney/proj/m3/git-merge/cm3' failed *** This is : Latest commit 800b2c2 4 days ago -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Thu Mar 11 20:03:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 11 Mar 2021 19:03:31 +0000 Subject: [M3devel] SocketPosixC.c won't compile In-Reply-To: <040628c8-fc62-5457-d607-8a5452e8162c@lcwb.coop> References: <040628c8-fc62-5457-d607-8a5452e8162c@lcwb.coop> Message-ID: Sorry about that, it is fixed now. We should setup some CI/CD. - Jay ________________________________ From: M3devel on behalf of Rodney M. Bates Sent: Thursday, March 11, 2021 5:56 PM To: m3devel Subject: [M3devel] SocketPosixC.c won't compile SocketPosixC.c won't compile: === package /home/rodney/proj/m3/git-merge/cm3/m3-libs/libm3 === +++ cm3 -build -DROOT='/home/rodney/proj/m3/git-merge/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/rodney/proj/m3/git-merge/cm3' +++ --- building in AMD64_LINUX --- ignoring ../src/m3overrides new source -> compiling SocketPosixC.c ../src/os/POSIX/SocketPosixC.c:32:19: error: uninitialized const ?refetchError? [-fpermissive] static const char refetchError; ^ ../src/os/POSIX/SocketPosixC.c:635:1: error: expected declaration before ?}? token } /* extern C */ ^ compile_c => 1 C compiler failed compiling: ../src/os/POSIX/SocketPosixC.c compilation failed => not building library "libm3.a" Fatal Error: package build failed *** execution of cm3 -build -DROOT='/home/rodney/proj/m3/git-merge/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/rodney/proj/m3/git-merge/cm3' failed *** This is : Latest commit 800b2c2 4 days ago -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C8d0bee2c71c543b0c51708d8e4b70199%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510821953999298%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1%2BQ30n3P9I4Jm5LbVKGnZv1hs4FjQn%2BijtLwI%2BSI%2BLU%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Thu Mar 11 20:12:56 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 11 Mar 2021 14:12:56 -0500 Subject: [M3devel] closures are a problem In-Reply-To: References: <7eca05e7-ace0-ae36-4dd8-d0e99afc3b51@lcwb.coop> <20210311002935.yuwwrzeswq4z5pid@topoi.pooq.com> Message-ID: <20210311191256.kqzgum7v5np45yks@topoi.pooq.com> On Thu, Mar 11, 2021 at 06:10:06AM +0000, Jay K wrote: > > Is it even possible in Modula 3 to call a C procedure nonstatically? > > Yes I believe so. > I believe it just works. > Perhaps it is not considered useful? > > > I'd say, if you need the "address" of a C procedure you just > > (at compile time) make a Modula 3 procedure that calls it > > without taking the address, and return the two=word "address" of > > the Modula 3 procedure instead. > > > > Is this hard? > > I think this is the start of a solution. > But I am not sure it is complete. > > Or rather, that part of the solution is convincing. > > But more generally, how do you pass function pointers e.g. in Trestle to Win32 or Xlib? > > There is function pointer interop between Modula-3 and C > and most scenarios work and are seamless. One scenario > does not work and crashes. > > That is, C and non-nested Modula-3 functions interop fine. > Address of nested Modula-3 function is the problem, as I have been saying. > > You end up being "stuck" with the single word C representation. > Which is why we have this if_closure thing. I guess. > > Question becomes, what amount of interop and what amount of features are a worthwhile tradeoff, > and how statically understood is the system, such as to enable workarounds? > I mean, do you really always know that a function is C and that address-of means to > generate a wrapper function that throws out the "static link" and passes on the rest > of the parameters? Maybe. > > Similarly, if I pass a Modula-3 function address to C, what does that look like? > Truncate the two words to one? Assert the second word is nil? IIRC, it is forbidden to pass a nested Modula-3 function address to C. Anybody know? This is a shortcoming of C, not of Modula 3. C programmers have been using the kludge, when necessary, of passing a void* pointer to the function they are calling when their programming gets to the point that they'd use closures ofnested functions in other languages. It is messy, and there is not a conventional, standard way to do this. I do not know whether or how the present Modula 3 checks at run time whether a function pointer of unknow origin can be passed to an external C program. With the two=part functino pointers, it could fill in the environment part of a pointer with NULL in the case of a top-level or external C function, and check it is NULL when passing it out to a C function. > > Maybe this can work. > > And then, the function if_closure just goes away. Perhaps all quite nice. > > I don't mind going through those thin thunks, my only concern is two part: > > - Are all the boundaries knowable where type translation has to occur under the covers? > > - Does anyone compare these things for equality? Does that work? > > Why did the original implementors go with the single word + if_closure approach? > > Thank you, > - Jay > > ________________________________ > From: M3devel on behalf of Hendrik Boom > Sent: Thursday, March 11, 2021 12:29 AM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] closures are a problem > > On Wed, Mar 10, 2021 at 08:34:12PM +0000, Jay K wrote: > > I thought I was repeatedly clear. > > > > It is a problem on systems that have execute-only memory. > > Such as Android..well, I thought, and it was, but not > > as much as I thought. > > And other systems. > > These systems "crash" in some form if you try to read code bytes. > > > > I believe it is problem on WebAssembly, but I will confirm soon. > > > > I suspect it is problem on ARM systems with the new "pointer authentication" feature. > > My M1 Mac I think does not have that enabled. > > It sounds like current iPhones work that way. I haven't tried. > > I don't know if Apple is going to require that in future or not. > > > > It is ugly to have this -1 marker and believe that means not-code. > > > > It is not always necessarily a problem. > > > > It would be nice, perhaps, to remove their use in-tree and then clearly > > reduce the problem. Though it would also be good to confirm > > the WebAssembly situation if the system becomes available there, > > so people know what they can do, > > and WebAssembly might gain support eventually. > > > > There might be other ways to deal with this. > > > > In particular, could you maybe always pass around two words? > > Or three words, the third being a type indicator? > > (maybe the nil-ness or lower bit of the second word could be the > > type indicator; the difference between 2 and 3 isn't important though). > > This is the standard way to implement nested procedures, invented way > back in the 60's for Algol 60. > > Of course if you don't need the "address" of the procedure, its environment > will be a subset ouf yours, and you don't need to make the two-word "address" > You just load up the right register and do the call. > > Andif you statically know that the procedure being called is a > top-level procedure (this is known for almost all procedure calle) > you don't need the environment pointer a all. > > Now if you're calling an external procedure using C calling conventions, > you just call it. > > Is it even possible in Modula 3 to call a C procedure nonstatically? > I'd say, if you need the "address" of a C procedure you just (at > compile time) make a Modula 3 procedure that calls it without taking > the address, and return the two=word "address" of the Modula 3 procedure > instead. > > Is this hard? > > -- hendrik > > > > > If the system was all Modula-3 then yes. > > > > Given interoperation with C, maybe. > > > > Question in my mind becomes, can the compiler (or perhaps libraries) > > see the boundaries between the systems? > > > > And when recieving a pointer from C, expand it. > > > > When passing a pointer to C, runtime check that it isn't a closure, > > which doesn't work today anyway (it'd crash), and shrink it. > > > > For example, and maybe this is worth implementing to prove it out.. > > You cannot pass a Modula-3 closure to stdlib.h/qsort. > > You can pass a Modula-3 non-closure to stdlib.h/qsort. > > If you then implement qsort in Modula-3, with the same apparent > > mistake of "missing context", ideally it would accept > > both context-free C and Modula-3 functions, and Modula-3 closures. > > And it'd just work seamlessly, except for possible compile-time > > or runtime errors for passing a closure to C. > > > > - Jay > > > > ________________________________ > > From: Rodney M. Bates > > Sent: Wednesday, March 10, 2021 4:09 PM > > To: Jay K ; Mika Nystrom ; rodney.m.bates at acm.org > > Cc: m3devel > > Subject: Re: [M3devel] closures are a problem > > > > > > > > On 3/9/21 5:30 PM, Jay K wrote: > > > To clarify and confirm, it does already work. > > > > > > > Well, that makes me very confused. Why then did you propose to eliminate the > > the feature (passing nested procedures as parameters) from Modula-3 and > > follow it with a litany of reasons to do so and why you couldn't or didn't > > want to implement it? > > > > Were you rolling? If so, it certainly worked on me. > > > > > > > AMD64_NT works, from long ago. > > > > > > ARM64_DARWIN works on the M1 MacBook. (I probably skipped gui at the time but it should work). > > > > > > RISCV64_LINUX works. It builds itself in qemu. > > > Debugging is also ok not great. > > > I can make it better. > > > > > > I believe I tested many of the ?old? systems too. We can/should delete the gcc backend. Really. > > > > > > Almost anything you can think of probably just works. It has become a trivial exercise to port now. A few I didn?t do but would be trivial include s390x and m68k. > > > > > > IA64 is harder. Because I think no qemu, and the ?two stacks?. This is another area I wish for a better and more portable solution. I.e. record precise locations of locals. > > > To fix the leaky conservative GC. > > > > > > Oh and, don?t use setjmp. It isn?t portable, like to AppleWatch. > > > But portability and performance are hard to combine here...you end up needing a lot thread local gyrations to get the GC roots. > > > AppleWatch is a tough one. > > > > > > WebAssembly I believe also will not work, > > > due to closures. But I should try. At least not multithreaded WebAssembly. Single threaded WebAssembly can pass the static link in a global, yuck. I would like a better if_closure than the read for -1 also. Ideas? > > > > > > Thank you, > > > - Jay > > > - > > -- > > Rodney Bates > > rodney.m.bates at acm.org > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026662066%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SwDerQFNqm%2BSrzBcV3Pb4Qs0WmCogmYwICuGu%2BmxrME%3D&reserved=0 > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cfcfe91a7fe7948294a9c08d8e424cdc6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637510194026672062%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gUmEr%2FVcDkuO0bw0D5Puwr5T67aRlhyFZvohoEWY2gc%3D&reserved=0 From jayk123 at hotmail.com Fri Mar 12 08:26:54 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 07:26:54 +0000 Subject: [M3devel] NT386 with current Visual C++? Message-ID: Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 12 09:45:02 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 08:45:02 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: Message-ID: Hm, the problem seems to be with the 2008 toolset also. Building with -MT instead of -MD fixes it. Hopefully that isn't the real fix. ? I'll dig a bit more. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Friday, March 12, 2021 7:26 AM To: m3devel at elegosoft.com Subject: [M3devel] NT386 with current Visual C++? Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 12 14:04:59 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 13:04:59 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: , Message-ID: Um, it seems maybe the alloca of jmpbufs isn't always done? I'll look looking. - Jay ________________________________ From: Jay K Sent: Friday, March 12, 2021 8:45 AM To: m3devel at elegosoft.com Subject: Re: NT386 with current Visual C++? Hm, the problem seems to be with the 2008 toolset also. Building with -MT instead of -MD fixes it. Hopefully that isn't the real fix. ? I'll dig a bit more. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Friday, March 12, 2021 7:26 AM To: m3devel at elegosoft.com Subject: [M3devel] NT386 with current Visual C++? Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Fri Mar 12 14:24:52 2021 From: vvm at tut.by (vvm at tut.by) Date: Fri, 12 Mar 2021 16:24:52 +0300 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: , Message-ID: <446221615555013@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 12 14:28:38 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 13:28:38 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: <446221615555013@mail.yandex.by> References: , , <446221615555013@mail.yandex.by> Message-ID: alloca: I don't know. It seems surprising. but that is what I seem to see. Like, m3front isn't complete? - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 12, 2021 1:24 PM To: Jay K ; m3devel at elegosoft.com Subject: Re: [M3devel] NT386 with current Visual C++? Hi! } alloca of jmpbufs isn't always done? With cm3-all-AMD64_NT-d5.11.0-VC110(2012)-2021-03-10_15-11 based cm3-1956ccc1068bc424241df0b4b806beb980e759b4 all work good, but with new config-no-install\NT.common With old "config-no-install\NT.common" we have old problem. ( I'll send path later) Best regards, Victor Miasnikov 12.03.2021, 16:05, "Jay K" : Um, it seems maybe the alloca of jmpbufs isn't always done? I'll look looking. - Jay ________________________________ From: Jay K > Sent: Friday, March 12, 2021 8:45 AM To: m3devel at elegosoft.com > Subject: Re: NT386 with current Visual C++? Hm, the problem seems to be with the 2008 toolset also. Building with -MT instead of -MD fixes it. Hopefully that isn't the real fix. ? I'll dig a bit more. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Friday, March 12, 2021 7:26 AM To: m3devel at elegosoft.com > Subject: [M3devel] NT386 with current Visual C++? Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Fri Mar 12 14:36:47 2021 From: vvm at tut.by (vvm at tut.by) Date: Fri, 12 Mar 2021 16:36:47 +0300 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: , , <446221615555013@mail.yandex.by> Message-ID: <431731615555862@mail.yandex.by> An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Fri Mar 12 16:50:24 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Fri, 12 Mar 2021 10:50:24 -0500 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: Message-ID: <20210312155024.eulii5s3xcln5sp2@topoi.pooq.com> On Fri, Mar 12, 2021 at 01:04:59PM +0000, Jay K wrote: > Um, it seems maybe the alloca of jmpbufs isn't always done? Just curious -- is alloca part of the C standard now? A few years (or is it decades) ago it wasn't, but just happened to be implemented by most systems. -- hendrik > I'll look looking. > > - Jay From jayk123 at hotmail.com Fri Mar 12 19:05:17 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 18:05:17 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: <20210312155024.eulii5s3xcln5sp2@topoi.pooq.com> References: , <20210312155024.eulii5s3xcln5sp2@topoi.pooq.com> Message-ID: It is not. But it is very widely available as you say. And setjmp is standard but not portable. And we use thread locals, and networking, and gui. Where is the line between what is we are allowed to use and not? Ultimately the alloca and setjmp will hopefully be replaced by C++ try, for much greater efficiency, and even more portability (it should work on AppleWatch where setjmp does not). - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Friday, March 12, 2021 3:50 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] NT386 with current Visual C++? On Fri, Mar 12, 2021 at 01:04:59PM +0000, Jay K wrote: > Um, it seems maybe the alloca of jmpbufs isn't always done? Just curious -- is alloca part of the C standard now? A few years (or is it decades) ago it wasn't, but just happened to be implemented by most systems. -- hendrik > I'll look looking. > > - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cbc5304a3a9004d4b919908d8e56e99b7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637511610491298091%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tTER776n8M4fOAiLgWDUCO2I5wkUwb0inuZGbPuNb1U%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 12 19:05:55 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 18:05:55 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: <431731615555862@mail.yandex.by> References: , , <446221615555013@mail.yandex.by> , <431731615555862@mail.yandex.by> Message-ID: The intent is to always call alloca, but maybe m3front does not. Maybe. I will look more. - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 12, 2021 1:36 PM To: Jay K ; m3devel at elegosoft.com Subject: Re: [M3devel] NT386 with current Visual C++? Hi! ( Sorry I have not needed files now... and should write without details...) Please, try with this: == - Alloca_jmpbuf := TRUE; + Alloca_jmpbuf := FALSE; == but on cm3-c62c53ddfc204b0623db343469db6418d37e527d and even early version or reverse some patches . . . V.M. 12.03.2021, 16:28, "Jay K" : alloca: I don't know. It seems surprising. but that is what I seem to see. Like, m3front isn't complete? - Jay ________________________________ From: vvm at tut.by > Sent: Friday, March 12, 2021 1:24 PM To: Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] NT386 with current Visual C++? Hi! } alloca of jmpbufs isn't always done? With cm3-all-AMD64_NT-d5.11.0-VC110(2012)-2021-03-10_15-11 based cm3-1956ccc1068bc424241df0b4b806beb980e759b4 all work good, but with new config-no-install\NT.common With old "config-no-install\NT.common" we have old problem. ( I'll send path later) Best regards, Victor Miasnikov 12.03.2021, 16:05, "Jay K" >: Um, it seems maybe the alloca of jmpbufs isn't always done? I'll look looking. - Jay ________________________________ From: Jay K > Sent: Friday, March 12, 2021 8:45 AM To: m3devel at elegosoft.com > Subject: Re: NT386 with current Visual C++? Hm, the problem seems to be with the 2008 toolset also. Building with -MT instead of -MD fixes it. Hopefully that isn't the real fix. ? I'll dig a bit more. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Friday, March 12, 2021 7:26 AM To: m3devel at elegosoft.com > Subject: [M3devel] NT386 with current Visual C++? Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 12 19:44:39 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 18:44:39 +0000 Subject: [M3devel] NT386 with current Visual C++? In-Reply-To: References: , , <446221615555013@mail.yandex.by> , <431731615555862@mail.yandex.by>, Message-ID: Hm maybe false alarm. Maybe an old broken bootstrap/release or something. Bootstrap from Linux via boot1 to NT386 using integrated backend worked. To 2008 tools. I will try current tools also then. And produce a working NT386 .msi hopefully. - Jay ________________________________ From: Jay K Sent: Friday, March 12, 2021 6:05 PM To: vvm at tut.by ; m3devel at elegosoft.com Subject: Re: [M3devel] NT386 with current Visual C++? The intent is to always call alloca, but maybe m3front does not. Maybe. I will look more. - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 12, 2021 1:36 PM To: Jay K ; m3devel at elegosoft.com Subject: Re: [M3devel] NT386 with current Visual C++? Hi! ( Sorry I have not needed files now... and should write without details...) Please, try with this: == - Alloca_jmpbuf := TRUE; + Alloca_jmpbuf := FALSE; == but on cm3-c62c53ddfc204b0623db343469db6418d37e527d and even early version or reverse some patches . . . V.M. 12.03.2021, 16:28, "Jay K" : alloca: I don't know. It seems surprising. but that is what I seem to see. Like, m3front isn't complete? - Jay ________________________________ From: vvm at tut.by > Sent: Friday, March 12, 2021 1:24 PM To: Jay K >; m3devel at elegosoft.com > Subject: Re: [M3devel] NT386 with current Visual C++? Hi! } alloca of jmpbufs isn't always done? With cm3-all-AMD64_NT-d5.11.0-VC110(2012)-2021-03-10_15-11 based cm3-1956ccc1068bc424241df0b4b806beb980e759b4 all work good, but with new config-no-install\NT.common With old "config-no-install\NT.common" we have old problem. ( I'll send path later) Best regards, Victor Miasnikov 12.03.2021, 16:05, "Jay K" >: Um, it seems maybe the alloca of jmpbufs isn't always done? I'll look looking. - Jay ________________________________ From: Jay K > Sent: Friday, March 12, 2021 8:45 AM To: m3devel at elegosoft.com > Subject: Re: NT386 with current Visual C++? Hm, the problem seems to be with the 2008 toolset also. Building with -MT instead of -MD fixes it. Hopefully that isn't the real fix. ? I'll dig a bit more. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Friday, March 12, 2021 7:26 AM To: m3devel at elegosoft.com > Subject: [M3devel] NT386 with current Visual C++? Has anyone run integrated NT386 backend with current Visual C++? I think it is broken in a tricky way. In particular you can no longer simply call the function setjmp. There is a compiler intrinsic. And if you don't call that, then longjmp fails. amd64_nt ran into this too, but it was easy to fix there. i.e. include setjmp.h instead of declaring setjmp ourselves. ffe4.60cc): Unknown exception - code 80000026 (!!! second chance !!!) VCRUNTIME140!___longjmp_internal+0x4e: 68f52f4e 5e pop esi 0:000> k ChildEBP RetAddr 016ff5a0 007f0876 VCRUNTIME140!___longjmp_internal+0x4e 016ff5cc 007f0621 cm3!MxConfig__FindConfig+0x3a 016ff5f0 00764df7 cm3!MxConfig__FindFile+0x91 016ff6b0 00765a03 cm3!Main__DoIt+0x1f4 016ff6bc 00a02051 cm3!Main_M3+0x1b 016ff700 00a015d2 cm3!RTLinker__RunMainBody+0x254 016ff718 00a01695 cm3!RTLinker__AddUnitI+0xf7 016ff738 00a482b3 cm3!RTLinker__AddUnit+0xbb 016ff744 00a4e46b cm3!main+0x23 (Inline) -------- cm3!invoke_main+0x1c 016ff78c 761afa29 cm3!__scrt_common_main_seh+0xfa 016ff79c 77bd76b4 KERNEL32!BaseThreadInitThunk+0x19 016ff7f8 77bd7684 ntdll!__RtlUserThreadStart+0x2f 016ff808 00000000 ntdll!_RtlUserThreadStart+0x1b 0:000> !error 80000026 Error code: (NTSTATUS) 0x80000026 (2147483686) - A long jump has been executed. I'll see if I can do something about it. Even if I can, there is always the problem of knowing what version we are using and acting accordingly. :( This is actually an advantage of the C backend. setjmp/longjmp, like so much else, are specified as source constructs, not object constructs. cm3 violates this abstraction boundary, a lot, which is a lot of what m3core/src/unix used have to deal with and somewhat still does. i.e. it still sometimes has to workaround "functions" being macros and still sometimes assumes they are not. (I can improve that..) This abstraction boundary violation is why this breaks. We assume it is simple callable function and not a special compiler intrinsic and we just output a call to it. Maybe for tonight's exercise of rebuilding NT386 I just use old Visual C++. :( We cannot wrap setjmp in a C layer like we do for other functions. It must occur at the site for longjmp to return. I don't really like the NT386 "platform" because of vagaries around suspending and getting context for threads that could be in native code, though there is a fix for that if you look at CoreCLR. Wow64 isn't compatible with the historical pattern here. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri Mar 12 20:49:04 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Mar 2021 13:49:04 -0600 Subject: [M3devel] Change in stdout Message-ID: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop> the compiler is now writing this line: g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o to stdout, which it formerly did not. It makes every compiler test case get a false failure, because stdout does not agree with what the test case has stored as expected output. This turns real failures into needles in a haystack, and makes it impossibly difficult to find then among the chaff. If all the expected outputs in the tests were changed, that would create an equivalent problem when running with an older compiler, and looking for changes old-to-new compiler would then be impossibly difficult. Can we get rid of it altogether, or make it somehow optional? -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Fri Mar 12 21:32:15 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 12 Mar 2021 14:32:15 -0600 Subject: [M3devel] Change in stdout In-Reply-To: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop> References: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop> Message-ID: On 3/12/21 1:49 PM, Rodney M. Bates wrote: > the compiler is now writing this line: > > ? g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o > > to stdout, which it formerly did not. > > It makes every compiler test case get a false failure, because stdout > does not agree with what the test case has stored as expected output. > This turns real failures into needles in a haystack, and makes it > impossibly difficult to find then among the chaff. > > If all the expected outputs in the tests were changed, that would > create an equivalent problem when running with an older compiler, > and looking for changes old-to-new compiler would then be impossibly > difficult. > > Can we get rid of it altogether, or make it somehow optional? Probably, it should appear only if -commands option to cm3. -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Sat Mar 13 00:38:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 12 Mar 2021 23:38:06 +0000 Subject: [M3devel] Change in stdout In-Reply-To: References: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop>, Message-ID: I like it. But I know there is a balance to find, and that maybe didn't find it yet, and that the tests can/should measure output. I am not super happy with "commands" and/or verbose, because, well, the balance. It issues all the "rm .generic.tmp" stuff. How about, perhaps..it is a little wonky, but there is the -verbose and -verbose -verbose pattern? Or, so I can get my default, -quiet for tests? Which begs the question though -- if not for tests, which I acknowledge, are people ok with this level of noisiness otherwise? You know, if not for the tests, is it not too noisy? 'cause then we can go with -quiet for tests, and I can even ratchet up the default? I like it because, I'm not sure. I debug a lot of stuff. Logs are often helpful. And then, I understand, there is never the right amount of logging. When things work, which is often, there is too much logging. When things fail, which is often, there is too little logging. Another pattern I have seen, but less often, and takes a bit of work and another bit of "negotation" (amongst us humans) is to always be terse to stdout and always be verbose to a file. I have worked a lot with such systems. The remaining controversy then becomes: - with an outlet for which it is ok to be verbose, the log file, it can get too noisy, large logs - The system I use drops the logs in current working directory by default. But configurable. So as you cd around and make, in this highly recursive make, you do end up log files strewn all over. Which isn't great but is imho tolerable and worth that it resolves the larger issue. So to be concrete, would it be ok if cm3 drops cm3.log every time it runs. Create, not append. ? The existance and path of the log should be optional, but a default to do it would be nice. Or driven by environment variables, perhaps. - Jay ________________________________ From: M3devel on behalf of Rodney M. Bates Sent: Friday, March 12, 2021 8:32 PM To: m3devel Subject: Re: [M3devel] Change in stdout On 3/12/21 1:49 PM, Rodney M. Bates wrote: > the compiler is now writing this line: > > g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o > > to stdout, which it formerly did not. > > It makes every compiler test case get a false failure, because stdout > does not agree with what the test case has stored as expected output. > This turns real failures into needles in a haystack, and makes it > impossibly difficult to find then among the chaff. > > If all the expected outputs in the tests were changed, that would > create an equivalent problem when running with an older compiler, > and looking for changes old-to-new compiler would then be impossibly > difficult. > > Can we get rid of it altogether, or make it somehow optional? Probably, it should appear only if -commands option to cm3. -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cf03981a8666c4651fb7808d8e5960187%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637511779727266499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sKcGBCT5KjiefA6f1E4OGEslQAUW7t%2Fm60MccVCKP9U%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 05:53:40 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 04:53:40 +0000 Subject: [M3devel] caltech even on Linux/amd64.. Message-ID: Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 06:30:18 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 05:30:18 +0000 Subject: [M3devel] Change in stdout In-Reply-To: References: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop>, , Message-ID: I was thinking of command line options like -verboseexec and -quietexec specifically to control command line echoing, but I'm not sure I like that. How about something really crude simple effective cm3 -DAT= vs. cm3 -DAT=@ Too low level or just right? or cm3 -DNOECHO= vs. cm3 -DNOECHO=@ I don't love options that start "no", because they can lead to confusing double negatives. cm3 -DQUIET=@ vs. cm3 -DQUIET= So, when you form a command you want usually hidden, you put @ on it. Like today. When you form a command you want shown and hard to hide, you put nothing on it. Like today, but don't do that. When you form a command that I want to see but you don't, you prefix it with QUIET. Quake will say: if not defind("QUIET") QUIET = "" % empty, do not be quiet end ? And people that really don't like that, can say: QUIET = "@" and the tests specifically can say that since they depend on it. I have to verify that command line defines are not read only. We should probably also probably make sure that arbitrary runs of @ are removed. So people can mess it up and put it in multiple commands. Breaking the ability to run commands that actually start "@", if that is ok. I admit this remains very crude -- the entire idea of "@" at the start of a command having special meaning. "If it is similar to make, or cmd, it might not be such a good idea!". Which leads me back to -quietexec and -showexec ???? But we cannot remove the meaning of @ w/o breaking things, alas. Embrace it? Maybe the double @ thing is bogus too. I'll try to get to it soon. So much to do.. - Jay ________________________________ From: Jay K Sent: Friday, March 12, 2021 11:38 PM To: m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Change in stdout I like it. But I know there is a balance to find, and that maybe didn't find it yet, and that the tests can/should measure output. I am not super happy with "commands" and/or verbose, because, well, the balance. It issues all the "rm .generic.tmp" stuff. How about, perhaps..it is a little wonky, but there is the -verbose and -verbose -verbose pattern? Or, so I can get my default, -quiet for tests? Which begs the question though -- if not for tests, which I acknowledge, are people ok with this level of noisiness otherwise? You know, if not for the tests, is it not too noisy? 'cause then we can go with -quiet for tests, and I can even ratchet up the default? I like it because, I'm not sure. I debug a lot of stuff. Logs are often helpful. And then, I understand, there is never the right amount of logging. When things work, which is often, there is too much logging. When things fail, which is often, there is too little logging. Another pattern I have seen, but less often, and takes a bit of work and another bit of "negotation" (amongst us humans) is to always be terse to stdout and always be verbose to a file. I have worked a lot with such systems. The remaining controversy then becomes: - with an outlet for which it is ok to be verbose, the log file, it can get too noisy, large logs - The system I use drops the logs in current working directory by default. But configurable. So as you cd around and make, in this highly recursive make, you do end up log files strewn all over. Which isn't great but is imho tolerable and worth that it resolves the larger issue. So to be concrete, would it be ok if cm3 drops cm3.log every time it runs. Create, not append. ? The existance and path of the log should be optional, but a default to do it would be nice. Or driven by environment variables, perhaps. - Jay ________________________________ From: M3devel on behalf of Rodney M. Bates Sent: Friday, March 12, 2021 8:32 PM To: m3devel Subject: Re: [M3devel] Change in stdout On 3/12/21 1:49 PM, Rodney M. Bates wrote: > the compiler is now writing this line: > > g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o > > to stdout, which it formerly did not. > > It makes every compiler test case get a false failure, because stdout > does not agree with what the test case has stored as expected output. > This turns real failures into needles in a haystack, and makes it > impossibly difficult to find then among the chaff. > > If all the expected outputs in the tests were changed, that would > create an equivalent problem when running with an older compiler, > and looking for changes old-to-new compiler would then be impossibly > difficult. > > Can we get rid of it altogether, or make it somehow optional? Probably, it should appear only if -commands option to cm3. -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Cf03981a8666c4651fb7808d8e5960187%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637511779727266499%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sKcGBCT5KjiefA6f1E4OGEslQAUW7t%2Fm60MccVCKP9U%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 06:30:55 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 05:30:55 +0000 Subject: [M3devel] caltech even on Linux/amd64.. In-Reply-To: References: Message-ID: I expect I will be disabling this stuff shorty, sorry. I suppose we could repair the assembly. But I don't like the idea of checked in generated assembly either. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, March 13, 2021 4:53 AM To: Mika Nystrom ; m3devel Subject: [M3devel] caltech even on Linux/amd64.. Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 07:04:34 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 06:04:34 +0000 Subject: [M3devel] paths .m3 => mg? Message-ID: cm3: ../AMD64_LINUX/FinDateArraySort.m3 => /cm3/pkg/libm3/src/sort/ArraySort.mg:86:0: warning: '' may be used uninitialized in this function [-Wmaybe-uninitialized] Nevermind the actual warning. The "foo.m3 => bar.mg" is one source file path, synthesized by m3front. That entire strong, with two embedded paths, is considered one path. Why? Is this useful? What were they thinking? The .m3 is what is being compiled. But is just one line, is not useful to the programmer receiving warnings/errors/assertion failures. The .mg is a "real" useful source file. If that was the only thing printed, I think people would be satisified. It is a *little* useful to see the other line to see the generic parameter. Perhaps?? So why the overall thing? I am guessing, that it is a problem, for the same source file to be compiled more than once?? So some "form" of the right is needed?? Or maybe the "little useful" part?? C systems have to deal with this anyway: echo "int name() { return value ;}" > foo.c cc -c -Dname=one -Dvalue=1 foo.c cc -c -Dname=two -Dvalue=2 foo.c cc -c -Dname=three -Dvalue=3oops foo.c => error foo.c:"3oops" and the ambiguity of errors in foo.c, if you don't see the command line, well, nevermind? But I guess that has no bearing on Modula3 and perhaps the same file compiled twice would confuse the build system? So I guess I'll mostly leave this alone. I have to change it, somewhat, to unify C targets, i.e. the "AMD64_LINUX" part is bad. I can just remove it or change it to or , for the C backend only. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sat Mar 13 15:29:40 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 13 Mar 2021 09:29:40 -0500 Subject: [M3devel] Change in stdout In-Reply-To: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop> References: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop> Message-ID: <20210313142939.ydi6t5cdkzxkyrnv@topoi.pooq.com> On Fri, Mar 12, 2021 at 01:49:04PM -0600, Rodney M. Bates wrote: > the compiler is now writing this line: > > g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o > > to stdout, which it formerly did not. What is the purpose of writing this line? -- hendrik From jayk123 at hotmail.com Sat Mar 13 16:27:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 15:27:31 +0000 Subject: [M3devel] Change in stdout In-Reply-To: <20210313142939.ydi6t5cdkzxkyrnv@topoi.pooq.com> References: <902b7aa2-4efa-d413-f6b9-8d39470f8198@lcwb.coop>, <20210313142939.ydi6t5cdkzxkyrnv@topoi.pooq.com> Message-ID: Each specific line has very little value on its own, esp. when things are working. What is the value of saying "new source foo.m3"? It is sort of progress indicator, I suppose? When things fail, you want to know what ran. And then some lines become more valuable. It is hard to predict what will be valuable. So you log preemptively. One common strategy upon failure is to rerun commands manually. cm3 could report the last line upon failure, but one line might not be enough, and that could still break test scenarios that want to check for specific errors. Dropping cm3.log in current working directory has an obvious improvement, we could drop it in the output directory. Per package. That has advantages and disadvantages. The logs become smaller and part of the "realclean" cleanup. But then you do not have a unified log, which can be useful. - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Saturday, March 13, 2021 2:29 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] Change in stdout On Fri, Mar 12, 2021 at 01:49:04PM -0600, Rodney M. Bates wrote: > the compiler is now writing this line: > > g++ -g -m64 -fPIC -g -c -xc++ _m3main.c -o _m3main.o > > to stdout, which it formerly did not. What is the purpose of writing this line? -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C17c79bd6b472409bd21408d8e62c7bf0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637512426028417672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=y3WIvv5I2LpydftnICKCA3B8l5ZZxtt2nCSDC5exCEU%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 16:51:57 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 15:51:57 +0000 Subject: [M3devel] caltech even on Linux/amd64.. In-Reply-To: References: , Message-ID: aha. I see that now too. -gstabs+ produces demonstrably incorrect symbols on "current" gcc on Linux. Current shipping by Debian and/or Ubuntu. Not current head of gcc. Gcc developers are clearly not a fan of this switch. They want to remove. I don't think it was ever really needed, but added because it was thought to be relevant to the stabs used by m3cc. It doesn't even seem to be needed for linking m3cc+stabs output. But now we are dependent on it, in this one directory. We have some provision for different commands for assembling than compiling C, but that doesn't actually work today. We could see if "ends in .s" and then use -gstabs+. I have to see if Quake offers "ends with". We can add it. We could try and if failure, try again with -gstabs+. That is a little gross, and inefficient if the failure is something else, but will work. Would it help if we required gfortran? And I acknowledge this is much ado about little. Would it be horrible to disable. Maybe regenerate the files w/o symbols? - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 3:20 PM To: Jay K Cc: m3devel Subject: Re: caltech even on Linux/amd64.. yeah without the -gstabs+ I get the same error as you. I'm assuming it comes from here (cm3/bin/config/AMD64_LINUX) but am not sure SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" % C compiler On Sat, Mar 13, 2021 at 7:12 AM Mika Nystrom > wrote: Here is the command in my build: new source -> compiling update.s gcc -gstabs+ -m64 -fPIC -g -c ../src/update.s -o update.o I thought there was a problem not using -gstabs+ ? (Not related to compiling my hacked-up Fortran-assembly...) On Sat, Mar 13, 2021 at 6:52 AM Mika Nystrom > wrote: The ultimate source is in there too. But it's in Fortran and we don't have that enabled in the quake. I was thinking that the right way to do it would be to allow f_source in quake, but you know, projects... time commitments... Not sure why the .s isn't working for you. I compiled with ifort and it's worked on a number of systems for me. Mika On Fri, Mar 12, 2021, 9:30 PM Jay K > wrote: I expect I will be disabling this stuff shorty, sorry. I suppose we could repair the assembly. But I don't like the idea of checked in generated assembly either. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Saturday, March 13, 2021 4:53 AM To: Mika Nystrom >; m3devel > Subject: [M3devel] caltech even on Linux/amd64.. Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 17:30:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 16:30:21 +0000 Subject: [M3devel] caltech even on Linux/amd64.. In-Reply-To: References: , , Message-ID: Maybe regenerate those files w/o debug info? There is no great answer. We should not have checked in derived files. But we have no Fortran support. There should be only one debug format, and it should always work. Nobody should write Fortran? ? Use f2c? Given a flawed world, one more small flaw: config: For the sake of caltech-other/newuoa, ? modula3/cm3 at fecf78b (github.com) ? No other package depends on this, I think. So disabling it is also ok with me, but perhaps that is rude to you. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, March 13, 2021 3:51 PM To: Mika Nystrom Cc: m3devel Subject: Re: [M3devel] caltech even on Linux/amd64.. aha. I see that now too. -gstabs+ produces demonstrably incorrect symbols on "current" gcc on Linux. Current shipping by Debian and/or Ubuntu. Not current head of gcc. Gcc developers are clearly not a fan of this switch. They want to remove. I don't think it was ever really needed, but added because it was thought to be relevant to the stabs used by m3cc. It doesn't even seem to be needed for linking m3cc+stabs output. But now we are dependent on it, in this one directory. We have some provision for different commands for assembling than compiling C, but that doesn't actually work today. We could see if "ends in .s" and then use -gstabs+. I have to see if Quake offers "ends with". We can add it. We could try and if failure, try again with -gstabs+. That is a little gross, and inefficient if the failure is something else, but will work. Would it help if we required gfortran? And I acknowledge this is much ado about little. Would it be horrible to disable. Maybe regenerate the files w/o symbols? - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 3:20 PM To: Jay K Cc: m3devel Subject: Re: caltech even on Linux/amd64.. yeah without the -gstabs+ I get the same error as you. I'm assuming it comes from here (cm3/bin/config/AMD64_LINUX) but am not sure SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" % C compiler On Sat, Mar 13, 2021 at 7:12 AM Mika Nystrom > wrote: Here is the command in my build: new source -> compiling update.s gcc -gstabs+ -m64 -fPIC -g -c ../src/update.s -o update.o I thought there was a problem not using -gstabs+ ? (Not related to compiling my hacked-up Fortran-assembly...) On Sat, Mar 13, 2021 at 6:52 AM Mika Nystrom > wrote: The ultimate source is in there too. But it's in Fortran and we don't have that enabled in the quake. I was thinking that the right way to do it would be to allow f_source in quake, but you know, projects... time commitments... Not sure why the .s isn't working for you. I compiled with ifort and it's worked on a number of systems for me. Mika On Fri, Mar 12, 2021, 9:30 PM Jay K > wrote: I expect I will be disabling this stuff shorty, sorry. I suppose we could repair the assembly. But I don't like the idea of checked in generated assembly either. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Saturday, March 13, 2021 4:53 AM To: Mika Nystrom >; m3devel > Subject: [M3devel] caltech even on Linux/amd64.. Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 17:33:35 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 16:33:35 +0000 Subject: [M3devel] caltech even on Linux/amd64.. In-Reply-To: References: , , , Message-ID: To be clear though, above all the rest, I believe do-cm3-all.py and make-dist.py and boot2.py should have a higher chance of succeeding. That's how I ran into this of course. I already have to fight with X Windows and OpenGL and ODBC on every new system and I'm not sure what, if anything, to do about that. We could break up into multiple packages, but then we'd just be randomly missing some. But then, I just thought, libraries that are "just bindings" should probably always be standalone. They tend to have little or no code, and little or no state. (Sometimes some custom bindings require code or state?) Little code is ok. No state should be required. I wonder if cm3 can help us here. build_standalone_if_no_globals()? ? - Jay ________________________________ From: Jay K Sent: Saturday, March 13, 2021 4:30 PM To: Mika Nystrom Cc: m3devel Subject: Re: caltech even on Linux/amd64.. Maybe regenerate those files w/o debug info? There is no great answer. We should not have checked in derived files. But we have no Fortran support. There should be only one debug format, and it should always work. Nobody should write Fortran? ? Use f2c? Given a flawed world, one more small flaw: config: For the sake of caltech-other/newuoa, ? modula3/cm3 at fecf78b (github.com) ? No other package depends on this, I think. So disabling it is also ok with me, but perhaps that is rude to you. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, March 13, 2021 3:51 PM To: Mika Nystrom Cc: m3devel Subject: Re: [M3devel] caltech even on Linux/amd64.. aha. I see that now too. -gstabs+ produces demonstrably incorrect symbols on "current" gcc on Linux. Current shipping by Debian and/or Ubuntu. Not current head of gcc. Gcc developers are clearly not a fan of this switch. They want to remove. I don't think it was ever really needed, but added because it was thought to be relevant to the stabs used by m3cc. It doesn't even seem to be needed for linking m3cc+stabs output. But now we are dependent on it, in this one directory. We have some provision for different commands for assembling than compiling C, but that doesn't actually work today. We could see if "ends in .s" and then use -gstabs+. I have to see if Quake offers "ends with". We can add it. We could try and if failure, try again with -gstabs+. That is a little gross, and inefficient if the failure is something else, but will work. Would it help if we required gfortran? And I acknowledge this is much ado about little. Would it be horrible to disable. Maybe regenerate the files w/o symbols? - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 3:20 PM To: Jay K Cc: m3devel Subject: Re: caltech even on Linux/amd64.. yeah without the -gstabs+ I get the same error as you. I'm assuming it comes from here (cm3/bin/config/AMD64_LINUX) but am not sure SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" % C compiler On Sat, Mar 13, 2021 at 7:12 AM Mika Nystrom > wrote: Here is the command in my build: new source -> compiling update.s gcc -gstabs+ -m64 -fPIC -g -c ../src/update.s -o update.o I thought there was a problem not using -gstabs+ ? (Not related to compiling my hacked-up Fortran-assembly...) On Sat, Mar 13, 2021 at 6:52 AM Mika Nystrom > wrote: The ultimate source is in there too. But it's in Fortran and we don't have that enabled in the quake. I was thinking that the right way to do it would be to allow f_source in quake, but you know, projects... time commitments... Not sure why the .s isn't working for you. I compiled with ifort and it's worked on a number of systems for me. Mika On Fri, Mar 12, 2021, 9:30 PM Jay K > wrote: I expect I will be disabling this stuff shorty, sorry. I suppose we could repair the assembly. But I don't like the idea of checked in generated assembly either. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Saturday, March 13, 2021 4:53 AM To: Mika Nystrom >; m3devel > Subject: [M3devel] caltech even on Linux/amd64.. Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 13 17:54:14 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 16:54:14 +0000 Subject: [M3devel] paths .m3 => mg? In-Reply-To: References: , Message-ID: But..these paths are sent to the backend..and backend's rarely issue any diagnostics. Assuming m3front has done a good job. The below is an exception, and I cannot explain it easily. I can imagine seeing both paths is more useful when m3front finds mistakes, which is more common than the backend finding mistakes? That said, these paths are used by the C for line directives and, except for these cases, contribute greatly to a decent debugging experience. You can step through the lines of Modula-3. These wierd paths break that, and "C convergence". On the theory that the paths are formed in only one place, both for m3front diagnostics and the backend, I guess I'll just change them in the C backend. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 2:49 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] paths .m3 => mg? Jay, the use of generics in libm3 et al. is pretty simplistic, just some ADTs, really. In general you can do much more complicated things with generics, so seeing names of both the .m3 and .mg can be very useful.. On Fri, Mar 12, 2021, 10:04 PM Jay K > wrote: cm3: ../AMD64_LINUX/FinDateArraySort.m3 => /cm3/pkg/libm3/src/sort/ArraySort.mg:86:0: warning: '' may be used uninitialized in this function [-Wmaybe-uninitialized] Nevermind the actual warning. The "foo.m3 => bar.mg" is one source file path, synthesized by m3front. That entire strong, with two embedded paths, is considered one path. Why? Is this useful? What were they thinking? The .m3 is what is being compiled. But is just one line, is not useful to the programmer receiving warnings/errors/assertion failures. The .mg is a "real" useful source file. If that was the only thing printed, I think people would be satisified. It is a *little* useful to see the other line to see the generic parameter. Perhaps?? So why the overall thing? I am guessing, that it is a problem, for the same source file to be compiled more than once?? So some "form" of the right is needed?? Or maybe the "little useful" part?? C systems have to deal with this anyway: echo "int name() { return value ;}" > foo.c cc -c -Dname=one -Dvalue=1 foo.c cc -c -Dname=two -Dvalue=2 foo.c cc -c -Dname=three -Dvalue=3oops foo.c => error foo.c:"3oops" and the ambiguity of errors in foo.c, if you don't see the command line, well, nevermind? But I guess that has no bearing on Modula3 and perhaps the same file compiled twice would confuse the build system? So I guess I'll mostly leave this alone. I have to change it, somewhat, to unify C targets, i.e. the "AMD64_LINUX" part is bad. I can just remove it or change it to or , for the C backend only. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Mar 13 22:36:25 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 13 Mar 2021 15:36:25 -0600 Subject: [M3devel] packedVars branch is now merged into master. Message-ID: The packedVars branch is now merged back into master. It has many changes. A summary: Enhanced handling of packed values. Allow more cases, fix bugs, handle them in new ways, e.g. passing as parameters. Packed scalars can not straddle word boundaries, but arrays and records can if their scanar elements don't. Fields and array elements can be sub-byte sized, with limits. Open array elements are limited to 1, 2, or 4 bytes, as a consequence. Packed fields/elements are handled in new places, e.g. as actual parameters. Sets are given the smallest of 8, 16, 32, or 64 that will hold them. But Sets larger than a word are still multiples of whole words. Sets can be packed to smaller sizes. Bugs in array and record constructors are fixed, including previously undetected range errors in fields/elements of constant constructors. Most cases of errors that are called runtime by Modula-3 but statically detectable only warn at compile time, but fail at runtime, unless the subject code is always guaranteed to be executed. Previously, nested constructors (including multi-dimensional arrays) were time- and space-inefficient, by building all the subconstructors separately, then copying to the elements of the outer constructor. Moreover, there were many cases of building at runtime, array dope with static shape, the later runtime-checking the shape against constants. This is eliminated. Array constructors almost always have static shape even with open type, denoted by the number of values between the braces. These are constructed and used with fixed array representation. There remain complex cases where runtime shapes are necessary. These changes apply to both all-constant constructors and those with some non-constant elements. These are handled quite differently. Pickles and m3gdb debug info are extend to support these changes. Internally, a confusion of alignments of pointers and alignments of their referents is untangled by maintaining bot values. A number of new and expanded test cases are included, for bug fixes and new behaviors. Many messages have section references to the Modula-3 definition. Assorted comment and error message improvements. Recompiles itself twice. -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Sun Mar 14 00:48:34 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 23:48:34 +0000 Subject: [M3devel] caltech even on Linux/amd64.. In-Reply-To: References: , Message-ID: "reasonable" is subjective. This works..just a little gross.. config: For the sake of caltech-other/newuoa, ? modula3/cm3 at fecf78b (github.com) I think we should remove the debugging stuff from, or regenerate w/ different switches? Or, get it to build with gfortran, if gfortran is found by exec. I think I tried that though and it fail to compile. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 10:54 PM To: Jay K Cc: m3devel Subject: Re: caltech even on Linux/amd64.. No it's not at all rude to disable this stuff if there's no reasonable way to make it work at the moment. On Sat, Mar 13, 2021 at 8:30 AM Jay K > wrote: Maybe regenerate those files w/o debug info? There is no great answer. We should not have checked in derived files. But we have no Fortran support. There should be only one debug format, and it should always work. Nobody should write Fortran? ? Use f2c? Given a flawed world, one more small flaw: config: For the sake of caltech-other/newuoa, ? modula3/cm3 at fecf78b (github.com) ? No other package depends on this, I think. So disabling it is also ok with me, but perhaps that is rude to you. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Saturday, March 13, 2021 3:51 PM To: Mika Nystrom > Cc: m3devel > Subject: Re: [M3devel] caltech even on Linux/amd64.. aha. I see that now too. -gstabs+ produces demonstrably incorrect symbols on "current" gcc on Linux. Current shipping by Debian and/or Ubuntu. Not current head of gcc. Gcc developers are clearly not a fan of this switch. They want to remove. I don't think it was ever really needed, but added because it was thought to be relevant to the stabs used by m3cc. It doesn't even seem to be needed for linking m3cc+stabs output. But now we are dependent on it, in this one directory. We have some provision for different commands for assembling than compiling C, but that doesn't actually work today. We could see if "ends in .s" and then use -gstabs+. I have to see if Quake offers "ends with". We can add it. We could try and if failure, try again with -gstabs+. That is a little gross, and inefficient if the failure is something else, but will work. Would it help if we required gfortran? And I acknowledge this is much ado about little. Would it be horrible to disable. Maybe regenerate the files w/o symbols? - Jay ________________________________ From: Mika Nystrom > Sent: Saturday, March 13, 2021 3:20 PM To: Jay K > Cc: m3devel > Subject: Re: caltech even on Linux/amd64.. yeah without the -gstabs+ I get the same error as you. I'm assuming it comes from here (cm3/bin/config/AMD64_LINUX) but am not sure SYSTEM_CC = "gcc -gstabs+ -m64 -fPIC" % C compiler On Sat, Mar 13, 2021 at 7:12 AM Mika Nystrom > wrote: Here is the command in my build: new source -> compiling update.s gcc -gstabs+ -m64 -fPIC -g -c ../src/update.s -o update.o I thought there was a problem not using -gstabs+ ? (Not related to compiling my hacked-up Fortran-assembly...) On Sat, Mar 13, 2021 at 6:52 AM Mika Nystrom > wrote: The ultimate source is in there too. But it's in Fortran and we don't have that enabled in the quake. I was thinking that the right way to do it would be to allow f_source in quake, but you know, projects... time commitments... Not sure why the .s isn't working for you. I compiled with ifort and it's worked on a number of systems for me. Mika On Fri, Mar 12, 2021, 9:30 PM Jay K > wrote: I expect I will be disabling this stuff shorty, sorry. I suppose we could repair the assembly. But I don't like the idea of checked in generated assembly either. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Saturday, March 13, 2021 4:53 AM To: Mika Nystrom >; m3devel > Subject: [M3devel] caltech even on Linux/amd64.. Doesn't build. compile_c => 1 C compiler failed compiling: ../src/trsapp.s new source -> compiling update.s gcc -g -m64 -fPIC -g -c ../src/update.s -o update.o ../src/update.s: Assembler messages: ../src/update.s: Fatal error: duplicate .debug_line sections compile_c => 1 C compiler failed compiling: ../src/update.s **** PARALLEL BACK-END BUILD, M3_PARALLEL_BACK = 20 gcc -g -m64 -fPIC -g -c NewUOA.i3.c -o NewUOA_i.o gcc -g -m64 -fPIC -g -c NewUOA_M3.i3.c -o NewU -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 00:51:03 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 13 Mar 2021 23:51:03 +0000 Subject: [M3devel] packaging formats and sets? Message-ID: I have a nagging feeling I went overboard here, and the result is too confusing. reference Release 20210312 (March 12 2021) ? modula3/cm3 (github.com) For Windows we have roughly: cm3-min-win32.zip cm3-min-win32-symbols.zip cm3-all-win32.zip cm3-all-win32-symbols.zip cm3-min-win32.msi cm3-all-win32.msi cm3-boot.tgz For Unix we have roughly: cm3-min-win32.tgz cm3-all-win32.tgz cm3-min-win32.deb cm3-all-win32.deb cm3-boot.tgz The .deb and .msi files are little more than all.tgz. The .deb is compressed better, to half the size. The .msi installs to c:\cm3 (user selectable?) and adds c:\cm3\bin to %PATH%. Something like that. min is just cm3 + mklib + m3core + libm3 + maybe cm3cg (gcc backend) + config files all is everything I could build boot is only enough to build cm3 with minimal dependnecies, i.e. a C toolset boot is perhaps esoteric but also quite important It could be augmented easily to build m3core and libm3, it already has their source or assembly In the 3.6 release there was just boot per target that was assembly source, and C for m3build I think. (Critical Mass reimplemened quake in Modula-3.) Plus the full source release. It was good enough then? Or people wants things easier these days? And there are fewer platforms today so that makes it easier? I generally do not strip Unix binaries, as debugging matters too much. Current Windows builds, whatever I am running, contain tar, that can create/extract tgz. I know GUI zip support goes back longer, but there remains no command line zip support. I have to add a zip from where to create the packages. It seems well motivated, but seems like it is too much also. Thoughts? My thinking drifts towards: 1. Eliminate "min"? 2. Combine symbols with binaries? 3. Consider eliminating deb and msi? I realize they are pretty nice to have though. It might also be easy enough..would require some research to make a very small .msi that downloads the all.tgz or something. I put together the .msi and .deb very trivially. 4. Consider replacing tgz with tar.xz or similar. I don't know. As I said, everything has some motivation. min is much smaller than all, etc. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 06:32:36 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 05:32:36 +0000 Subject: [M3devel] declare_set not multiple of integer size Message-ID: new source -> compiling FloatMode.i3 "..\src\float\IEEE-default\FloatMode.i3", line 12: declare_set not multiple of integer size Line 12 is: "INTERFACE FloatMode;" "..\src\runtime\common\RTHeapRep.i3", line 97: declare_set not multiple of integer size Desc = RECORD space : BITS 2 FOR Space; generation: BITS 1 FOR Generation; pure : BITS 1 FOR BOOLEAN; note : BITS 3 FOR Note; gray : BITS 1 FOR BOOLEAN; clean : BITS 1 FOR BOOLEAN; locked : BITS 1 FOR BOOLEAN; link: BITS BITSIZE(ADDRESS) - LogAdrPerPage FOR Page := Nil; END; PageHdr = RECORD desc: Desc; nb: CARDINAL := 1; END; RefPage = UNTRACED REF PageHdr; (* line 97 *) TYPE Space = {Unallocated, Free, Previous, Current}; I kinda never liked Desc using bitfields. It is like only one of two places in the system that use them, causing big/little-endian C to vary. The other is tearing apart floating point numbers. Aha these errors are from the C backend. :( PROCEDURE declare_set(self: DeclareTypes_t; typeid, domain_type: TypeUID; bit_size: BitSize) = VAR x := self.self; BEGIN IF DebugVerbose(x) THEN x.comment("declare_set typeid:" & TypeIDToText(typeid) & " domain_type:" & TypeIDToText(domain_type) & " bit_size:" & IntToDec(bit_size)); ELSE x.comment("declare_set"); END; IF (bit_size MOD Target.Integer.size) # 0 THEN Err(x, "declare_set not multiple of integer size"); END; IF bit_size = Target.Integer.size THEN x.Type_Init(NEW(Integer_t, cgtype := Target.Integer.cg_type, typeid := typeid)); ELSE self.declare_array(typeid, UID_WORD, UID_WORD, bit_size); END; END declare_set; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 06:34:16 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 05:34:16 +0000 Subject: [M3devel] declare_set not multiple of integer size In-Reply-To: References: Message-ID: Eh, I guess this is only for hypothetical strong typing/debugging of the output. m3x86 for example does nothing with this. So I guess no big deal. I need to setup that CI... - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Sunday, March 14, 2021 5:32 AM To: m3devel Subject: [M3devel] declare_set not multiple of integer size new source -> compiling FloatMode.i3 "..\src\float\IEEE-default\FloatMode.i3", line 12: declare_set not multiple of integer size Line 12 is: "INTERFACE FloatMode;" "..\src\runtime\common\RTHeapRep.i3", line 97: declare_set not multiple of integer size Desc = RECORD space : BITS 2 FOR Space; generation: BITS 1 FOR Generation; pure : BITS 1 FOR BOOLEAN; note : BITS 3 FOR Note; gray : BITS 1 FOR BOOLEAN; clean : BITS 1 FOR BOOLEAN; locked : BITS 1 FOR BOOLEAN; link: BITS BITSIZE(ADDRESS) - LogAdrPerPage FOR Page := Nil; END; PageHdr = RECORD desc: Desc; nb: CARDINAL := 1; END; RefPage = UNTRACED REF PageHdr; (* line 97 *) TYPE Space = {Unallocated, Free, Previous, Current}; I kinda never liked Desc using bitfields. It is like only one of two places in the system that use them, causing big/little-endian C to vary. The other is tearing apart floating point numbers. Aha these errors are from the C backend. :( PROCEDURE declare_set(self: DeclareTypes_t; typeid, domain_type: TypeUID; bit_size: BitSize) = VAR x := self.self; BEGIN IF DebugVerbose(x) THEN x.comment("declare_set typeid:" & TypeIDToText(typeid) & " domain_type:" & TypeIDToText(domain_type) & " bit_size:" & IntToDec(bit_size)); ELSE x.comment("declare_set"); END; IF (bit_size MOD Target.Integer.size) # 0 THEN Err(x, "declare_set not multiple of integer size"); END; IF bit_size = Target.Integer.size THEN x.Type_Init(NEW(Integer_t, cgtype := Target.Integer.cg_type, typeid := typeid)); ELSE self.declare_array(typeid, UID_WORD, UID_WORD, bit_size); END; END declare_set; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 09:15:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 08:15:06 +0000 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: References: , Message-ID: win32 system appears to be suffer with the new packed rules? new source -> compiling Time.m3 "..\src\time\WIN32\Time.m3", line 18: CM3 restriction: VAR actual cannot be packed. (2.3.2): (FileTime) 1 error encountered m3front failed compiling: ..\src\time\WIN32\Time.m3 new source -> compiling DateWin32.m3 "..\src\time\WIN32\DateWin32.m3", line 65: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) "..\src\time\WIN32\DateWin32.m3", line 137: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) 2 errors encountered -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 14 09:38:13 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 14 Mar 2021 11:38:13 +0300 Subject: [M3devel] packaging formats and sets? In-Reply-To: References: Message-ID: <526631615709234@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 09:38:27 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 08:38:27 +0000 Subject: [M3devel] declare_set not multiple of integer size In-Reply-To: References: , Message-ID: m3c:Accomodate sets smaller than a word. ? modula3/cm3 at f31d108 (github.com) I guess. ________________________________ From: M3devel on behalf of Jay K Sent: Sunday, March 14, 2021 5:34 AM To: m3devel Subject: Re: [M3devel] declare_set not multiple of integer size Eh, I guess this is only for hypothetical strong typing/debugging of the output. m3x86 for example does nothing with this. So I guess no big deal. I need to setup that CI... - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Sunday, March 14, 2021 5:32 AM To: m3devel Subject: [M3devel] declare_set not multiple of integer size new source -> compiling FloatMode.i3 "..\src\float\IEEE-default\FloatMode.i3", line 12: declare_set not multiple of integer size Line 12 is: "INTERFACE FloatMode;" "..\src\runtime\common\RTHeapRep.i3", line 97: declare_set not multiple of integer size Desc = RECORD space : BITS 2 FOR Space; generation: BITS 1 FOR Generation; pure : BITS 1 FOR BOOLEAN; note : BITS 3 FOR Note; gray : BITS 1 FOR BOOLEAN; clean : BITS 1 FOR BOOLEAN; locked : BITS 1 FOR BOOLEAN; link: BITS BITSIZE(ADDRESS) - LogAdrPerPage FOR Page := Nil; END; PageHdr = RECORD desc: Desc; nb: CARDINAL := 1; END; RefPage = UNTRACED REF PageHdr; (* line 97 *) TYPE Space = {Unallocated, Free, Previous, Current}; I kinda never liked Desc using bitfields. It is like only one of two places in the system that use them, causing big/little-endian C to vary. The other is tearing apart floating point numbers. Aha these errors are from the C backend. :( PROCEDURE declare_set(self: DeclareTypes_t; typeid, domain_type: TypeUID; bit_size: BitSize) = VAR x := self.self; BEGIN IF DebugVerbose(x) THEN x.comment("declare_set typeid:" & TypeIDToText(typeid) & " domain_type:" & TypeIDToText(domain_type) & " bit_size:" & IntToDec(bit_size)); ELSE x.comment("declare_set"); END; IF (bit_size MOD Target.Integer.size) # 0 THEN Err(x, "declare_set not multiple of integer size"); END; IF bit_size = Target.Integer.size THEN x.Type_Init(NEW(Integer_t, cgtype := Target.Integer.cg_type, typeid := typeid)); ELSE self.declare_array(typeid, UID_WORD, UID_WORD, bit_size); END; END declare_set; - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 14 09:43:43 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 14 Mar 2021 11:43:43 +0300 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: References: , Message-ID: <520141615711185@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 09:52:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 08:52:21 +0000 Subject: [M3devel] packaging formats and sets? In-Reply-To: <526631615709234@mail.yandex.by> References: , <526631615709234@mail.yandex.by> Message-ID: "Eliminate" in this case would mean, "stop producing" and uploading. Not "delete whatever is out there" nor likely "Delete the code that does it". (see make-dist.py and pylib.py) I forgot to say, the pdbs don't add too much size to the download, at least. Thus combining the archives, perhaps. The reason I shy away from xz/lzma though, is that they are much slower to produce and are slower and use memory to decompress. So much of the production is just to test and throw away also. At least until we have a decent CI pipeline. The decompression does seem fast on modern hardware but I read that OpenBSD sticks to gzip for its efficiency esp. on old hardware. And I just checked, they still use .tgz. But I can try. Esp. if Windows tar supports it. :) I like .pdb in bin because debuggers find them most easily when they are next to the code, at least for non-remote debugging. I know they can look elsewhere. Perhaps bickering over placement is well resolved by the separate download? You can just extract where yo wish. I don't know. As I said, debugging is too important, to use strip. But building your own isn't horribly difficult. Arguable. There's just too many archives for beginners to ponder. And experts build their own. Perhaps. I am reminded of Eclipse, which is confusing to install. Too many choices. > Please, avoid any type of using LAN/WAN/Internet access ( "downloading") in .msi / .deb. Good point, though the .tgz provide that. Long ago the release was split into not just min and all, but some variety. I find that also gets confusing which is why I reduced it to just min and all. Perhaps I was lazy in reimplementing the packages, perhaps. (You see, I wrote none of the .sh, but all of the .py, including make-dist.py). More important is presumably to setup GitHub actions. Currently this stuff is insufficiently automated. If anyone else wants to take on some of this stuff, please do. I will probably run out of time again soon. :( - Jay ________________________________ From: vvm at tut.by Sent: Sunday, March 14, 2021 8:38 AM To: Jay K ; m3devel Subject: Re: [M3devel] packaging formats and sets? Hi! ( eliminate eq "completely remove" and(or) "destroy"? ) 1. Eliminate "min"? 2. Combine symbols with binaries? 3. Consider eliminating deb and msi? I realize they are pretty nice to have though. It might also be easy enough..would require some research to make a very small .msi that downloads the all.tgz or something. I put together the .msi and .deb very trivially. 4. Consider replacing tgz with tar.xz or similar. Important first: z) Yes, please replace .tar.gzip to .tar.xz. .xz has a smaller size. z) It's better replace zip to 7Zip ( .zip to .7z). y) Please, avoid any type of using LAN/WAN/Internet access ( "downloading") in .msi / .deb. We must have possibility of full __offline__ installation of M3 in DMZ or equal environment. By security reasons. 3) Avoid "completely destroying" .msi and .deb. Save both cm3-min and cm3-all variants. 2) Please, continue storing .pdb separately. May be best store it with .m3 generated from .m3g. And don't store .pdb in bin folder. Store .pdb in folders that srore in correlated .exe/.dll I specifically place source code in c:\cm3\src\ before compiling. 1) cm3-min must contain mklib.exe. There are usefull both variants of distributies. I.e. need cm3-min and cm3-all. May be best place cube.exe, games, etc to cm3-examples-binary.7z , but it doesn't principal. Thanks! Beat regards, Victor Miasnikov 14.03.2021, 02:51, "Jay K" : I have a nagging feeling I went overboard here, and the result is too confusing. reference Release 20210312 (March 12 2021) ? modula3/cm3 (github.com) For Windows we have roughly: cm3-min-win32.zip cm3-min-win32-symbols.zip cm3-all-win32.zip cm3-all-win32-symbols.zip cm3-min-win32.msi cm3-all-win32.msi cm3-boot.tgz For Unix we have roughly: cm3-min-win32.tgz cm3-all-win32.tgz cm3-min-win32.deb cm3-all-win32.deb cm3-boot.tgz The .deb and .msi files are little more than all.tgz. The .deb is compressed better, to half the size. The .msi installs to c:\cm3 (user selectable?) and adds c:\cm3\bin to %PATH%. Something like that. min is just cm3 + mklib + m3core + libm3 + maybe cm3cg (gcc backend) + config files all is everything I could build boot is only enough to build cm3 with minimal dependnecies, i.e. a C toolset boot is perhaps esoteric but also quite important It could be augmented easily to build m3core and libm3, it already has their source or assembly In the 3.6 release there was just boot per target that was assembly source, and C for m3build I think. (Critical Mass reimplemened quake in Modula-3.) Plus the full source release. It was good enough then? Or people wants things easier these days? And there are fewer platforms today so that makes it easier? I generally do not strip Unix binaries, as debugging matters too much. Current Windows builds, whatever I am running, contain tar, that can create/extract tgz. I know GUI zip support goes back longer, but there remains no command line zip support. I have to add a zip from where to create the packages. It seems well motivated, but seems like it is too much also. Thoughts? My thinking drifts towards: 1. Eliminate "min"? 2. Combine symbols with binaries? 3. Consider eliminating deb and msi? I realize they are pretty nice to have though. It might also be easy enough..would require some research to make a very small .msi that downloads the all.tgz or something. I put together the .msi and .deb very trivially. 4. Consider replacing tgz with tar.xz or similar. I don't know. As I said, everything has some motivation. min is much smaller than all, etc. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 09:53:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 08:53:21 +0000 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: <520141615711185@mail.yandex.by> References: , , <520141615711185@mail.yandex.by> Message-ID: Looks like we can just remove the BITS FOR statement. Which makes the new rules seem, kinda pointless at least in this case. BITS FOR didn't actually change anything, but validity. In this case. - Jay ________________________________ From: vvm at tut.by Sent: Sunday, March 14, 2021 8:43 AM To: Jay K ; m3devel Subject: Re: [M3devel] win32 vs. new packing rules? Hi! I didn't compile "packed" variant M3 now. But I have big amount of errors when I compiled it in september of 2019 and march of 2020. Best regards, Victor Miasnikov 14.03.2021, 11:15, "Jay K" : win32 system appears to be suffer with the new packed rules? new source -> compiling Time.m3 "..\src\time\WIN32\Time.m3", line 18: CM3 restriction: VAR actual cannot be packed. (2.3.2): (FileTime) 1 error encountered m3front failed compiling: ..\src\time\WIN32\Time.m3 new source -> compiling DateWin32.m3 "..\src\time\WIN32\DateWin32.m3", line 65: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) "..\src\time\WIN32\DateWin32.m3", line 137: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) 2 errors encountered , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 10:04:04 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 09:04:04 +0000 Subject: [M3devel] push to master vs. PR. Message-ID: I noticed commiting PRs sends email. But pushing right to master doesn't or doesn't always. ? So maybe prefer PRs? If you self-approve and complete? I know it is a lot of extra clicks. Also, I have once or twice, right after commit to master (PR or not), really wanted to edit/fixup. So I have force pushed to master. Very rarely, very soon after master edited. I know some/most orgs are strongly against that. But sometimes I really want to. So question, do we strictly forbid, or just midly/strongly? I know it makes it difficult if anyone has managed to pull in the intervening time. But if the window is short, ok? Perhaps CI is the real answer. ? i.e. if it is quick and thorough and strict, the need/desire for small follow ups will reduce. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 14 11:14:42 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 14 Mar 2021 13:14:42 +0300 Subject: [M3devel] packaging formats and sets? In-Reply-To: References: , <526631615709234@mail.yandex.by> Message-ID: <532331615714601@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 12:24:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 11:24:00 +0000 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms Message-ID: This: PROCEDURE Init32 () = BEGIN (* Change size and aligment: *) Integer := Int32; Word := Word32; Address := Word32; Address.cg_type := CGType.Addr; (* Change only alignment. Size is always 64: *) Longint.align := 32; <== Long.align := 32; <== Longreal.align := 32; <== Extended.align := 32; <== END Init32; is not great. Perhaps terrible. Because x86 has InterlockedCompareExchange64, that wants 64 alignment. The Longreal part is also a bad idea. This is an old way of achieving 64bit alignment, before the advent of 64bit integers: union LARGE_INTEGER { struct { int HighPart, LowPart }; double Alignment; }; And it should continue to work. The 32bit and 64bit rules should be more the same. I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. But it is likely difficult/impossible. The interlocked operations could be over in C. Also this maybe disagrees with C alignment. Let's see: C:\s\cm3\scripts\python>type 1.c struct { __int64 a; int b; } a; int b = sizeof(a); #pragma pack(1) struct { __int64 a; int b; } c; int d = sizeof(c); C:\s\cm3\scripts\python>cl /c /FAsc 1.c Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 Copyright (C) Microsoft Corporation. All rights reserved. 1.c C:\s\cm3\scripts\python>more 1.cod ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 COMM _a:BYTE:010H _b DD 010H _d DD 0cH So this seems like it could really break things. i.e. struct stat in m3core. Except that I sorted things by size so it'll tend to work out. But still, really, alignment of 64bit integers should be 64, on all platforms. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 13:05:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 12:05:21 +0000 Subject: [M3devel] assert false "..\src\exprs\CallExpr.m3", line 179 Message-ID: This is my mistake: PROCEDURE SetupNamingConventions (mach : Quake.Machine) = BEGIN (* Refactor to avoid creating garbage. *) SetupNamingConventionsInternal (NEW (State), mach); END SetupNamingConventions; ..\src\Builder.m3", line 260: VAR actual must be a designator (2.3.2) (s) this is maybe someone else's? *** *** runtime error: *** <*ASSERT*> failed: FALSE *** file "..\src\exprs\CallExpr.m3", line 179 *** PROCEDURE NotAddressable (<*UNUSED*> t: T) = BEGIN <* ASSERT FALSE *> END NotAddressable; Well recording it here anyway. NEW (State) was initializing a VAR and I guess it most. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Mar 14 14:29:58 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 14 Mar 2021 09:29:58 -0400 Subject: [M3devel] packaging formats and sets? In-Reply-To: References: Message-ID: <20210314132958.spvp5rldfrh2dl3u@topoi.pooq.com> On Sat, Mar 13, 2021 at 11:51:03PM +0000, Jay K wrote: > I have a nagging feeling I went overboard here, and the result is too confusing. > > reference Release 20210312 (March 12 2021) ? modula3/cm3 (github.com) > > For Windows we have roughly: > > cm3-min-win32.zip > cm3-min-win32-symbols.zip > cm3-all-win32.zip > cm3-all-win32-symbols.zip > cm3-min-win32.msi > cm3-all-win32.msi > cm3-boot.tgz > > For Unix we have roughly: > > cm3-min-win32.tgz > cm3-all-win32.tgz > cm3-min-win32.deb > cm3-all-win32.deb > cm3-boot.tgz I'm curious wh the Unix downloads have win32 in thir names. > > > The .deb and .msi files are little more than all.tgz. > The .deb is compressed better, to half the size. > The .msi installs to c:\cm3 (user selectable?) and adds c:\cm3\bin to %PATH%. > Something like that. > > min is just cm3 + mklib + m3core + libm3 + maybe cm3cg (gcc backend) + config files > all is everything I could build > boot is only enough to build cm3 with minimal dependnecies, i.e. a C toolset > boot is perhaps esoteric but also quite important > It could be augmented easily to build m3core and libm3, it already > has their source or assembly > > In the 3.6 release there was just boot per target that was assembly source, > and C for m3build I think. (Critical Mass reimplemened quake in Modula-3.) > > Plus the full source release. > > It was good enough then? Or people wants things easier these days? > And there are fewer platforms today so that makes it easier? > > I generally do not strip Unix binaries, as debugging matters too much. > > Current Windows builds, whatever I am running, contain tar, that can create/extract tgz. > I know GUI zip support goes back longer, but there remains no command line zip support. > I have to add a zip from where to create the packages. > > It seems well motivated, but seems like it is too much also. > > Thoughts? > > My thinking drifts towards: > 1. Eliminate "min"? > 2. Combine symbols with binaries? > 3. Consider eliminating deb and msi? > I realize they are pretty nice to have though. .deb is quite important -- there are a lot of Linux distros that are derived from Debian. I'd try to avoid systemd dependencies, though. > It might also be easy enough..would require some research to > make a very small .msi that downloads the all.tgz or something. > I put together the .msi and .deb very trivially. Does the .deb contain the executables? Or just the boot and source code so that the main work of the installation is done in a post-install script? If the latter, mke sure you have install-time dependencies on all the tools you'll need. > 4. Consider replacing tgz with tar.xz or similar. That might require a new install-time dependency. > > I don't know. As I said, everything has some motivation. > min is much smaller than all, etc. There might be modula3 bindings for significant libraries that some installations do not use or need (such as fancy database systems and the like). Those bindings should be in separate packages -- otherwise you'll end up making those big subsystems mandatory even for users who do not need them. -- hendrik > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jayk123 at hotmail.com Sun Mar 14 19:22:45 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 18:22:45 +0000 Subject: [M3devel] packaging formats and sets? In-Reply-To: <20210314132958.spvp5rldfrh2dl3u@topoi.pooq.com> References: , <20210314132958.spvp5rldfrh2dl3u@topoi.pooq.com> Message-ID: > I'm curious wh the Unix downloads have win32 in thir names. Typos, sorry. > .deb is quite important Agreed ad not. In general, yes. apt-get install is great. But in this case maybe not. We don't have things for sources.list to point to. Personally I never use dpkg -i manually. The .debs are super simple. The version is constant and there are no dependencies. It is just a way to extract the .tgz (or .tar.other) to /usr/local. I don't think path is managed, so you are left running /usr/local/cm3/bin or changing PATH yourself. I doubt anyone has used the .debs I produce. Really getting into a distribution is problematic due to licensing and build reasons (i.e. sort of, being written in not-C). > Does the .deb contain the executables? Or just the boot and source code > so that the main work of the installation is done in a post-install > script? If the latter, mke sure you have install-time dependencies on > all the tools you'll need. Oh, I had not thought of that actually, .deb as automated build. The .deb is just executables, .so, .a, and the pkg store which includes source for debugging. This is kinda a new idea to me. I mean, I know debs to some install stuff, and even the Mono .debs do compilation but I hadn't considered it here. Do people like that idea? It might be quite good. The .debs could actually be cross-distribution then. > .gz vs. .xz > That might require a new install-time dependency. Agreed, maybe not worth it. The size would be halved though. > Those bindings should be in separate packages Yeah, kinda. I put no dependencies in the .deb. The installed stuff "just" might not work. Which I realize is generally not how .debs work, but maybe adequate. I also have a related theory that bindings-only libraries should be build_standalone, which doesn't exactly change this equation..but it does, if the .debs become things that compile and link, instead of just extract. Bindings tend to be just .i3 files and no code or state. In which case build_standalone has no downside, no shared state to unshare. Sometimes they have a small amount of code or state, which is unlikely to require sharing and is likely small. We should consider this separately. Also, for .debs that do contain executables, there becomes the problem of being "very" distribution specific, i.e. version specific. Furthermore, if we did meet various licensing/build requirements, building .debs might be better left to the distribution maintainers. Really, sorry, I think the current .debs are just not useful. But again this idea of a .deb as build automation is somewhat intriguing. I'm not sure if it beats something like, a boot+source .tgz that offers make + make install though. Which we don't have yet. - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Sunday, March 14, 2021 1:29 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] packaging formats and sets? On Sat, Mar 13, 2021 at 11:51:03PM +0000, Jay K wrote: > I have a nagging feeling I went overboard here, and the result is too confusing. > > reference Release 20210312 (March 12 2021) ? modula3/cm3 (github.com) > > For Windows we have roughly: > > cm3-min-win32.zip > cm3-min-win32-symbols.zip > cm3-all-win32.zip > cm3-all-win32-symbols.zip > cm3-min-win32.msi > cm3-all-win32.msi > cm3-boot.tgz > > For Unix we have roughly: > > cm3-min-win32.tgz > cm3-all-win32.tgz > cm3-min-win32.deb > cm3-all-win32.deb > cm3-boot.tgz I'm curious wh the Unix downloads have win32 in thir names. > > > The .deb and .msi files are little more than all.tgz. > The .deb is compressed better, to half the size. > The .msi installs to c:\cm3 (user selectable?) and adds c:\cm3\bin to %PATH%. > Something like that. > > min is just cm3 + mklib + m3core + libm3 + maybe cm3cg (gcc backend) + config files > all is everything I could build > boot is only enough to build cm3 with minimal dependnecies, i.e. a C toolset > boot is perhaps esoteric but also quite important > It could be augmented easily to build m3core and libm3, it already > has their source or assembly > > In the 3.6 release there was just boot per target that was assembly source, > and C for m3build I think. (Critical Mass reimplemened quake in Modula-3.) > > Plus the full source release. > > It was good enough then? Or people wants things easier these days? > And there are fewer platforms today so that makes it easier? > > I generally do not strip Unix binaries, as debugging matters too much. > > Current Windows builds, whatever I am running, contain tar, that can create/extract tgz. > I know GUI zip support goes back longer, but there remains no command line zip support. > I have to add a zip from where to create the packages. > > It seems well motivated, but seems like it is too much also. > > Thoughts? > > My thinking drifts towards: > 1. Eliminate "min"? > 2. Combine symbols with binaries? > 3. Consider eliminating deb and msi? > I realize they are pretty nice to have though. .deb is quite important -- there are a lot of Linux distros that are derived from Debian. I'd try to avoid systemd dependencies, though. > It might also be easy enough..would require some research to > make a very small .msi that downloads the all.tgz or something. > I put together the .msi and .deb very trivially. Does the .deb contain the executables? Or just the boot and source code so that the main work of the installation is done in a post-install script? If the latter, mke sure you have install-time dependencies on all the tools you'll need. > 4. Consider replacing tgz with tar.xz or similar. That might require a new install-time dependency. > > I don't know. As I said, everything has some motivation. > min is much smaller than all, etc. There might be modula3 bindings for significant libraries that some installations do not use or need (such as fancy database systems and the like). Those bindings should be in separate packages -- otherwise you'll end up making those big subsystems mandatory even for users who do not need them. -- hendrik > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4034222522d247de6c1c08d8e6ed54f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637513254304337788%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eea%2B0hItjEzpez0h6K6HMQC1bPzLn215sh1TuUSUNPA%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4034222522d247de6c1c08d8e6ed54f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637513254304337788%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eea%2B0hItjEzpez0h6K6HMQC1bPzLn215sh1TuUSUNPA%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 14 19:30:41 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 18:30:41 +0000 Subject: [M3devel] paths .m3 => mg? In-Reply-To: References: , , Message-ID: I have the ../derived/foo.m3 => ../src/foo.mg thing reasonably worked around in the C backend now. Now I notice that ../derived/foo.m3 occurs more, because presumably it is the source being built. I don't have a solution for this. One "easy" solution is to have ../generic-derived/foo.m3 ../generic-derived/target-specific-drived/foo.obj But this could be problematic. - You have to be certain foo.m3 is target-independent..but that is kinda the point, to make it happen, verify it once, and then either assume it or continue to confirm it. - it breaks building or cleaning multiple targets concurrently; maybe that is ok - it means clean and/or realclean might need to change; might. And maybe that is ok. I'd think the paths could just be ./foo.m3 or foo.m3, but the builder logic is a bit difficult to read through. I made some guesses for this but nothing worked yet. Another option is to assume the sameness, make up the targets C_UNIX64_EL, etc., and then make it so. Except, I'm not sure this is the right direction, in that the .obj file and such do still need somewhere to be stored. So we do need to specify "AMD64_LINUX" as an output directory somewhere. That is, even if the .c files are identical and we achieve portable distribution format, build would still have a default per-target output directory. In terms of line directives I think I can just hack it to remove the prefix. But that still leaves these paths occuring in some of the globals. I think for asserts. This is embedded by value in a larger struct. It is conceivable I could change it to a pointer and change it to reference "__FILE__". I'll look at it more.. - Jay ________________________________ From: Jay K Sent: Saturday, March 13, 2021 4:54 PM To: Mika Nystrom Cc: m3devel Subject: Re: [M3devel] paths .m3 => mg? But..these paths are sent to the backend..and backend's rarely issue any diagnostics. Assuming m3front has done a good job. The below is an exception, and I cannot explain it easily. I can imagine seeing both paths is more useful when m3front finds mistakes, which is more common than the backend finding mistakes? That said, these paths are used by the C for line directives and, except for these cases, contribute greatly to a decent debugging experience. You can step through the lines of Modula-3. These wierd paths break that, and "C convergence". On the theory that the paths are formed in only one place, both for m3front diagnostics and the backend, I guess I'll just change them in the C backend. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 13, 2021 2:49 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] paths .m3 => mg? Jay, the use of generics in libm3 et al. is pretty simplistic, just some ADTs, really. In general you can do much more complicated things with generics, so seeing names of both the .m3 and .mg can be very useful.. On Fri, Mar 12, 2021, 10:04 PM Jay K > wrote: cm3: ../AMD64_LINUX/FinDateArraySort.m3 => /cm3/pkg/libm3/src/sort/ArraySort.mg:86:0: warning: '' may be used uninitialized in this function [-Wmaybe-uninitialized] Nevermind the actual warning. The "foo.m3 => bar.mg" is one source file path, synthesized by m3front. That entire strong, with two embedded paths, is considered one path. Why? Is this useful? What were they thinking? The .m3 is what is being compiled. But is just one line, is not useful to the programmer receiving warnings/errors/assertion failures. The .mg is a "real" useful source file. If that was the only thing printed, I think people would be satisified. It is a *little* useful to see the other line to see the generic parameter. Perhaps?? So why the overall thing? I am guessing, that it is a problem, for the same source file to be compiled more than once?? So some "form" of the right is needed?? Or maybe the "little useful" part?? C systems have to deal with this anyway: echo "int name() { return value ;}" > foo.c cc -c -Dname=one -Dvalue=1 foo.c cc -c -Dname=two -Dvalue=2 foo.c cc -c -Dname=three -Dvalue=3oops foo.c => error foo.c:"3oops" and the ambiguity of errors in foo.c, if you don't see the command line, well, nevermind? But I guess that has no bearing on Modula3 and perhaps the same file compiled twice would confuse the build system? So I guess I'll mostly leave this alone. I have to change it, somewhat, to unify C targets, i.e. the "AMD64_LINUX" part is bad. I can just remove it or change it to or , for the C backend only. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Mar 14 20:51:10 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 14 Mar 2021 14:51:10 -0500 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: References: <520141615711185@mail.yandex.by> Message-ID: <80dace68-3cda-e1f0-0271-0bba13b1527e@lcwb.coop> Yes, removing the BITS 64 FOR is the right fix here. And yes, it doesn't change anything the code does. There is a subtle language and implementation issue here: Since a formal is never a record field or array element, BITS ... FOR will never affect its size or alignment. These will be selected by the compiler as usual. But a corresponding actual of type BITS n FOR.., even with the same n, could be a field or element and have an alignment of only one bit. And different formals in different calls could differ from each otner. This would mean pass by reference would have to be via a variable bit pointer, and all the uses in the procedure would have to handle it. This is not a reasonable implementation. It could be accepted in specific cases, this case in point, if the actual that had the same alignment as the natural alignment as the formal. But excessively conditional rules like this in a language is bad language design and a start down the slippery slope that leads to programming languages with 300, 400, or 500 pages of definition that almost nobody understands. So when I encountered problems with this in the existing compiler, I fixed it with the unconditional rule that a perameter passed VAR cannot be BITS FOR. No such restriction on VAR or READONLY. This is within Modula-3's allowing of restrictions on packed types. I have started writing a statement of all the implementation restrictions regarding them, which belongs in implementation documentation. On 3/14/21 3:53 AM, Jay K wrote: > Looks like we can just remove the BITS FOR statement. > Which makes the new rules seem, kinda pointless at least in this case. > BITS FOR didn't actually change anything, but validity. In this case. > > ?- Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* vvm at tut.by > *Sent:* Sunday, March 14, 2021 8:43 AM > *To:* Jay K ; m3devel > *Subject:* Re: [M3devel] win32 vs. new packing rules? > Hi! > > > ?I didn't compile "packed" variant M3 now. > > But I have big amount of errors when I compiled it in september of 2019 and march of 2020. > > Best regards, Victor Miasnikov > > 14.03.2021, 11:15, "Jay K" : > > win32 system appears to be suffer with the new packed rules? > > new source -> compiling Time.m3 > "..\src\time\WIN32\Time.m3", line 18: CM3 restriction: VAR actual cannot be packed. (2.3.2): ?(FileTime) > 1 error encountered > m3front failed compiling: ..\src\time\WIN32\Time.m3 > new source -> compiling DateWin32.m3 > "..\src\time\WIN32\DateWin32.m3", line 65: CM3 restriction: VAR actual cannot be packed. (2.3.2): ?(ft) > "..\src\time\WIN32\DateWin32.m3", line 137: CM3 restriction: VAR actual cannot be packed. (2.3.2): ?(ft) > 2 errors encountered > > > , > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Sun Mar 14 21:18:29 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 20:18:29 +0000 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: <80dace68-3cda-e1f0-0271-0bba13b1527e@lcwb.coop> References: <520141615711185@mail.yandex.by> , <80dace68-3cda-e1f0-0271-0bba13b1527e@lcwb.coop> Message-ID: Thank you. I understand it might be complicated. My naive idea is "run the logic twice both ways and see if it the same". But "the logic" might devolve to "the entire compiler". Not a good answer. Perhaps, and I am guessing, the logic could be: Run the logic just once, with the stated BITS for, and notice if there is any padding inserted in between any fields, or at the end of the record, and if no to both, then it would be same either way? In terms of documentation, I would like clarity on ABI. Aka clarity on "what does look like in C". i.e. clarify on pass by value vs. pointer. e.g. what does "READONLY" mean? Perhaps it means "by reference like VAR but readonly"?? I understand it might be complicated, because of passing/returning records by semantic value, and sets. I understand there is some desire to separate semantics ("mutable reference or value") from implementations details ("pointer or not"). But I cannot avoid wanting to know. Sorry can you repeat though, what are the rules (I guess you are going to write it up) and maybe the rationale? I am not sure "the language spec says so" is always the best rationale. In particular, if you look at C (sorry, but this is a good idea, I will always believe), you can see a few things: - the official standard is very vague about padding and alignment - In actual implementations, there is a lot of flexibility and a lot works. In particular, you can align many things in many ways, and no matter what, you can pass/return records/structs by value or pointer. Alignment/padding/packing does not affect what you can pass. Well, at least in terms of entire records. There are generally restrictions around member data/fields crossing alignment boundaries, and you cannot..and maybe this is the crux of it..you cannot take the address of a bitfield. So I *gather* what Modula-3 is doing is trying to capture that last thing. Inability to address bitfields. You might quibble, that sometimes bitfields end up reasonable to address. But C just bars it entirely. Is that what is going on here? Teasing out some VAR-ability of bitfields, if they end up nicely aligned? But then, there should be no impact on the overall RECORD, right? Or RECORDs might themselves kinda look like bitfields? Can I say this: TYPE OneBit = BITS 1 FOR RECORD bool: BITS 1 FOR BOOLEAN; END; TWO TwoBits = BITS 2 FOR RECORD a,b: BITS 1 FOR OneBit; END; Some redundancy there? TYPE Byte = ARRAY [0..3] OF TwoBits; <*ASSERT BYTESIZE(OneBit) = 1*> <*ASSERT BYTESIZE(TwoBits) = 1*> <*ASSERT BYTESIZE(Byte) = 1*> b: Byte; b[1].b.bool ? Is that what these rules enable? Which admittedly you cannot do in C. And I even had a use case for a few years ago. Encoding fonts in constant data. C bitfields aren't great for that. In C structs and array elements have BITSIZE MOD 8 == 0. (On some ARM targets there is apparently a notion of worse.) Except one time though the byte-size granularity of C structs and array elements has seemed fine. - Jay ________________________________ From: Rodney M. Bates Sent: Sunday, March 14, 2021 7:51 PM To: Jay K ; vvm at tut.by ; m3devel Subject: Re: [M3devel] win32 vs. new packing rules? Yes, removing the BITS 64 FOR is the right fix here. And yes, it doesn't change anything the code does. There is a subtle language and implementation issue here: Since a formal is never a record field or array element, BITS ... FOR will never affect its size or alignment. These will be selected by the compiler as usual. But a corresponding actual of type BITS n FOR.., even with the same n, could be a field or element and have an alignment of only one bit. And different formals in different calls could differ from each otner. This would mean pass by reference would have to be via a variable bit pointer, and all the uses in the procedure would have to handle it. This is not a reasonable implementation. It could be accepted in specific cases, this case in point, if the actual that had the same alignment as the natural alignment as the formal. But excessively conditional rules like this in a language is bad language design and a start down the slippery slope that leads to programming languages with 300, 400, or 500 pages of definition that almost nobody understands. So when I encountered problems with this in the existing compiler, I fixed it with the unconditional rule that a perameter passed VAR cannot be BITS FOR. No such restriction on VAR or READONLY. This is within Modula-3's allowing of restrictions on packed types. I have started writing a statement of all the implementation restrictions regarding them, which belongs in implementation documentation. On 3/14/21 3:53 AM, Jay K wrote: > Looks like we can just remove the BITS FOR statement. > Which makes the new rules seem, kinda pointless at least in this case. > BITS FOR didn't actually change anything, but validity. In this case. > > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* vvm at tut.by > *Sent:* Sunday, March 14, 2021 8:43 AM > *To:* Jay K ; m3devel > *Subject:* Re: [M3devel] win32 vs. new packing rules? > Hi! > > > I didn't compile "packed" variant M3 now. > > But I have big amount of errors when I compiled it in september of 2019 and march of 2020. > > Best regards, Victor Miasnikov > > 14.03.2021, 11:15, "Jay K" : > > win32 system appears to be suffer with the new packed rules? > > new source -> compiling Time.m3 > "..\src\time\WIN32\Time.m3", line 18: CM3 restriction: VAR actual cannot be packed. (2.3.2): (FileTime) > 1 error encountered > m3front failed compiling: ..\src\time\WIN32\Time.m3 > new source -> compiling DateWin32.m3 > "..\src\time\WIN32\DateWin32.m3", line 65: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) > "..\src\time\WIN32\DateWin32.m3", line 137: CM3 restriction: VAR actual cannot be packed. (2.3.2): (ft) > 2 errors encountered > > > , > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C54695049a0964cfd4d8f08d8e7228c6d%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637513482871390262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=G0bZE5LWnrsoVbEXOJOhPMZmYtAhnNai4i7QGDmDviE%3D&reserved=0 > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C54695049a0964cfd4d8f08d8e7228c6d%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637513482871390262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=G0bZE5LWnrsoVbEXOJOhPMZmYtAhnNai4i7QGDmDviE%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Mar 14 21:31:28 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 14 Mar 2021 15:31:28 -0500 Subject: [M3devel] assert false "..\src\exprs\CallExpr.m3", line 179 In-Reply-To: References: Message-ID: Should be fixed in bb5f298af172e8d67d8e155fdc3ec42c1b140699 On 3/14/21 7:05 AM, Jay K wrote: > This is my mistake: > > PROCEDURE SetupNamingConventions (mach : Quake.Machine) = > ? BEGIN > ? ? (* Refactor to avoid creating garbage. *) > ? ? SetupNamingConventionsInternal (NEW (State), mach); > ? END SetupNamingConventions; > > > > ..\src\Builder.m3", line 260: VAR actual must be a designator (2.3.2) ?(s) > > > this is maybe someone else's? > > > > *** > *** runtime error: > *** ? ?<*ASSERT*> failed: FALSE > *** ? ?file "..\src\exprs\CallExpr.m3", line 179 > *** > PROCEDURE NotAddressable (<*UNUSED*> t: T) = > ? BEGIN > ? ? <* ASSERT FALSE *> > ? END NotAddressable; > > > Well recording it here anyway. > > NEW (State) was initializing a VAR and I guess it most. > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Mon Mar 15 00:43:30 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 14 Mar 2021 23:43:30 +0000 Subject: [M3devel] can text be a record? Message-ID: Is it possible to make TEXT into a record? Not even a ref record? Because, then, Text.Sub would not need to make a heap allocation? Just copy around a type pointer, a start pointer (possibly offset from the start of another text), and a length? Lately other systems calls this "span" or something else, but, I don't mean just a pointer and length, but a type pointer and data pointer and length, to accomodate the unicode stuff. I'd like to see if a certain string occors within another string, but at an offset. I see this idiom which seems slow, even for a simpler operation: PROCEDURE StartsWith(s, t : TEXT; caseSensitive := TRUE) : BOOLEAN = VAR tlen := Text.Length(t); slen := Text.Length(s); sub: TEXT; BEGIN IF tlen > slen THEN RETURN FALSE; END; sub := Text.Sub(s, 0, tlen); <= ouch? IF caseSensitive THEN RETURN Text.Equal(sub, t); END; RETURN TextExtras_CIEqual(sub, t); END StartsWith; Otherwise I think these "need" to be written using loops over GetChar, which is ok, but also not my preference. i.e. I'd like this StartsWith to be efficient and allocation-free, as it is already written. C++ tends not to work this way, because the lifetime of the pointer is a problem. But garbage collected systems I think tend to be able to do it. Sometimes C++ uses reference counting though. There is no free lunch. Reference counting, copying, and garbage collection are all costly. Sometimes the lifetime can be somehow guaranteed without any of those. For now I'll just use loops. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 08:47:25 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 07:47:25 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? Message-ID: The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 08:58:33 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 10:58:33 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: Message-ID: <620021615794703@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 09:13:25 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 08:13:25 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: <620021615794703@mail.yandex.by> References: , <620021615794703@mail.yandex.by> Message-ID: The dependency on C is..I tried to explain it earlier... there were two reasons: 1. The better one: avoid rewriting /usr/include 1. Build system machinations. That is the case here..but not even that really. I produce these "bootstrap C" archives. They consist mostly of Modula-3 compiled to C. But for any actual C in the system, the files are just copied. That is an ancient functionality. I noticed, and leverage profitably. Then, on the eventual host system, you compile there. The C is ifdefed. Or it includes /usr/include. Either way, the point is to defer resolution to the eventual host system. I am actually a bit fan of the C preprocessor. I recognize its effects on the language, in that it interweaves a separate scopeless language, and it slows things down. It is a kinda horrible, but it is also very useful both for macros and conditional compilation. Conditional compilation, sometimes, not always, is "better" than separate source files. (Part of the horribleness is that to learn C and C++ you also have to learn this other unrelated language.) But, point being, when I bring these things up, everyone kinda gets upset. Modula-3 doesn't have a preprocessor people like it that way. I do see the advantages, and the disadvantages. Some of the disadvantages are mitigated by Quake. But not all. But in the unlikely case that there was some decision to add a preprocessor, it'd have to be designed and implemented. All three are unlikely. So that is why I do that. But it isn't just C. It is about constant vs. variable. You can also detect endian at runtime in Modula-3 with loophole. Changing these to runtime checks in Modula-3 is also ok. For example you could also get the environment variable $OS and if it is Windows_NT then you are on Win32, otherwise Unix. A little gross, but ok? ThisEndian is used very little, but at first glance, the code does want a constant. Unless maybe I can ripple through a const to var change there. Sometimes const is just convenient and efficient, sometimes it is required. (Notice how I changed all the errno values from const to var, so I converted case statements to if/else if ladders, less efficient but probably good enough.) To repeat, it is more about const vs. var, than C vs. Modula-3. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 7:58 AM To: Jay K ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } Used quite a bit. There are two values: Win32 and Posix. Q: Cygwin is Posix? } Move (...) to C under #ifdef _WIN32 Again we depends "C language"... Exist preprocessor write on Modila-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. If need "ifdef" then we can use this preprocessor/ Best regards, Victor Miasnikov 15.03.2021, 10:47, "Jay K" : The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 09:29:55 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 11:29:55 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> Message-ID: <665341615796681@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 09:40:11 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 08:40:11 +0000 Subject: [M3devel] sbrk vs. mmap Message-ID: Wow. So, you know, I've been diffing platform "object code" ( C :) ). For now RISCV64_LINUX to AMD64_LINUX, to vary the least..and what do I find: readonly RTOSbrk = { % % memory from mmap seemed to be too spread out for the allocator's data structures to describe % "AMD64_LINUX" : 1, % % Several platforms don't define MAP_ANON or MAP_ANONYMOUS or even have Umman, % and sbrk was the historical implementation, but we probably don't care. % see http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/POSIX/RTOS.m3?rev=1.1;content-type=text%2Fplain % % This is very much a guess and should be confirmed if systems are available. % "AIX386" : 1, "IBMR2" : 1, "IBMRT" : 1, "IRIX5" : 1, % Outdated? Seems not. "SPARC" : 1, % SunOS 4.x } I have little direct recollection of this, esp. the AMD64_LINUX part, but.. c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 13) % memory from mmap seemed to be too spread out for the allocator's data structures to describe c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 14) % c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 15) "AMD64_LINUX" : 1, Does anyone believe that? That we need sbrk here? On this one heavily used platform? I mean. I guess, sbrk will just keep incrementing or decrementing. mmap could be kinda random. But this seems kinda crazy/horrible to be using these days. On just one platform (the others there don't count). I both hate to change this and hate to leave it this way. I'll try changing it. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 09:41:22 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 08:41:22 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: <665341615796681@mail.yandex.by> References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by> Message-ID: I didn't say unicode is unimportant. Though, really, it probably is. UTF8 has gained large traction. I just mean, there are few users, so maybe changing things, and keeping things working, isn't difficult. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 8:29 AM To: Jay K ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } To repeat, it is more about const vs. var, than C vs. Modula-3. I hope... }git grep ThisEndian } Used very little -- libunicode libunicode is very important. VVM}} Exist preprocessor write on Modula-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. } Modula-3 doesn't have a preprocessor (...) } But in the unlikely case that there was some decision to add a preprocessor Ok, may be implement "D language" "ifdef style"? Best regards, Victor Miasnikov 15.03.2021, 11:13, "Jay K" : The dependency on C is..I tried to explain it earlier... there were two reasons: 1. The better one: avoid rewriting /usr/include 1. Build system machinations. That is the case here..but not even that really. I produce these "bootstrap C" archives. They consist mostly of Modula-3 compiled to C. But for any actual C in the system, the files are just copied. That is an ancient functionality. I noticed, and leverage profitably. Then, on the eventual host system, you compile there. The C is ifdefed. Or it includes /usr/include. Either way, the point is to defer resolution to the eventual host system. I am actually a bit fan of the C preprocessor. I recognize its effects on the language, in that it interweaves a separate scopeless language, and it slows things down. It is a kinda horrible, but it is also very useful both for macros and conditional compilation. Conditional compilation, sometimes, not always, is "better" than separate source files. (Part of the horribleness is that to learn C and C++ you also have to learn this other unrelated language.) But, point being, when I bring these things up, everyone kinda gets upset. Modula-3 doesn't have a preprocessor people like it that way. I do see the advantages, and the disadvantages. Some of the disadvantages are mitigated by Quake. But not all. But in the unlikely case that there was some decision to add a preprocessor, it'd have to be designed and implemented. All three are unlikely. So that is why I do that. But it isn't just C. It is about constant vs. variable. You can also detect endian at runtime in Modula-3 with loophole. Changing these to runtime checks in Modula-3 is also ok. For example you could also get the environment variable $OS and if it is Windows_NT then you are on Win32, otherwise Unix. A little gross, but ok? ThisEndian is used very little, but at first glance, the code does want a constant. Unless maybe I can ripple through a const to var change there. Sometimes const is just convenient and efficient, sometimes it is required. (Notice how I changed all the errno values from const to var, so I converted case statements to if/else if ladders, less efficient but probably good enough.) To repeat, it is more about const vs. var, than C vs. Modula-3. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 7:58 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } Used quite a bit. There are two values: Win32 and Posix. Q: Cygwin is Posix? } Move (...) to C under #ifdef _WIN32 Again we depends "C language"... Exist preprocessor write on Modila-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. If need "ifdef" then we can use this preprocessor/ Best regards, Victor Miasnikov 15.03.2021, 10:47, "Jay K" >: The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 09:50:40 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 08:50:40 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, Message-ID: > Q: Cygwin is Posix? I think so. The difference is mainly this exact line in several m3makefiles: include_dir (OS_TYPE) So then using open/read/write/close vs. CreateFile/ReadFile/WriteFile/CloseHandle. And pthreads vs. Win32 threads (which can probably be unified, if you drop pre-Vista support). And look for "OnUnix" throughout the tree, which is mainly about chosing '/' or '\\', even though Win32 supports '/' just fine. So it is a mostly pointless distinction. Except, if you print the paths, like in errors, '/' might work in notepad or on the command line etc. Because too much code does too much path manipulation and does it incorrectly. Win32 CreateFile et. al. support '/'. It is layers above it that sometimes break it. The problem with Cygwin in my experience is the noticably slow fork. The Interix port was noticably much faster. But these days, WSL anyway. I see little point in Cygwin now. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 8:41 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? I didn't say unicode is unimportant. Though, really, it probably is. UTF8 has gained large traction. I just mean, there are few users, so maybe changing things, and keeping things working, isn't difficult. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 8:29 AM To: Jay K ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } To repeat, it is more about const vs. var, than C vs. Modula-3. I hope... }git grep ThisEndian } Used very little -- libunicode libunicode is very important. VVM}} Exist preprocessor write on Modula-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. } Modula-3 doesn't have a preprocessor (...) } But in the unlikely case that there was some decision to add a preprocessor Ok, may be implement "D language" "ifdef style"? Best regards, Victor Miasnikov 15.03.2021, 11:13, "Jay K" : The dependency on C is..I tried to explain it earlier... there were two reasons: 1. The better one: avoid rewriting /usr/include 1. Build system machinations. That is the case here..but not even that really. I produce these "bootstrap C" archives. They consist mostly of Modula-3 compiled to C. But for any actual C in the system, the files are just copied. That is an ancient functionality. I noticed, and leverage profitably. Then, on the eventual host system, you compile there. The C is ifdefed. Or it includes /usr/include. Either way, the point is to defer resolution to the eventual host system. I am actually a bit fan of the C preprocessor. I recognize its effects on the language, in that it interweaves a separate scopeless language, and it slows things down. It is a kinda horrible, but it is also very useful both for macros and conditional compilation. Conditional compilation, sometimes, not always, is "better" than separate source files. (Part of the horribleness is that to learn C and C++ you also have to learn this other unrelated language.) But, point being, when I bring these things up, everyone kinda gets upset. Modula-3 doesn't have a preprocessor people like it that way. I do see the advantages, and the disadvantages. Some of the disadvantages are mitigated by Quake. But not all. But in the unlikely case that there was some decision to add a preprocessor, it'd have to be designed and implemented. All three are unlikely. So that is why I do that. But it isn't just C. It is about constant vs. variable. You can also detect endian at runtime in Modula-3 with loophole. Changing these to runtime checks in Modula-3 is also ok. For example you could also get the environment variable $OS and if it is Windows_NT then you are on Win32, otherwise Unix. A little gross, but ok? ThisEndian is used very little, but at first glance, the code does want a constant. Unless maybe I can ripple through a const to var change there. Sometimes const is just convenient and efficient, sometimes it is required. (Notice how I changed all the errno values from const to var, so I converted case statements to if/else if ladders, less efficient but probably good enough.) To repeat, it is more about const vs. var, than C vs. Modula-3. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 7:58 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } Used quite a bit. There are two values: Win32 and Posix. Q: Cygwin is Posix? } Move (...) to C under #ifdef _WIN32 Again we depends "C language"... Exist preprocessor write on Modila-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. If need "ifdef" then we can use this preprocessor/ Best regards, Victor Miasnikov 15.03.2021, 10:47, "Jay K" >: The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 09:53:17 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 08:53:17 +0000 Subject: [M3devel] Windows Trestle breakage? Message-ID: This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 10:27:06 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 12:27:06 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by> Message-ID: <242351615799934@mail.yandex.by> An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 10:35:44 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 12:35:44 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, Message-ID: <232461615800513@mail.yandex.by> An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 10:42:47 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 12:42:47 +0300 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: References: Message-ID: <698741615801031@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 10:46:49 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 09:46:49 +0000 Subject: [M3devel] cm3 command line vs. config? Message-ID: I'd like to say cm3 -DBUILD_DIR=foo but the order of evaluation..and scoping, in quake is all kinda wierd? It evaluates the command line before the config file..but the config file seems to run in a nested scope? So the if not defined("BUILD_DIR") readonly BUILD_DIR = x runs and then the command line is evaluated a second time, and there is an error assigning a read only variable. I think I put in the early evaluation. Which still makes sense to me. But maybe I did it wrong? I'll dig more. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 10:48:05 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 09:48:05 +0000 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: <698741615801031@mail.yandex.by> References: , <698741615801031@mail.yandex.by> Message-ID: Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 10:51:18 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 09:51:18 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: <232461615800513@mail.yandex.by> References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, , <232461615800513@mail.yandex.by> Message-ID: Well it is off topic, but WSL should be in server builds, and seriously try it. I am quite a happy user. > And Cygwin is useful as environment for testing M3. Well, we can test arbitrary environments. Is it something people use other than for testing? Or does it simulate closely enough something they do? That is not clear at all. Yes it simulates Posix, but maybe not closely enough. And you can use "closer simulations" pretty easy and faster, i.e. WSL or a Linux VM etc. It is all subjective, I realize, but Cygwin really was painfully slow in the past. Maybe better..if...with the C backend..and if I run gcc just once per library instead of per source, which I've been wanting to do anyway. There is a performance tradeoff there depending on environment. like in Qemu I think it JITs gcc each time, so better to run it once. Though there is an incrementality problem them. Anyway, later... - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:35 AM To: Jay K ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } WSL anyway. WSL doesn't include in ReactOs, Windows Server 2019 and Windows Server 2016. May be exist in Windows Server 2022 "Insider"/Beta version. } I see little point in Cygwin now And Cygwin is useful as environment for testing M3. Best regards, Victor Miasnikov P.S. } '/' might work in notepad or on the command line etc. I known -), an even often use it. 15.03.2021, 11:51, "Jay K" : > Q: Cygwin is Posix? I think so. The difference is mainly this exact line in several m3makefiles: include_dir (OS_TYPE) So then using open/read/write/close vs. CreateFile/ReadFile/WriteFile/CloseHandle. And pthreads vs. Win32 threads (which can probably be unified, if you drop pre-Vista support). And look for "OnUnix" throughout the tree, which is mainly about chosing '/' or '\\', even though Win32 supports '/' just fine. So it is a mostly pointless distinction. Except, if you print the paths, like in errors, '/' might work in notepad or on the command line etc. Because too much code does too much path manipulation and does it incorrectly. Win32 CreateFile et. al. support '/'. It is layers above it that sometimes break it. The problem with Cygwin in my experience is the noticably slow fork. The Interix port was noticably much faster. But these days, WSL anyway. I see little point in Cygwin now. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Monday, March 15, 2021 8:41 AM To: vvm at tut.by >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? I didn't say unicode is unimportant. Though, really, it probably is. UTF8 has gained large traction. I just mean, there are few users, so maybe changing things, and keeping things working, isn't difficult. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 8:29 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } To repeat, it is more about const vs. var, than C vs. Modula-3. I hope... }git grep ThisEndian } Used very little -- libunicode libunicode is very important. VVM}} Exist preprocessor write on Modula-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. } Modula-3 doesn't have a preprocessor (...) } But in the unlikely case that there was some decision to add a preprocessor Ok, may be implement "D language" "ifdef style"? Best regards, Victor Miasnikov 15.03.2021, 11:13, "Jay K" >: The dependency on C is..I tried to explain it earlier... there were two reasons: 1. The better one: avoid rewriting /usr/include 1. Build system machinations. That is the case here..but not even that really. I produce these "bootstrap C" archives. They consist mostly of Modula-3 compiled to C. But for any actual C in the system, the files are just copied. That is an ancient functionality. I noticed, and leverage profitably. Then, on the eventual host system, you compile there. The C is ifdefed. Or it includes /usr/include. Either way, the point is to defer resolution to the eventual host system. I am actually a bit fan of the C preprocessor. I recognize its effects on the language, in that it interweaves a separate scopeless language, and it slows things down. It is a kinda horrible, but it is also very useful both for macros and conditional compilation. Conditional compilation, sometimes, not always, is "better" than separate source files. (Part of the horribleness is that to learn C and C++ you also have to learn this other unrelated language.) But, point being, when I bring these things up, everyone kinda gets upset. Modula-3 doesn't have a preprocessor people like it that way. I do see the advantages, and the disadvantages. Some of the disadvantages are mitigated by Quake. But not all. But in the unlikely case that there was some decision to add a preprocessor, it'd have to be designed and implemented. All three are unlikely. So that is why I do that. But it isn't just C. It is about constant vs. variable. You can also detect endian at runtime in Modula-3 with loophole. Changing these to runtime checks in Modula-3 is also ok. For example you could also get the environment variable $OS and if it is Windows_NT then you are on Win32, otherwise Unix. A little gross, but ok? ThisEndian is used very little, but at first glance, the code does want a constant. Unless maybe I can ripple through a const to var change there. Sometimes const is just convenient and efficient, sometimes it is required. (Notice how I changed all the errno values from const to var, so I converted case statements to if/else if ladders, less efficient but probably good enough.) To repeat, it is more about const vs. var, than C vs. Modula-3. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 7:58 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } Used quite a bit. There are two values: Win32 and Posix. Q: Cygwin is Posix? } Move (...) to C under #ifdef _WIN32 Again we depends "C language"... Exist preprocessor write on Modila-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. If need "ifdef" then we can use this preprocessor/ Best regards, Victor Miasnikov 15.03.2021, 10:47, "Jay K" >: The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 11:10:09 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 13:10:09 +0300 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by> Message-ID: <681511615802339@mail.yandex.by> An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 11:18:19 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 13:18:19 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, , <232461615800513@mail.yandex.by> Message-ID: <246151615803158@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 11:21:09 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 10:21:09 +0000 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: <246151615803158@mail.yandex.by> References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, , <232461615800513@mail.yandex.by> , <246151615803158@mail.yandex.by> Message-ID: WSL1 has been around for several years now, and WSL2 a while now too. I don't think I'm using an Insider build. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 10:18 AM To: Jay K ; m3devel Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } you can use "closer simulations" pretty easy and faster, i.e. WSL or a Linux VM etc. Yes: support Cygwin is "only alternative" } but WSL should be in server builds "WSL should" stay true for non-insiders in September 2021 -) Best regards, Victor Miasnikov 15.03.2021, 12:51, "Jay K" : Well it is off topic, but WSL should be in server builds, and seriously try it. I am quite a happy user. > And Cygwin is useful as environment for testing M3. Well, we can test arbitrary environments. Is it something people use other than for testing? Or does it simulate closely enough something they do? That is not clear at all. Yes it simulates Posix, but maybe not closely enough. And you can use "closer simulations" pretty easy and faster, i.e. WSL or a Linux VM etc. It is all subjective, I realize, but Cygwin really was painfully slow in the past. Maybe better..if...with the C backend..and if I run gcc just once per library instead of per source, which I've been wanting to do anyway. There is a performance tradeoff there depending on environment. like in Qemu I think it JITs gcc each time, so better to run it once. Though there is an incrementality problem them. Anyway, later... - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 9:35 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } WSL anyway. WSL doesn't include in ReactOs, Windows Server 2019 and Windows Server 2016. May be exist in Windows Server 2022 "Insider"/Beta version. } I see little point in Cygwin now And Cygwin is useful as environment for testing M3. Best regards, Victor Miasnikov P.S. } '/' might work in notepad or on the command line etc. I known -), an even often use it. 15.03.2021, 11:51, "Jay K" >: > Q: Cygwin is Posix? I think so. The difference is mainly this exact line in several m3makefiles: include_dir (OS_TYPE) So then using open/read/write/close vs. CreateFile/ReadFile/WriteFile/CloseHandle. And pthreads vs. Win32 threads (which can probably be unified, if you drop pre-Vista support). And look for "OnUnix" throughout the tree, which is mainly about chosing '/' or '\\', even though Win32 supports '/' just fine. So it is a mostly pointless distinction. Except, if you print the paths, like in errors, '/' might work in notepad or on the command line etc. Because too much code does too much path manipulation and does it incorrectly. Win32 CreateFile et. al. support '/'. It is layers above it that sometimes break it. The problem with Cygwin in my experience is the noticably slow fork. The Interix port was noticably much faster. But these days, WSL anyway. I see little point in Cygwin now. - Jay ________________________________ From: M3devel > on behalf of Jay K > Sent: Monday, March 15, 2021 8:41 AM To: vvm at tut.by >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? I didn't say unicode is unimportant. Though, really, it probably is. UTF8 has gained large traction. I just mean, there are few users, so maybe changing things, and keeping things working, isn't difficult. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 8:29 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } To repeat, it is more about const vs. var, than C vs. Modula-3. I hope... }git grep ThisEndian } Used very little -- libunicode libunicode is very important. VVM}} Exist preprocessor write on Modula-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. } Modula-3 doesn't have a preprocessor (...) } But in the unlikely case that there was some decision to add a preprocessor Ok, may be implement "D language" "ifdef style"? Best regards, Victor Miasnikov 15.03.2021, 11:13, "Jay K" >: The dependency on C is..I tried to explain it earlier... there were two reasons: 1. The better one: avoid rewriting /usr/include 1. Build system machinations. That is the case here..but not even that really. I produce these "bootstrap C" archives. They consist mostly of Modula-3 compiled to C. But for any actual C in the system, the files are just copied. That is an ancient functionality. I noticed, and leverage profitably. Then, on the eventual host system, you compile there. The C is ifdefed. Or it includes /usr/include. Either way, the point is to defer resolution to the eventual host system. I am actually a bit fan of the C preprocessor. I recognize its effects on the language, in that it interweaves a separate scopeless language, and it slows things down. It is a kinda horrible, but it is also very useful both for macros and conditional compilation. Conditional compilation, sometimes, not always, is "better" than separate source files. (Part of the horribleness is that to learn C and C++ you also have to learn this other unrelated language.) But, point being, when I bring these things up, everyone kinda gets upset. Modula-3 doesn't have a preprocessor people like it that way. I do see the advantages, and the disadvantages. Some of the disadvantages are mitigated by Quake. But not all. But in the unlikely case that there was some decision to add a preprocessor, it'd have to be designed and implemented. All three are unlikely. So that is why I do that. But it isn't just C. It is about constant vs. variable. You can also detect endian at runtime in Modula-3 with loophole. Changing these to runtime checks in Modula-3 is also ok. For example you could also get the environment variable $OS and if it is Windows_NT then you are on Win32, otherwise Unix. A little gross, but ok? ThisEndian is used very little, but at first glance, the code does want a constant. Unless maybe I can ripple through a const to var change there. Sometimes const is just convenient and efficient, sometimes it is required. (Notice how I changed all the errno values from const to var, so I converted case statements to if/else if ladders, less efficient but probably good enough.) To repeat, it is more about const vs. var, than C vs. Modula-3. - Jay ________________________________ From: vvm at tut.by > Sent: Monday, March 15, 2021 7:58 AM To: Jay K >; m3devel > Subject: Re: [M3devel] Compiler.ThisOS, Endian, Platform? Hi! } Used quite a bit. There are two values: Win32 and Posix. Q: Cygwin is Posix? } Move (...) to C under #ifdef _WIN32 Again we depends "C language"... Exist preprocessor write on Modila-2 ( i.e. we can convert it by m2tom3) for language with (* *) style comments. If need "ifdef" then we can use this preprocessor/ Best regards, Victor Miasnikov 15.03.2021, 10:47, "Jay K" >: The following are "problematic". By "problematic", I mean, they cause otherwise identical C backend output to vary. git grep ThisPlatform Not used at all (within Critical Mass). And the worst of them, because it is different for every system. git grep ThisOS Used quite a bit. There are two values: Win32 and Posix. git grep ThisEndian Used very little -- libunicode and ordsets, that is all. ordsets because pickles. I propose: Remove ThisPlatform. Move ThisOS to C under #ifdef _WIN32 Move ThisEndian to C under a runtime check. I understand, constant propagation would be lost. Oh, actually, ThisEndian's const-ness is heavily used by its few users. So maybe leave that alone for now. ThisOS I think can be made a function call easily enough. Or a variable that initialized early. It is true that Endian and OS are factors that are somewhat rampant in the system and fixing these instances is not enough to collapse the matrix. But ThisPlatform is trivial. It expands the matrix by n and isn't used at all. Thank you, - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 11:21:33 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 10:21:33 +0000 Subject: [M3devel] cm3 command line vs. config? In-Reply-To: References: Message-ID: oh sorry duh. Some of the config files say if not defined("BUILD_DIR") some do not. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:46 AM To: m3devel Subject: cm3 command line vs. config? I'd like to say cm3 -DBUILD_DIR=foo but the order of evaluation..and scoping, in quake is all kinda wierd? It evaluates the command line before the config file..but the config file seems to run in a nested scope? So the if not defined("BUILD_DIR") readonly BUILD_DIR = x runs and then the command line is evaluated a second time, and there is an error assigning a read only variable. I think I put in the early evaluation. Which still makes sense to me. But maybe I did it wrong? I'll dig more. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 11:28:56 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 13:28:56 +0300 Subject: [M3devel] Compiler.ThisOS, Endian, Platform? In-Reply-To: References: , <620021615794703@mail.yandex.by> , <665341615796681@mail.yandex.by>, , <232461615800513@mail.yandex.by> , <246151615803158@mail.yandex.by> Message-ID: <731641615803874@mail.yandex.by> An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Mon Mar 15 11:41:44 2021 From: dmuysers at hotmail.com (Dirk Muysers) Date: Mon, 15 Mar 2021 11:41:44 +0100 Subject: [M3devel] m3 code base Message-ID: The Klagenfurt (MS-DOS) M3 compiler was also C-based and didn`t make its team any happier. Maybe we should restart from PM3 and? and avoid C and dependency on proprietary platform tools as much as possible. Dear Jay, your C++ and MS evangelism is nerve-racking. And, please, think with your brain instead of your mailer. Mail us the the good news after you found nice solutions that respect the integrity of the Modula-3 language. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Mon Mar 15 11:53:35 2021 From: vvm at tut.by (vvm at tut.by) Date: Mon, 15 Mar 2021 13:53:35 +0300 Subject: [M3devel] How about ReactOs? Re: m3 code base In-Reply-To: References: Message-ID: <705211615805087@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 11:58:24 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 10:58:24 +0000 Subject: [M3devel] m3 code base In-Reply-To: References: Message-ID: > propietary You can use gcc or clang on Linux and Mac. Cygwin did work also on Windows, but it was slow. You can probably use that though. Or Mingw, I also had working, probably still does. The Critical Mass system always depended on the Visual C++ linker. You might try the LLVM lld these days. It probably works. But why? And even so, there are so many choices I don't understand the complaint. C dependency: Again, rewriting /usr/include is a bad idea. Really. Having the old gcc fork doesn't seem great. Or does it? A C backend can replace the gcc backend. You know, do you want ARM64_DARWIN? Or RISCV64_LINUX? Those were really trivial to get working. Or maybe LLVM is working and move to that? Maybe. Also, again, what I am trying to get to now, is easier to download and build. i.e. something kinda source-based, instead of binaries. Like if you imagine the world is all 64bit little endian Unix, then wget something.tar.gz -- for very few choices of something.tar.gz tar xf something.tar.gz cd something make make install Doesn't that seem worth something? But then after that, probably, the usual cm3 and m3makefiles. (Though generating cmake input seems maybe nice, to stop maintaining the config files..but the config files have also largely converged -- just always run gcc with the same switches). It partly comes down to, what do people want to maintain? The frontend? Multiple backends? A gcc fork? A..to describe it funny..a libtool clone? Garbage collector? Which parts are fun? Which are too costly? Which are just fine so nevermind leave it alone? And, perhaps, licensing problem and getting into Debian. There are many possible questions and answers. If people are happy with the unmaintained old gcc fork, then maybe nevermind. If LLVM is looking good, then also maybe nevermind. If people are happy requiring binaries to get started then also maybe nevermind. But those things do kinda bother me. There are other things too. Like the platform specific signal handling and thread suspend and get context. Might it might be nice if that was much less code and much more portable so whatever comes along, easier to just keep it running? Maybe, maybe not. - Jay ________________________________ From: M3devel on behalf of Dirk Muysers Sent: Monday, March 15, 2021 10:41 AM To: m3devel at elegosoft.com Subject: [M3devel] m3 code base The Klagenfurt (MS-DOS) M3 compiler was also C-based and didn`t make its team any happier. Maybe we should restart from PM3 and and avoid C and dependency on proprietary platform tools as much as possible. Dear Jay, your C++ and MS evangelism is nerve-racking. And, please, think with your brain instead of your mailer. Mail us the the good news after you found nice solutions that respect the integrity of the Modula-3 language. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 13:48:45 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 12:48:45 +0000 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, Message-ID: Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 13:54:33 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 12:54:33 +0000 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, , Message-ID: > You can likely reproduce this quite easily: er..well, not on non-Windows I guess, darn..stuff to fix. ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 12:48 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 13:56:23 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 12:56:23 +0000 Subject: [M3devel] Windows Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, , , Message-ID: you can but the scripts don't work: Even on Linux host: jay at deb1:/s/cm3/m3-ui/X11R4$ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm cd ../ui $ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm reproduces it. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:54 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? > You can likely reproduce this quite easily: er..well, not on non-Windows I guess, darn..stuff to fix. ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 12:48 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Mon Mar 15 15:06:44 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 15 Mar 2021 10:06:44 -0400 Subject: [M3devel] m3 code base In-Reply-To: References: Message-ID: <20210315140644.g67vbrhii6udijmt@topoi.pooq.com> On Mon, Mar 15, 2021 at 10:58:24AM +0000, Jay K wrote: > And, perhaps, licensing problem and getting into Debian. If licensing is *ever* going to he addressed we should be careful now to licence all new code inder something like the MIT licence, to keep the licensing problem from spreading into our new code at least. -- hendrik From jayk123 at hotmail.com Mon Mar 15 17:03:05 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 16:03:05 +0000 Subject: [M3devel] m3 code base In-Reply-To: <20210315140644.g67vbrhii6udijmt@topoi.pooq.com> References: , <20210315140644.g67vbrhii6udijmt@topoi.pooq.com> Message-ID: Right but problem to seems to be, I think, when a file is started, by copying an existing file. It's tenous though, because often the file ends up much different. - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Monday, March 15, 2021 2:06 PM To: m3devel at elegosoft.com Subject: Re: [M3devel] m3 code base On Mon, Mar 15, 2021 at 10:58:24AM +0000, Jay K wrote: > And, perhaps, licensing problem and getting into Debian. If licensing is *ever* going to he addressed we should be careful now to licence all new code inder something like the MIT licence, to keep the licensing problem from spreading into our new code at least. -- hendrik _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C146ecd2f44dd42a1693a08d8e7bb9b68%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637514140251688261%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gSZF%2BNvloAkOD1WrCELhdHsdJvRzMGzQ0H%2FGtu5eGJw%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 17:30:44 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 16:30:44 +0000 Subject: [M3devel] Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, , , , Message-ID: Related, on Linux? new source -> compiling GraphVBT.m3 "../src/GraphVBT.m3", line 991: Array constructor is not assignable to expected type. *** *** runtime error: *** <*ASSERT*> failed: FALSE *** file "../src/exprs/CallExpr.m3", line 191 *** - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:56 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? you can but the scripts don't work: Even on Linux host: jay at deb1:/s/cm3/m3-ui/X11R4$ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm cd ../ui $ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm reproduces it. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:54 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? > You can likely reproduce this quite easily: er..well, not on non-Windows I guess, darn..stuff to fix. ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 12:48 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 17:33:26 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 16:33:26 +0000 Subject: [M3devel] Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, , , , , Message-ID: == package /s/cm3/m3-ui/mgkit == NEW(LinearResize, graph := graph, shape := vertexHighlight.vertex.shape, corner0 := ARRAY [0 .. 1] OF R2.T{SWFromCenter(center0PP, size0), SWFromCenter(center1PP, size1)}, LinearResize = Animate.T BRANDED OBJECT (* READONLY after initialization: MUST be initialized by client *) (* CONST *) graph: T; shape: VertexShape; corner0, corner1: ARRAY [0 .. 1] (* time *) OF R2.T; (* coordinates, in points *) Looks like it should work? - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 4:30 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Trestle breakage? Related, on Linux? new source -> compiling GraphVBT.m3 "../src/GraphVBT.m3", line 991: Array constructor is not assignable to expected type. *** *** runtime error: *** <*ASSERT*> failed: FALSE *** file "../src/exprs/CallExpr.m3", line 191 *** - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:56 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? you can but the scripts don't work: Even on Linux host: jay at deb1:/s/cm3/m3-ui/X11R4$ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm cd ../ui $ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm reproduces it. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:54 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? > You can likely reproduce this quite easily: er..well, not on non-Windows I guess, darn..stuff to fix. ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 12:48 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 18:02:16 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 17:02:16 +0000 Subject: [M3devel] Trestle breakage? In-Reply-To: References: , <698741615801031@mail.yandex.by>, , , , , Message-ID: On the Windows break, on a recent older commit, it builds, the call IR is the same but I also see: declare_record typeid:T79A5AEBB bit_size:64 field_count:1 declare_record typeid:T23E73F69 bit_size:64 field_count:1 which I think head misses. I will look again.. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 4:30 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Trestle breakage? Related, on Linux? new source -> compiling GraphVBT.m3 "../src/GraphVBT.m3", line 991: Array constructor is not assignable to expected type. *** *** runtime error: *** <*ASSERT*> failed: FALSE *** file "../src/exprs/CallExpr.m3", line 191 *** - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:56 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? you can but the scripts don't work: Even on Linux host: jay at deb1:/s/cm3/m3-ui/X11R4$ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm cd ../ui $ cm3 -boot -boot -keep -DM3_BACKEND_MODE=C -DBUILD_DIR=AMD64_NTc -build -override -DROOT=/s/cm reproduces it. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 12:54 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? > You can likely reproduce this quite easily: er..well, not on non-Windows I guess, darn..stuff to fix. ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 12:48 PM To: vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] Windows Trestle breakage? Error looks to be here: PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); <== j.trueChild := ch; EVAL j.init(child) END InitChild; . . . #line 41 "..\src\trestle\InstalledVBT.m3" /* start_call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably grandChild #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param presumably ch #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T79A5AEBB byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T79A5AEBB* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_struct typeid:T23E73F69 byte_size:8 */ pop #line 41 "..\src\trestle\InstalledVBT.m3" /* ERROR:pop_struct: unknown typeid:T23E73F69* */ #line 41 "..\src\trestle\InstalledVBT.m3" /* load_address var:L_45_L_46 offset:72 */ #line 41 "..\src\trestle\InstalledVBT.m3" /* pop_param */ pop_param #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /* check_nil */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_79_L_80))=(ADDRESS)(((ADDRESS)(L_77_L_78))); #line 41 "..\src\trestle\InstalledVBT.m3" /* load */ #line 41 "..\src\trestle\InstalledVBT.m3" /*check_nil*/if(!L_79_L_80)M_InstalledVBT_L_47_CRASH(1316); #line 41 "..\src\trestle\InstalledVBT.m3" /* call_indirect */ #line 41 "..\src\trestle\InstalledVBT.m3" /* store */ #line 41 "..\src\trestle\InstalledVBT.m3" (*(ADDRESS*)(&L_75_L_76))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_79_L_80)( ((ADDRESS)(grandChild_L_55)), ((ADDRESS)(ch_L_58)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(984)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1008)+((ADDRESS)(&M_InstalledVBT_L_47)))))))))), <== ((ADDRESS)(INT64_(72)+((ADDRESS)(&L_45_L_46)))))))); Public = Filter.T OBJECT METHODS <* LL.sup <= VBT.mu *> init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T So the PaintOp.T and Pixmap.T params. PaintOp: TYPE T = RECORD op:INTEGER END; Pixmap same: TYPE T = RECORD pm: INTEGER END; Rodney ring a bell? Records that just contain an integer? Did you maybe convert small structs from scalars to structs but don't declare them? I can try building w/o your changes and see what is the IR, or look otherwise, etc. Maybe I broke it. I have not been building everything often enough, just the compiler. :( The other backends are not so sensitive to having types declared, but I wanted the C backend prepared to strongly type stuff so debugging is good. You can likely reproduce this quite easily: cd scripts/python ./do-cm3-all.py boot c amd64_nt keep yeah kinda of annoying and then incremental: ./do-pkg.py ui boot c amd64_nt keep build (boot ought to imply build but doesn't always, sorry; and c should be implied for targets w/o gcc backend) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 9:48 AM To: vvm at tut.by ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Thank you for the data point. Maybe I'll double check that. Maybe there is another Win64 header rewrite problem? But things are worse now. I'll get to it eventually/soon hopefully.. - Jay ________________________________ From: vvm at tut.by Sent: Monday, March 15, 2021 9:42 AM To: Jay K ; m3devel Subject: Re: [M3devel] Windows Trestle breakage? Hi! ( It's not about exactly this issue, but about M3 GUI in Windows environment ) cm3-all-x86-d5.11.0-VC90-20210312\bin\solitaire.exe has good size of window on start .exe cm3-all-AMD64_NT-d5.11.0-VC2019-20210312\bin\solitaire.exe start with good size of window and minimize it in next steps imm. after start .exe I think it's strange. Best regards, Victor Miasnikov 15.03.2021, 11:53, "Jay K" : This used to work. I haven't dug in. Presumably me or the packed vars caused it.. == package C:\s\cm3\m3-ui\ui == +++ C:\cm3\bin\cm3.exe -DM3_BACKEND_MODE=C -build -DROOT=C:/s/cm3 +++ --- building in AMD64_NT --- ignoring ..\src\m3overrides new source -> compiling VBT.i3 new source -> compiling VBT.m3 new source -> compiling InstalledVBT.m3 "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T79A5AEBB* "..\src\trestle\InstalledVBT.m3", line 18: pop_struct: unknown typeid:T23E73F69* 2 errors encountered - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 19:45:55 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 18:45:55 +0000 Subject: [M3devel] packedVars broke Win32 record layout Message-ID: Rodney, Currently, with the packedVars work, cm3 on Win64 doesn't get far. It tries to allocate too much memory. Here: 0:000> k # Child-SP RetAddr Call Site 00 (Inline Function) --------`-------- KERNELBASE!DebugBreak [minkernel\kernelbase\debug.c @ 143] 01 000000cf`fe8fe7b8 00007ff7`41a25c73 KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] 02 000000cf`fe8fe7c0 00007ff7`419dee5f cm3!RTOS__Crash+0x43 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 26] 03 000000cf`fe8fe810 00007ff7`41a0d64f cm3!RTProcess__Crash+0x6f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66] 04 000000cf`fe8fe850 00007ff7`41a0d280 cm3!RTError__EndError+0x3f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119] 05 000000cf`fe8fe890 00007ff7`41a25b57 cm3!RTError__MsgS+0xa0 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41] 06 000000cf`fe8fe8d0 00007ff7`41a2575e cm3!RTException__Crash+0x317 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81] 07 000000cf`fe8fe960 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0xce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39] 08 000000cf`fe8fe9d0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 09 000000cf`fe8fea10 00007ff7`41a2582a cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0a 000000cf`fe8fea70 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0x19a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49] 0b 000000cf`fe8feae0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 0c 000000cf`fe8feb20 00007ff7`419f7a33 cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0d 000000cf`fe8feb80 00007ff7`41a2726a cm3!RTHooks__Raise+0xb3 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80] 0e 000000cf`fe8fec10 00007ff7`419df419 cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63] 0f 000000cf`fe8fec60 00007ff7`416eef46 cm3!RTHooks__AllocateOpenArray+0x59 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 146] 10 000000cf`fe8fecb0 00007ff7`416e6820 cm3!QScanner__Init+0x206 [C:\s\cm3\m3-sys\m3quake\src\QScanner.m3 @ 48] 11 000000cf`fe8fef00 00007ff7`416e03bd cm3!QCompiler__CompileFile+0x290 [C:\s\cm3\m3-sys\m3quake\src\QCompiler.m3 @ 47] 12 000000cf`fe8ff290 00007ff7`415f5d3a cm3!Quake__Run+0x9d [C:\s\cm3\m3-sys\m3quake\src\Quake.m3 @ 19] 13 000000cf`fe8ff2f0 00007ff7`415f5552 cm3!Main__DoIt+0x75a [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 96] 14 000000cf`fe8ff6a0 00007ff7`419dda77 cm3!Main_M3+0x22 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 233] 15 000000cf`fe8ff6e0 00007ff7`419ddcfb cm3!RTLinker__RunMainBody+0x3e7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 408] 16 000000cf`fe8ff780 00007ff7`419dd478 cm3!RTLinker__AddUnitI+0x11b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 116] 17 000000cf`fe8ff7c0 00007ff7`415b1038 cm3!RTLinker__AddUnit+0x118 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 125] 18 000000cf`fe8ff820 00007ff7`41a47554 cm3!main+0x38 [C:\s\cm3\m3-sys\cm3\AMD64_NTc\_m3main.c @ 23] 19 (Inline Function) --------`-------- cm3!invoke_main+0x22 [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 1a 000000cf`fe8ff850 00007ffb`74c17034 cm3!__scrt_common_main_seh+0x10c [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 1b 000000cf`fe8ff890 00007ffb`74fc2651 KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] 1c 000000cf`fe8ff8c0 00000000`00000000 ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. Perhaps I should. The cloning really is still a bad idea. ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** But they no longer do. Here is the mismatch: cm3!FileWin32__FileStatus+0x2f4: 00007ff7`419c4194 e847350800 call cm3!__security_check_cookie (00007ff7`41a476e0) 0:000> dv _result_L_103 = 0x000000cf`fe8fee30 h_L_104 = 0x000001f3`fb2b9fc8 "p" L_166_L_167 = 294960936511.49194336 L_168_L_169 = 0n889058230273 L_162_L_163 = 0n1 L_160_L_161 = 1 status_L_101 = struct T53320305 ffd_L_100 = struct TCCCF928E <== This. ft_L_102 = 1 _frame = struct FileWin32__FileStatus_Frame_t L_164_L_165 = 0x000001f3`fb2f0228 "???" 0:000> dx -r1 (*((cm3!TCCCF928E *)0xcffe8fec48)) (*((cm3!TCCCF928E *)0xcffe8fec48)) [Type: TCCCF928E] [+0x000] dwFileAttributes : 0x20 [Type: unsigned int] [+0x004] L_9 [Type: unsigned char [4]] <== Modula-3 inserted padded for alignment here [+0x008] ftCreationTime [Type: T7AF5436C] [+0x010] ftLastAccessTime [Type: T7AF5436C] [+0x018] ftLastWriteTime [Type: T7AF5436C] [+0x020] dwVolumeSerialNumber : 0x0 [Type: unsigned int] [+0x024] nFileSizeHigh : 0xcf [Type: unsigned int] [+0x028] nFileSizeLow : 0x1 [Type: unsigned int] [+0x02c] nNumberOfLinks : 0x190000 [Type: unsigned int] [+0x030] nFileIndexHigh : 0x30a228 [Type: unsigned int] [+0x034] nFileIndexLow : 0x0 [Type: unsigned int] 0:000> dt BY_HANDLE_FILE_INFORMATION ntdll!BY_HANDLE_FILE_INFORMATION +0x000 dwFileAttributes : Uint4B +0x004 ftCreationTime : _FILETIME <== So the offset is wrong here and the rest. +0x00c ftLastAccessTime : _FILETIME +0x014 ftLastWriteTime : _FILETIME +0x01c dwVolumeSerialNumber : Uint4B +0x020 nFileSizeHigh : Uint4B +0x024 nFileSizeLow : Uint4B +0x028 nNumberOfLinks : Uint4B +0x02c nFileIndexHigh : Uint4B +0x030 nFileIndexLow : Uint4B So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. i.e. widening everything to LONGINT or LONGREAL to avoid padding. But, arguably I should not have to. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 20:01:19 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 19:01:19 +0000 Subject: [M3devel] packedVars broke Win32 record layout In-Reply-To: References: Message-ID: Maybe the Win32 header cloning was always wrong? I will try putting BITS n FOR all over and see if that helps. ? - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 6:45 PM To: rodney.m.bates at acm.org ; m3devel Subject: packedVars broke Win32 record layout Rodney, Currently, with the packedVars work, cm3 on Win64 doesn't get far. It tries to allocate too much memory. Here: 0:000> k # Child-SP RetAddr Call Site 00 (Inline Function) --------`-------- KERNELBASE!DebugBreak [minkernel\kernelbase\debug.c @ 143] 01 000000cf`fe8fe7b8 00007ff7`41a25c73 KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] 02 000000cf`fe8fe7c0 00007ff7`419dee5f cm3!RTOS__Crash+0x43 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 26] 03 000000cf`fe8fe810 00007ff7`41a0d64f cm3!RTProcess__Crash+0x6f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66] 04 000000cf`fe8fe850 00007ff7`41a0d280 cm3!RTError__EndError+0x3f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119] 05 000000cf`fe8fe890 00007ff7`41a25b57 cm3!RTError__MsgS+0xa0 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41] 06 000000cf`fe8fe8d0 00007ff7`41a2575e cm3!RTException__Crash+0x317 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81] 07 000000cf`fe8fe960 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0xce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39] 08 000000cf`fe8fe9d0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 09 000000cf`fe8fea10 00007ff7`41a2582a cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0a 000000cf`fe8fea70 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0x19a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49] 0b 000000cf`fe8feae0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 0c 000000cf`fe8feb20 00007ff7`419f7a33 cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0d 000000cf`fe8feb80 00007ff7`41a2726a cm3!RTHooks__Raise+0xb3 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80] 0e 000000cf`fe8fec10 00007ff7`419df419 cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63] 0f 000000cf`fe8fec60 00007ff7`416eef46 cm3!RTHooks__AllocateOpenArray+0x59 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 146] 10 000000cf`fe8fecb0 00007ff7`416e6820 cm3!QScanner__Init+0x206 [C:\s\cm3\m3-sys\m3quake\src\QScanner.m3 @ 48] 11 000000cf`fe8fef00 00007ff7`416e03bd cm3!QCompiler__CompileFile+0x290 [C:\s\cm3\m3-sys\m3quake\src\QCompiler.m3 @ 47] 12 000000cf`fe8ff290 00007ff7`415f5d3a cm3!Quake__Run+0x9d [C:\s\cm3\m3-sys\m3quake\src\Quake.m3 @ 19] 13 000000cf`fe8ff2f0 00007ff7`415f5552 cm3!Main__DoIt+0x75a [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 96] 14 000000cf`fe8ff6a0 00007ff7`419dda77 cm3!Main_M3+0x22 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 233] 15 000000cf`fe8ff6e0 00007ff7`419ddcfb cm3!RTLinker__RunMainBody+0x3e7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 408] 16 000000cf`fe8ff780 00007ff7`419dd478 cm3!RTLinker__AddUnitI+0x11b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 116] 17 000000cf`fe8ff7c0 00007ff7`415b1038 cm3!RTLinker__AddUnit+0x118 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 125] 18 000000cf`fe8ff820 00007ff7`41a47554 cm3!main+0x38 [C:\s\cm3\m3-sys\cm3\AMD64_NTc\_m3main.c @ 23] 19 (Inline Function) --------`-------- cm3!invoke_main+0x22 [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 1a 000000cf`fe8ff850 00007ffb`74c17034 cm3!__scrt_common_main_seh+0x10c [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 1b 000000cf`fe8ff890 00007ffb`74fc2651 KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] 1c 000000cf`fe8ff8c0 00000000`00000000 ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. Perhaps I should. The cloning really is still a bad idea. ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** But they no longer do. Here is the mismatch: cm3!FileWin32__FileStatus+0x2f4: 00007ff7`419c4194 e847350800 call cm3!__security_check_cookie (00007ff7`41a476e0) 0:000> dv _result_L_103 = 0x000000cf`fe8fee30 h_L_104 = 0x000001f3`fb2b9fc8 "p" L_166_L_167 = 294960936511.49194336 L_168_L_169 = 0n889058230273 L_162_L_163 = 0n1 L_160_L_161 = 1 status_L_101 = struct T53320305 ffd_L_100 = struct TCCCF928E <== This. ft_L_102 = 1 _frame = struct FileWin32__FileStatus_Frame_t L_164_L_165 = 0x000001f3`fb2f0228 "???" 0:000> dx -r1 (*((cm3!TCCCF928E *)0xcffe8fec48)) (*((cm3!TCCCF928E *)0xcffe8fec48)) [Type: TCCCF928E] [+0x000] dwFileAttributes : 0x20 [Type: unsigned int] [+0x004] L_9 [Type: unsigned char [4]] <== Modula-3 inserted padded for alignment here [+0x008] ftCreationTime [Type: T7AF5436C] [+0x010] ftLastAccessTime [Type: T7AF5436C] [+0x018] ftLastWriteTime [Type: T7AF5436C] [+0x020] dwVolumeSerialNumber : 0x0 [Type: unsigned int] [+0x024] nFileSizeHigh : 0xcf [Type: unsigned int] [+0x028] nFileSizeLow : 0x1 [Type: unsigned int] [+0x02c] nNumberOfLinks : 0x190000 [Type: unsigned int] [+0x030] nFileIndexHigh : 0x30a228 [Type: unsigned int] [+0x034] nFileIndexLow : 0x0 [Type: unsigned int] 0:000> dt BY_HANDLE_FILE_INFORMATION ntdll!BY_HANDLE_FILE_INFORMATION +0x000 dwFileAttributes : Uint4B +0x004 ftCreationTime : _FILETIME <== So the offset is wrong here and the rest. +0x00c ftLastAccessTime : _FILETIME +0x014 ftLastWriteTime : _FILETIME +0x01c dwVolumeSerialNumber : Uint4B +0x020 nFileSizeHigh : Uint4B +0x024 nFileSizeLow : Uint4B +0x028 nNumberOfLinks : Uint4B +0x02c nFileIndexHigh : Uint4B +0x030 nFileIndexLow : Uint4B So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. i.e. widening everything to LONGINT or LONGREAL to avoid padding. But, arguably I should not have to. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 20:51:19 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 19:51:19 +0000 Subject: [M3devel] packedVars broke Win32 record layout In-Reply-To: References: , Message-ID: Ok this seems to fix. I will test a bit more. diff --git a/m3-libs/m3core/src/win32/WinBase.i3 b/m3-libs/m3core/src/win32/WinBase.i3 index a3e471f23..0afb27de8 100644 --- a/m3-libs/m3core/src/win32/WinBase.i3 +++ b/m3-libs/m3core/src/win32/WinBase.i3 @@ -178,8 +178,8 @@ TYPE PFILETIME = UNTRACED REF FILETIME; LPFILETIME = PFILETIME; (* compat *) FILETIME = RECORD - dwLowDateTime : UINT32; - dwHighDateTime: UINT32; + dwLowDateTime : BITS BITSIZE(UINT32) FOR UINT32; + dwHighDateTime: BITS BITSIZE(UINT32) FOR UINT32; END; (* System time is represented with the following structure: *) @@ -1585,16 +1585,16 @@ TYPE PBY_HANDLE_FILE_INFORMATION = UNTRACED REF BY_HANDLE_FILE_INFORMATION; LPBY_HANDLE_FILE_INFORMATION = UNTRACED REF BY_HANDLE_FILE_INFORMATION; BY_HANDLE_FILE_INFORMATION = RECORD - dwFileAttributes : UINT32; - ftCreationTime : FILETIME; - ftLastAccessTime : FILETIME; - ftLastWriteTime : FILETIME; - dwVolumeSerialNumber: UINT32; - nFileSizeHigh : UINT32; - nFileSizeLow : UINT32; - nNumberOfLinks : UINT32; - nFileIndexHigh : UINT32; - nFileIndexLow : UINT32; + dwFileAttributes : BITS BITSIZE(UINT32) FOR UINT32; + ftCreationTime : BITS 64 FOR FILETIME; + ftLastAccessTime : BITS 64 FOR FILETIME; + ftLastWriteTime : BITS 64 FOR FILETIME; + dwVolumeSerialNumber: BITS BITSIZE(UINT32) FOR UINT32; + nFileSizeHigh : BITS BITSIZE(UINT32) FOR UINT32; + nFileSizeLow : BITS BITSIZE(UINT32) FOR UINT32; + nNumberOfLinks : BITS BITSIZE(UINT32) FOR UINT32; + nFileIndexHigh : BITS BITSIZE(UINT32) FOR UINT32; + nFileIndexLow : BITS BITSIZE(UINT32) FOR UINT32; END; <*EXTERNAL GetFileInformationByHandle:WINAPI*> @@ -2280,14 +2280,14 @@ TYPE PWIN32_FIND_DATAA = UNTRACED REF WIN32_FIND_DATAA; LPWIN32_FIND_DATAA = UNTRACED REF WIN32_FIND_DATAA; WIN32_FIND_DATAA = RECORD - dwFileAttributes : UINT32; - ftCreationTime : FILETIME; - ftLastAccessTime : FILETIME; - ftLastWriteTime : FILETIME; - nFileSizeHigh : UINT32; - nFileSizeLow : UINT32; - dwReserved0 : UINT32; - dwReserved1 : UINT32; + dwFileAttributes : BITS BITSIZE(UINT32) FOR UINT32; + ftCreationTime : BITS 64 FOR FILETIME; + ftLastAccessTime : BITS 64 FOR FILETIME; + ftLastWriteTime : BITS 64 FOR FILETIME; + nFileSizeHigh : BITS BITSIZE(UINT32) FOR UINT32; + nFileSizeLow : BITS BITSIZE(UINT32) FOR UINT32; + dwReserved0 : BITS BITSIZE(UINT32) FOR UINT32; + dwReserved1 : BITS BITSIZE(UINT32) FOR UINT32; cFileName : ARRAY [0 .. MAX_PATH - 1] OF CHAR; cAlternateFileName: ARRAY [0 .. 14 - 1] OF CHAR; END; @@ -2295,14 +2295,14 @@ TYPE PWIN32_FIND_DATAW = UNTRACED REF WIN32_FIND_DATAW; LPWIN32_FIND_DATAW = UNTRACED REF WIN32_FIND_DATAW; WIN32_FIND_DATAW = RECORD - dwFileAttributes : UINT32; - ftCreationTime : FILETIME; - ftLastAccessTime : FILETIME; - ftLastWriteTime : FILETIME; - nFileSizeHigh : UINT32; - nFileSizeLow : UINT32; - dwReserved0 : UINT32; - dwReserved1 : UINT32; + dwFileAttributes : BITS BITSIZE(UINT32) FOR UINT32; + ftCreationTime : BITS 64 FOR FILETIME; + ftLastAccessTime : BITS 64 FOR FILETIME; + ftLastWriteTime : BITS 64 FOR FILETIME; + nFileSizeHigh : BITS BITSIZE(UINT32) FOR UINT32; + nFileSizeLow : BITS BITSIZE(UINT32) FOR UINT32; + dwReserved0 : BITS BITSIZE(UINT32) FOR UINT32; + dwReserved1 : BITS BITSIZE(UINT32) FOR UINT32; cFileName : ARRAY [0 .. MAX_PATH - 1] OF WCHAR; cAlternateFileName: ARRAY [0 .. 14 - 1] OF WCHAR; END; I'm still a bit unnerved about this though. I was going to apply BITS for more broadly, but as I understand, it is not always needed, even this might be overkill, and I don't really want to vouch for the most of the content in these files. Header cloning and layout rules changing, not my favorites. Maybe fix, and then remove all users in-tree.. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 7:01 PM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] packedVars broke Win32 record layout Maybe the Win32 header cloning was always wrong? I will try putting BITS n FOR all over and see if that helps. ? - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 6:45 PM To: rodney.m.bates at acm.org ; m3devel Subject: packedVars broke Win32 record layout Rodney, Currently, with the packedVars work, cm3 on Win64 doesn't get far. It tries to allocate too much memory. Here: 0:000> k # Child-SP RetAddr Call Site 00 (Inline Function) --------`-------- KERNELBASE!DebugBreak [minkernel\kernelbase\debug.c @ 143] 01 000000cf`fe8fe7b8 00007ff7`41a25c73 KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] 02 000000cf`fe8fe7c0 00007ff7`419dee5f cm3!RTOS__Crash+0x43 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 26] 03 000000cf`fe8fe810 00007ff7`41a0d64f cm3!RTProcess__Crash+0x6f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66] 04 000000cf`fe8fe850 00007ff7`41a0d280 cm3!RTError__EndError+0x3f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119] 05 000000cf`fe8fe890 00007ff7`41a25b57 cm3!RTError__MsgS+0xa0 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41] 06 000000cf`fe8fe8d0 00007ff7`41a2575e cm3!RTException__Crash+0x317 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81] 07 000000cf`fe8fe960 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0xce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39] 08 000000cf`fe8fe9d0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 09 000000cf`fe8fea10 00007ff7`41a2582a cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0a 000000cf`fe8fea70 00007ff7`41a2564b cm3!RTException__DefaultBackstop+0x19a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49] 0b 000000cf`fe8feae0 00007ff7`41a25f1a cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] 0c 000000cf`fe8feb20 00007ff7`419f7a33 cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] 0d 000000cf`fe8feb80 00007ff7`41a2726a cm3!RTHooks__Raise+0xb3 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80] 0e 000000cf`fe8fec10 00007ff7`419df419 cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63] 0f 000000cf`fe8fec60 00007ff7`416eef46 cm3!RTHooks__AllocateOpenArray+0x59 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 146] 10 000000cf`fe8fecb0 00007ff7`416e6820 cm3!QScanner__Init+0x206 [C:\s\cm3\m3-sys\m3quake\src\QScanner.m3 @ 48] 11 000000cf`fe8fef00 00007ff7`416e03bd cm3!QCompiler__CompileFile+0x290 [C:\s\cm3\m3-sys\m3quake\src\QCompiler.m3 @ 47] 12 000000cf`fe8ff290 00007ff7`415f5d3a cm3!Quake__Run+0x9d [C:\s\cm3\m3-sys\m3quake\src\Quake.m3 @ 19] 13 000000cf`fe8ff2f0 00007ff7`415f5552 cm3!Main__DoIt+0x75a [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 96] 14 000000cf`fe8ff6a0 00007ff7`419dda77 cm3!Main_M3+0x22 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 233] 15 000000cf`fe8ff6e0 00007ff7`419ddcfb cm3!RTLinker__RunMainBody+0x3e7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 408] 16 000000cf`fe8ff780 00007ff7`419dd478 cm3!RTLinker__AddUnitI+0x11b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 116] 17 000000cf`fe8ff7c0 00007ff7`415b1038 cm3!RTLinker__AddUnit+0x118 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 125] 18 000000cf`fe8ff820 00007ff7`41a47554 cm3!main+0x38 [C:\s\cm3\m3-sys\cm3\AMD64_NTc\_m3main.c @ 23] 19 (Inline Function) --------`-------- cm3!invoke_main+0x22 [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 1a 000000cf`fe8ff850 00007ffb`74c17034 cm3!__scrt_common_main_seh+0x10c [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 1b 000000cf`fe8ff890 00007ffb`74fc2651 KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] 1c 000000cf`fe8ff8c0 00000000`00000000 ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. Perhaps I should. The cloning really is still a bad idea. ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** But they no longer do. Here is the mismatch: cm3!FileWin32__FileStatus+0x2f4: 00007ff7`419c4194 e847350800 call cm3!__security_check_cookie (00007ff7`41a476e0) 0:000> dv _result_L_103 = 0x000000cf`fe8fee30 h_L_104 = 0x000001f3`fb2b9fc8 "p" L_166_L_167 = 294960936511.49194336 L_168_L_169 = 0n889058230273 L_162_L_163 = 0n1 L_160_L_161 = 1 status_L_101 = struct T53320305 ffd_L_100 = struct TCCCF928E <== This. ft_L_102 = 1 _frame = struct FileWin32__FileStatus_Frame_t L_164_L_165 = 0x000001f3`fb2f0228 "???" 0:000> dx -r1 (*((cm3!TCCCF928E *)0xcffe8fec48)) (*((cm3!TCCCF928E *)0xcffe8fec48)) [Type: TCCCF928E] [+0x000] dwFileAttributes : 0x20 [Type: unsigned int] [+0x004] L_9 [Type: unsigned char [4]] <== Modula-3 inserted padded for alignment here [+0x008] ftCreationTime [Type: T7AF5436C] [+0x010] ftLastAccessTime [Type: T7AF5436C] [+0x018] ftLastWriteTime [Type: T7AF5436C] [+0x020] dwVolumeSerialNumber : 0x0 [Type: unsigned int] [+0x024] nFileSizeHigh : 0xcf [Type: unsigned int] [+0x028] nFileSizeLow : 0x1 [Type: unsigned int] [+0x02c] nNumberOfLinks : 0x190000 [Type: unsigned int] [+0x030] nFileIndexHigh : 0x30a228 [Type: unsigned int] [+0x034] nFileIndexLow : 0x0 [Type: unsigned int] 0:000> dt BY_HANDLE_FILE_INFORMATION ntdll!BY_HANDLE_FILE_INFORMATION +0x000 dwFileAttributes : Uint4B +0x004 ftCreationTime : _FILETIME <== So the offset is wrong here and the rest. +0x00c ftLastAccessTime : _FILETIME +0x014 ftLastWriteTime : _FILETIME +0x01c dwVolumeSerialNumber : Uint4B +0x020 nFileSizeHigh : Uint4B +0x024 nFileSizeLow : Uint4B +0x028 nNumberOfLinks : Uint4B +0x02c nFileIndexHigh : Uint4B +0x030 nFileIndexLow : Uint4B So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. i.e. widening everything to LONGINT or LONGREAL to avoid padding. But, arguably I should not have to. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Mar 15 20:57:16 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 15 Mar 2021 14:57:16 -0500 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms In-Reply-To: References: Message-ID: On 3/14/21 6:24 AM, Jay K wrote: > This: > > PROCEDURE Init32 () = > ? BEGIN > ? ? (* Change size and aligment: *) > ? ? Integer := Int32; > ? ? Word := Word32; > ? ? Address := Word32; > ? ? Address.cg_type := CGType.Addr; > ? ? (* Change only alignment. ?Size is always 64: *) > *? ? Longint.align := 32; ? ?<==* > *? ? Long.align := 32; ? ? ? <==* > *? ? Longreal.align := 32; ? <==* > *? ? Extended.align := 32; <==* > ? END Init32; > > is not great. Perhaps terrible. > > Because x86 has InterlockedCompareExchange64, that wants 64 alignment. > > The Longreal part is also a bad idea. > > This is an old way of achieving 64bit alignment, before the advent of 64bit integers: > > union LARGE_INTEGER > { > ? struct { int HighPart, LowPart }; > ? double Alignment; > }; > > And it should continue to work. > > The 32bit and 64bit rules should be more the same. > > I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. > If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. > But it is likely difficult/impossible. The interlocked operations could be over in C. We had a discussion a while back about 64-bit alignment of 64-bit scalars on 32-bit machines. If I recall correctly, somebody (Jay?) advocated for doing it. I remember realizing later that this would be just about untenable. As I recall, some of the reasons were that all allocated blocks would have to be 64-bit aligned to begin with. This would include things OS and library-provided things like malloc, object module sections, loader, etc. Also the GC (every object) and even runtime adjustment of activation records on calls on machines that do runtime operand stack pushes. > > Also this maybe disagrees with C alignment. Let's see: > > C:\s\cm3\scripts\python>type 1.c > struct { > __int64 a; > int b; > } a; > > int b = sizeof(a); > > #pragma pack(1) > > struct { > __int64 a; > int b; > } c; > > int d = sizeof(c); > > C:\s\cm3\scripts\python>cl /c /FAsc 1.c > Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 > Copyright (C) Microsoft Corporation. ?All rights reserved. > > 1.c > > C:\s\cm3\scripts\python>more 1.cod > ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 > > COMM ? ?_a:BYTE:010H > _b ? ? ?DD ? ? ?010H > _d ? ? ?DD ? ? ?0cH > > So this seems like it /could really /break things. > i.e. struct stat in m3core. > Except that I sorted things by size so it'll tend to work out. > > But still, really, alignment of 64bit integers should be 64, on all platforms. > > ? - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Mon Mar 15 21:09:31 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 20:09:31 +0000 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms In-Reply-To: References: , Message-ID: Heap alignment on Win32 is 2 pointers, so that part is covered. Object sections are generally at least page-aligned. That's why sections exist, to have different page protections. So that is covered too. In old obscure cases they would be 1-aligned, like to optimize the size of drivers and make them read/write/execute, back before security mattered and read-only and execute-only was worth a little size. But stack, hm. This prints 8 for alignment but indeed sometimes the assert fails. #include #include int main() { __int64 a; printf("%d %p\n", __alignof(a), &a); assert (((size_t)&a % __alignof(a)) == 0); } Interlocked operations on stack are less common though, because stack tends not to be shared across threads. Good enough?? I agree it seems wierd. I'm sure Visual C++ does sometimes adjust stack alignment. Presumably for double/sse2 stuff, not mere int64. It is also profitable to have more similar rules across platforms. So pickles and similar don't need conversion. Granted, it might not be possible to eliminate all conversion. And also, again, to match the C compiler. #include #include int main() { struct { __int64 a; int b; } c; struct { int d[3]; } e; printf("%d %d\n", (int)sizeof(c), (int)sizeof(e)); } is 16 and 12, but presumably both 16 in analogous 32bit Modula-3. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, March 15, 2021 7:57 PM To: Jay K ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms On 3/14/21 6:24 AM, Jay K wrote: > This: > > PROCEDURE Init32 () = > BEGIN > (* Change size and aligment: *) > Integer := Int32; > Word := Word32; > Address := Word32; > Address.cg_type := CGType.Addr; > (* Change only alignment. Size is always 64: *) > * Longint.align := 32; <==* > * Long.align := 32; <==* > * Longreal.align := 32; <==* > * Extended.align := 32; <==* > END Init32; > > is not great. Perhaps terrible. > > Because x86 has InterlockedCompareExchange64, that wants 64 alignment. > > The Longreal part is also a bad idea. > > This is an old way of achieving 64bit alignment, before the advent of 64bit integers: > > union LARGE_INTEGER > { > struct { int HighPart, LowPart }; > double Alignment; > }; > > And it should continue to work. > > The 32bit and 64bit rules should be more the same. > > I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. > If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. > But it is likely difficult/impossible. The interlocked operations could be over in C. We had a discussion a while back about 64-bit alignment of 64-bit scalars on 32-bit machines. If I recall correctly, somebody (Jay?) advocated for doing it. I remember realizing later that this would be just about untenable. As I recall, some of the reasons were that all allocated blocks would have to be 64-bit aligned to begin with. This would include things OS and library-provided things like malloc, object module sections, loader, etc. Also the GC (every object) and even runtime adjustment of activation records on calls on machines that do runtime operand stack pushes. > > Also this maybe disagrees with C alignment. Let's see: > > C:\s\cm3\scripts\python>type 1.c > struct { > __int64 a; > int b; > } a; > > int b = sizeof(a); > > #pragma pack(1) > > struct { > __int64 a; > int b; > } c; > > int d = sizeof(c); > > C:\s\cm3\scripts\python>cl /c /FAsc 1.c > Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 > Copyright (C) Microsoft Corporation. All rights reserved. > > 1.c > > C:\s\cm3\scripts\python>more 1.cod > ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 > > COMM _a:BYTE:010H > _b DD 010H > _d DD 0cH > > So this seems like it /could really /break things. > i.e. struct stat in m3core. > Except that I sorted things by size so it'll tend to work out. > > But still, really, alignment of 64bit integers should be 64, on all platforms. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4986a2de17d74387f46308d8e7ec9309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637514350553019597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qEe3m6pZ%2BZHayaHaaZAgA5EKWzTsUWqj2K13P6Ui0LA%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Mon Mar 15 21:15:20 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 20:15:20 +0000 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms In-Reply-To: References: , , Message-ID: Here, Visual C++ does do the dynamic stack alignment. No big deal? #include #include #include int main() { __m64 a; printf("%d %p\n", (int)__alignof(a), &a); } _main: 00000000: 53 push ebx 00000001: 8B DC mov ebx,esp 00000003: 83 EC 08 sub esp,8 00000006: 83 E4 F8 and esp,0FFFFFFF8h <=== 00000009: 83 C4 04 add esp,4 0000000C: 55 push ebp 0000000D: 8B 6B 04 mov ebp,dword ptr [ebx+4] 00000010: 89 6C 24 04 mov dword ptr [esp+4],ebp 00000014: 8B EC mov ebp,esp 00000016: 83 EC 08 sub esp,8 00000019: 8D 45 F8 lea eax,[ebp-8] 0000001C: 50 push eax 0000001D: 6A 08 push 8 0000001F: 68 00 00 00 00 push offset $SG27991 00000024: E8 00 00 00 00 call _printf - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 8:09 PM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms Heap alignment on Win32 is 2 pointers, so that part is covered. Object sections are generally at least page-aligned. That's why sections exist, to have different page protections. So that is covered too. In old obscure cases they would be 1-aligned, like to optimize the size of drivers and make them read/write/execute, back before security mattered and read-only and execute-only was worth a little size. But stack, hm. This prints 8 for alignment but indeed sometimes the assert fails. #include #include int main() { __int64 a; printf("%d %p\n", __alignof(a), &a); assert (((size_t)&a % __alignof(a)) == 0); } Interlocked operations on stack are less common though, because stack tends not to be shared across threads. Good enough?? I agree it seems wierd. I'm sure Visual C++ does sometimes adjust stack alignment. Presumably for double/sse2 stuff, not mere int64. It is also profitable to have more similar rules across platforms. So pickles and similar don't need conversion. Granted, it might not be possible to eliminate all conversion. And also, again, to match the C compiler. #include #include int main() { struct { __int64 a; int b; } c; struct { int d[3]; } e; printf("%d %d\n", (int)sizeof(c), (int)sizeof(e)); } is 16 and 12, but presumably both 16 in analogous 32bit Modula-3. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, March 15, 2021 7:57 PM To: Jay K ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms On 3/14/21 6:24 AM, Jay K wrote: > This: > > PROCEDURE Init32 () = > BEGIN > (* Change size and aligment: *) > Integer := Int32; > Word := Word32; > Address := Word32; > Address.cg_type := CGType.Addr; > (* Change only alignment. Size is always 64: *) > * Longint.align := 32; <==* > * Long.align := 32; <==* > * Longreal.align := 32; <==* > * Extended.align := 32; <==* > END Init32; > > is not great. Perhaps terrible. > > Because x86 has InterlockedCompareExchange64, that wants 64 alignment. > > The Longreal part is also a bad idea. > > This is an old way of achieving 64bit alignment, before the advent of 64bit integers: > > union LARGE_INTEGER > { > struct { int HighPart, LowPart }; > double Alignment; > }; > > And it should continue to work. > > The 32bit and 64bit rules should be more the same. > > I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. > If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. > But it is likely difficult/impossible. The interlocked operations could be over in C. We had a discussion a while back about 64-bit alignment of 64-bit scalars on 32-bit machines. If I recall correctly, somebody (Jay?) advocated for doing it. I remember realizing later that this would be just about untenable. As I recall, some of the reasons were that all allocated blocks would have to be 64-bit aligned to begin with. This would include things OS and library-provided things like malloc, object module sections, loader, etc. Also the GC (every object) and even runtime adjustment of activation records on calls on machines that do runtime operand stack pushes. > > Also this maybe disagrees with C alignment. Let's see: > > C:\s\cm3\scripts\python>type 1.c > struct { > __int64 a; > int b; > } a; > > int b = sizeof(a); > > #pragma pack(1) > > struct { > __int64 a; > int b; > } c; > > int d = sizeof(c); > > C:\s\cm3\scripts\python>cl /c /FAsc 1.c > Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 > Copyright (C) Microsoft Corporation. All rights reserved. > > 1.c > > C:\s\cm3\scripts\python>more 1.cod > ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 > > COMM _a:BYTE:010H > _b DD 010H > _d DD 0cH > > So this seems like it /could really /break things. > i.e. struct stat in m3core. > Except that I sorted things by size so it'll tend to work out. > > But still, really, alignment of 64bit integers should be 64, on all platforms. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4986a2de17d74387f46308d8e7ec9309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637514350553019597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qEe3m6pZ%2BZHayaHaaZAgA5EKWzTsUWqj2K13P6Ui0LA%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 16 00:19:23 2021 From: jayk123 at hotmail.com (Jay K) Date: Mon, 15 Mar 2021 23:19:23 +0000 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms In-Reply-To: References: , , , Message-ID: Just to confirm..the alignment in Windows/x86 does vary between globals, heap, and locals. I haven't looked into gcc, Linux etc. yet. - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 8:15 PM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms Here, Visual C++ does do the dynamic stack alignment. No big deal? #include #include #include int main() { __m64 a; printf("%d %p\n", (int)__alignof(a), &a); } _main: 00000000: 53 push ebx 00000001: 8B DC mov ebx,esp 00000003: 83 EC 08 sub esp,8 00000006: 83 E4 F8 and esp,0FFFFFFF8h <=== 00000009: 83 C4 04 add esp,4 0000000C: 55 push ebp 0000000D: 8B 6B 04 mov ebp,dword ptr [ebx+4] 00000010: 89 6C 24 04 mov dword ptr [esp+4],ebp 00000014: 8B EC mov ebp,esp 00000016: 83 EC 08 sub esp,8 00000019: 8D 45 F8 lea eax,[ebp-8] 0000001C: 50 push eax 0000001D: 6A 08 push 8 0000001F: 68 00 00 00 00 push offset $SG27991 00000024: E8 00 00 00 00 call _printf - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Monday, March 15, 2021 8:09 PM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms Heap alignment on Win32 is 2 pointers, so that part is covered. Object sections are generally at least page-aligned. That's why sections exist, to have different page protections. So that is covered too. In old obscure cases they would be 1-aligned, like to optimize the size of drivers and make them read/write/execute, back before security mattered and read-only and execute-only was worth a little size. But stack, hm. This prints 8 for alignment but indeed sometimes the assert fails. #include #include int main() { __int64 a; printf("%d %p\n", __alignof(a), &a); assert (((size_t)&a % __alignof(a)) == 0); } Interlocked operations on stack are less common though, because stack tends not to be shared across threads. Good enough?? I agree it seems wierd. I'm sure Visual C++ does sometimes adjust stack alignment. Presumably for double/sse2 stuff, not mere int64. It is also profitable to have more similar rules across platforms. So pickles and similar don't need conversion. Granted, it might not be possible to eliminate all conversion. And also, again, to match the C compiler. #include #include int main() { struct { __int64 a; int b; } c; struct { int d[3]; } e; printf("%d %d\n", (int)sizeof(c), (int)sizeof(e)); } is 16 and 12, but presumably both 16 in analogous 32bit Modula-3. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, March 15, 2021 7:57 PM To: Jay K ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms On 3/14/21 6:24 AM, Jay K wrote: > This: > > PROCEDURE Init32 () = > BEGIN > (* Change size and aligment: *) > Integer := Int32; > Word := Word32; > Address := Word32; > Address.cg_type := CGType.Addr; > (* Change only alignment. Size is always 64: *) > * Longint.align := 32; <==* > * Long.align := 32; <==* > * Longreal.align := 32; <==* > * Extended.align := 32; <==* > END Init32; > > is not great. Perhaps terrible. > > Because x86 has InterlockedCompareExchange64, that wants 64 alignment. > > The Longreal part is also a bad idea. > > This is an old way of achieving 64bit alignment, before the advent of 64bit integers: > > union LARGE_INTEGER > { > struct { int HighPart, LowPart }; > double Alignment; > }; > > And it should continue to work. > > The 32bit and 64bit rules should be more the same. > > I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. > If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. > But it is likely difficult/impossible. The interlocked operations could be over in C. We had a discussion a while back about 64-bit alignment of 64-bit scalars on 32-bit machines. If I recall correctly, somebody (Jay?) advocated for doing it. I remember realizing later that this would be just about untenable. As I recall, some of the reasons were that all allocated blocks would have to be 64-bit aligned to begin with. This would include things OS and library-provided things like malloc, object module sections, loader, etc. Also the GC (every object) and even runtime adjustment of activation records on calls on machines that do runtime operand stack pushes. > > Also this maybe disagrees with C alignment. Let's see: > > C:\s\cm3\scripts\python>type 1.c > struct { > __int64 a; > int b; > } a; > > int b = sizeof(a); > > #pragma pack(1) > > struct { > __int64 a; > int b; > } c; > > int d = sizeof(c); > > C:\s\cm3\scripts\python>cl /c /FAsc 1.c > Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 > Copyright (C) Microsoft Corporation. All rights reserved. > > 1.c > > C:\s\cm3\scripts\python>more 1.cod > ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 > > COMM _a:BYTE:010H > _b DD 010H > _d DD 0cH > > So this seems like it /could really /break things. > i.e. struct stat in m3core. > Except that I sorted things by size so it'll tend to work out. > > But still, really, alignment of 64bit integers should be 64, on all platforms. > > - Jay > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4986a2de17d74387f46308d8e7ec9309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637514350553019597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qEe3m6pZ%2BZHayaHaaZAgA5EKWzTsUWqj2K13P6Ui0LA%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Mar 16 00:29:10 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Mon, 15 Mar 2021 19:29:10 -0400 Subject: [M3devel] alignment of 64bit integers/double on 32bit platforms In-Reply-To: References: Message-ID: <20210315232909.6bgtmdbory5q4rf2@topoi.pooq.com> On Mon, Mar 15, 2021 at 11:19:23PM +0000, Jay K wrote: > Just to confirm..the alignment in Windows/x86 does vary between globals, heap, and locals. > I haven't looked into gcc, Linux etc. yet. SOme decades ago I was involved with compatibility with an IBM compiler on some IBM Unix machine -- I forget which one. It turned out that structures containing a long real as its recursively first element (recursively in case of structures containing structures) would be aligned on a long real boundary. It did not do long real alignment on account of any other placement of a long real element. So struct{long float fpp; } would be long-real-aligned. struct{float foo; long float fpp;} would not be. I thought this was weird. Just thought I'd mention it here as an example of how weird alignment rules can be. -- hendrik > > - Jay > ________________________________ > From: Jay K > Sent: Monday, March 15, 2021 8:15 PM > To: rodney.m.bates at acm.org ; m3devel > Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms > > Here, Visual C++ does do the dynamic stack alignment. > No big deal? > > #include > #include > #include > > int main() > { > __m64 a; > printf("%d %p\n", (int)__alignof(a), &a); > } > > _main: > 00000000: 53 push ebx > 00000001: 8B DC mov ebx,esp > 00000003: 83 EC 08 sub esp,8 > 00000006: 83 E4 F8 and esp,0FFFFFFF8h <=== > 00000009: 83 C4 04 add esp,4 > 0000000C: 55 push ebp > 0000000D: 8B 6B 04 mov ebp,dword ptr [ebx+4] > 00000010: 89 6C 24 04 mov dword ptr [esp+4],ebp > 00000014: 8B EC mov ebp,esp > 00000016: 83 EC 08 sub esp,8 > 00000019: 8D 45 F8 lea eax,[ebp-8] > 0000001C: 50 push eax > 0000001D: 6A 08 push 8 > 0000001F: 68 00 00 00 00 push offset $SG27991 > 00000024: E8 00 00 00 00 call _printf > > - Jay > > ________________________________ > From: M3devel on behalf of Jay K > Sent: Monday, March 15, 2021 8:09 PM > To: rodney.m.bates at acm.org ; m3devel > Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms > > Heap alignment on Win32 is 2 pointers, so that part is covered. > Object sections are generally at least page-aligned. That's why sections exist, to have different page protections. So that is covered too. > > In old obscure cases they would be 1-aligned, like to optimize the size of drivers and make them read/write/execute, back before security mattered and read-only and execute-only was worth a little size. > > But stack, hm. > This prints 8 for alignment but indeed sometimes the assert fails. > > #include > #include > > int main() > { > __int64 a; > printf("%d %p\n", __alignof(a), &a); > assert (((size_t)&a % __alignof(a)) == 0); > } > > Interlocked operations on stack are less common though, because stack tends not to be shared across threads. > Good enough?? > I agree it seems wierd. > I'm sure Visual C++ does sometimes adjust stack alignment. > Presumably for double/sse2 stuff, not mere int64. > > It is also profitable to have more similar rules across platforms. > So pickles and similar don't need conversion. > Granted, it might not be possible to eliminate all conversion. > > And also, again, to match the C compiler. > > #include > #include > > > int main() > { > struct > { > __int64 a; > int b; > } c; > > struct > { > int d[3]; > } e; > > printf("%d %d\n", (int)sizeof(c), (int)sizeof(e)); > > } > > is 16 and 12, but presumably both 16 in analogous 32bit Modula-3. > > - Jay > > ________________________________ > From: Rodney M. Bates > Sent: Monday, March 15, 2021 7:57 PM > To: Jay K ; rodney.m.bates at acm.org ; m3devel > Subject: Re: [M3devel] alignment of 64bit integers/double on 32bit platforms > > > > On 3/14/21 6:24 AM, Jay K wrote: > > This: > > > > PROCEDURE Init32 () = > > BEGIN > > (* Change size and aligment: *) > > Integer := Int32; > > Word := Word32; > > Address := Word32; > > Address.cg_type := CGType.Addr; > > (* Change only alignment. Size is always 64: *) > > * Longint.align := 32; <==* > > * Long.align := 32; <==* > > * Longreal.align := 32; <==* > > * Extended.align := 32; <==* > > END Init32; > > > > is not great. Perhaps terrible. > > > > Because x86 has InterlockedCompareExchange64, that wants 64 alignment. > > > > The Longreal part is also a bad idea. > > > > This is an old way of achieving 64bit alignment, before the advent of 64bit integers: > > > > union LARGE_INTEGER > > { > > struct { int HighPart, LowPart }; > > double Alignment; > > }; > > > > And it should continue to work. > > > > The 32bit and 64bit rules should be more the same. > > > > I realize, ignoring the double part, 64bit loads on 32bit platforms are usually done in two pieces, and this alignment usually suffices. > > If a longint could be marked with greater alignment, or we could notice the interlocked operations against, that might help. > > But it is likely difficult/impossible. The interlocked operations could be over in C. > > We had a discussion a while back about 64-bit alignment of 64-bit scalars on > 32-bit machines. If I recall correctly, somebody (Jay?) advocated for doing > it. > > I remember realizing later that this would be just about untenable. As I recall, > some of the reasons were that all allocated blocks would have to be 64-bit aligned > to begin with. This would include things OS and library-provided things > like malloc, object module sections, loader, etc. Also the GC (every > object) and even runtime adjustment of activation records on calls on > machines that do runtime operand stack pushes. > > > > > Also this maybe disagrees with C alignment. Let's see: > > > > C:\s\cm3\scripts\python>type 1.c > > struct { > > __int64 a; > > int b; > > } a; > > > > int b = sizeof(a); > > > > #pragma pack(1) > > > > struct { > > __int64 a; > > int b; > > } c; > > > > int d = sizeof(c); > > > > C:\s\cm3\scripts\python>cl /c /FAsc 1.c > > Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29337 for x86 > > Copyright (C) Microsoft Corporation. All rights reserved. > > > > 1.c > > > > C:\s\cm3\scripts\python>more 1.cod > > ; Listing generated by Microsoft (R) Optimizing Compiler Version 19.28.29337.0 > > > > COMM _a:BYTE:010H > > _b DD 010H > > _d DD 0cH > > > > So this seems like it /could really /break things. > > i.e. struct stat in m3core. > > Except that I sorted things by size so it'll tend to work out. > > > > But still, really, alignment of 64bit integers should be 64, on all platforms. > > > > - Jay > > > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C4986a2de17d74387f46308d8e7ec9309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637514350553019597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qEe3m6pZ%2BZHayaHaaZAgA5EKWzTsUWqj2K13P6Ui0LA%3D&reserved=0 > > > > -- > Rodney Bates > rodney.m.bates at acm.org > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jayk123 at hotmail.com Tue Mar 16 01:11:44 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 16 Mar 2021 00:11:44 +0000 Subject: [M3devel] sbrk vs. mmap In-Reply-To: References: Message-ID: System seems to work with mmap. Surely it must, given the behavior of all? the other systems? Unless mmap is somehow much more random on Linux than any others? Proceed? m3core: Switch Linux/amd64 from sbrk to mmap. by jaykrell ? Pull Request #175 ? modula3/cm3 (github.com) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 8:40 AM To: m3devel Subject: sbrk vs. mmap Wow. So, you know, I've been diffing platform "object code" ( C :) ). For now RISCV64_LINUX to AMD64_LINUX, to vary the least..and what do I find: readonly RTOSbrk = { % % memory from mmap seemed to be too spread out for the allocator's data structures to describe % "AMD64_LINUX" : 1, % % Several platforms don't define MAP_ANON or MAP_ANONYMOUS or even have Umman, % and sbrk was the historical implementation, but we probably don't care. % see http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/POSIX/RTOS.m3?rev=1.1;content-type=text%2Fplain % % This is very much a guess and should be confirmed if systems are available. % "AIX386" : 1, "IBMR2" : 1, "IBMRT" : 1, "IRIX5" : 1, % Outdated? Seems not. "SPARC" : 1, % SunOS 4.x } I have little direct recollection of this, esp. the AMD64_LINUX part, but.. c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 13) % memory from mmap seemed to be too spread out for the allocator's data structures to describe c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 14) % c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 15) "AMD64_LINUX" : 1, Does anyone believe that? That we need sbrk here? On this one heavily used platform? I mean. I guess, sbrk will just keep incrementing or decrementing. mmap could be kinda random. But this seems kinda crazy/horrible to be using these days. On just one platform (the others there don't count). I both hate to change this and hate to leave it this way. I'll try changing it. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 16 04:49:16 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 16 Mar 2021 03:49:16 +0000 Subject: [M3devel] sbrk vs. mmap In-Reply-To: References: , Message-ID: Actually, sorry.. Let's leave it for now. Let's make sure the GC data structures "scale" to arbitrarily spread out allocations, like a page at 0, or at least a page at pagesize, or some "very small" number, and a page at -pagesize or some "very large" number. Modern systems do not yet implement 64 bits of virtual address space decoding but maybe plan for that. Specifically I guess rather than being representing "like" an array of fixed sized pages from lowest seen to highest seen, there should be a tree of variably sized ranges, where the sizes of the ranges are still fixed size, but depend on the level in the tree. Like a page table. Alternatively, the sizes can be arbitrary and the structures even smaller, but that requires a binary search, and I imagine per pointer reference in a barrier, so too slow. I did a bad job earlier with git. There is pretty good evidence that I did see a problem and this did workaround it. I forced push to restore this. Let me know if that is really actually painful and I should in future always revert instead. My goal of unified backend output can be met w/o making this change. So let's split that up. Again, sorry. We should revisit this, but just switching to mmap, w/o fixing the data structures, with the reason of unified backend output, isn't great. https://github.com/modula3/cm3/commit/518f93e67ed8f3291a5cd5cf0a39d1fefbc79969 https://github.com/modula3/cm3/commit/384b3cc05fcedf4307f077f49cde3d6675b039c6 - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 16, 2021 1:31 AM To: Jay K Cc: m3devel Subject: Re: [M3devel] sbrk vs. mmap Any idea what the real problem was that caused amd64 to be set to using sbrk in the first place? On Mon, Mar 15, 2021 at 5:12 PM Jay K > wrote: System seems to work with mmap. Surely it must, given the behavior of all? the other systems? Unless mmap is somehow much more random on Linux than any others? Proceed? m3core: Switch Linux/amd64 from sbrk to mmap. by jaykrell ? Pull Request #175 ? modula3/cm3 (github.com) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 8:40 AM To: m3devel > Subject: sbrk vs. mmap Wow. So, you know, I've been diffing platform "object code" ( C :) ). For now RISCV64_LINUX to AMD64_LINUX, to vary the least..and what do I find: readonly RTOSbrk = { % % memory from mmap seemed to be too spread out for the allocator's data structures to describe % "AMD64_LINUX" : 1, % % Several platforms don't define MAP_ANON or MAP_ANONYMOUS or even have Umman, % and sbrk was the historical implementation, but we probably don't care. % see http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/POSIX/RTOS.m3?rev=1.1;content-type=text%2Fplain % % This is very much a guess and should be confirmed if systems are available. % "AIX386" : 1, "IBMR2" : 1, "IBMRT" : 1, "IRIX5" : 1, % Outdated? Seems not. "SPARC" : 1, % SunOS 4.x } I have little direct recollection of this, esp. the AMD64_LINUX part, but.. c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 13) % memory from mmap seemed to be too spread out for the allocator's data structures to describe c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 14) % c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 15) "AMD64_LINUX" : 1, Does anyone believe that? That we need sbrk here? On this one heavily used platform? I mean. I guess, sbrk will just keep incrementing or decrementing. mmap could be kinda random. But this seems kinda crazy/horrible to be using these days. On just one platform (the others there don't count). I both hate to change this and hate to leave it this way. I'll try changing it. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Tue Mar 16 06:11:28 2021 From: jayk123 at hotmail.com (Jay K) Date: Tue, 16 Mar 2021 05:11:28 +0000 Subject: [M3devel] sbrk vs. mmap In-Reply-To: References: , Message-ID: I hadn't read the code either. ? But quite possibly an array with an element per page. roughly speaking. typedef struct PageInfo_t { ... }; PageInfo_t pageInfo[]; #define PageSize 1MB pageInfo[ (highestSeenAddress - LowestSeenAddress) / PageSize] ... RTCollector.m3 PROCEDURE GrowHeap (pp: INTEGER): BOOLEAN = (* determine the new boundaries of the allocated pages *) IF p0 = Nil THEN newP0 := firstNewPage; newP1 := lastNewPage + 1; ELSIF firstNewPage < p0 THEN newP0 := firstNewPage; newP1 := p1; ELSIF p1 <= lastNewPage THEN newP0 := p0; newP1 := lastNewPage + 1; ELSE newP0 := p0; newP1 := p1; END; (* extend the side arrays if necessary *) so I think it is what I said, and solution is what I said. We just need to decide how many bits to peel at each level. 8? 10? 16? 20? x86 likes to peel 9 bits at a time (512gb, 1gb, 2mb, 4k), but we don't have to match, and they have the luxury of deciding the total number of bits, fewer than 64. We might want to plan, for something that lasts forever w/o revisiting. The fewer per bit jump, the smaller the minimum size and the smaller the waste, but the more jumps to span a large range. i.e. 16 bits at a time, walk up to 4 levels. Fewer walks if the data missing, actually 4 for present data. - Jay ________________________________ From: Mika Nystrom Sent: Tuesday, March 16, 2021 4:42 AM To: Jay K Cc: m3devel Subject: Re: [M3devel] sbrk vs. mmap I know I could read the code and find out myself, but what is the data structure in question? On Mon, Mar 15, 2021 at 8:49 PM Jay K > wrote: Actually, sorry.. Let's leave it for now. Let's make sure the GC data structures "scale" to arbitrarily spread out allocations, like a page at 0, or at least a page at pagesize, or some "very small" number, and a page at -pagesize or some "very large" number. Modern systems do not yet implement 64 bits of virtual address space decoding but maybe plan for that. Specifically I guess rather than being representing "like" an array of fixed sized pages from lowest seen to highest seen, there should be a tree of variably sized ranges, where the sizes of the ranges are still fixed size, but depend on the level in the tree. Like a page table. Alternatively, the sizes can be arbitrary and the structures even smaller, but that requires a binary search, and I imagine per pointer reference in a barrier, so too slow. I did a bad job earlier with git. There is pretty good evidence that I did see a problem and this did workaround it. I forced push to restore this. Let me know if that is really actually painful and I should in future always revert instead. My goal of unified backend output can be met w/o making this change. So let's split that up. Again, sorry. We should revisit this, but just switching to mmap, w/o fixing the data structures, with the reason of unified backend output, isn't great. https://github.com/modula3/cm3/commit/518f93e67ed8f3291a5cd5cf0a39d1fefbc79969 https://github.com/modula3/cm3/commit/384b3cc05fcedf4307f077f49cde3d6675b039c6 - Jay ________________________________ From: Mika Nystrom > Sent: Tuesday, March 16, 2021 1:31 AM To: Jay K > Cc: m3devel > Subject: Re: [M3devel] sbrk vs. mmap Any idea what the real problem was that caused amd64 to be set to using sbrk in the first place? On Mon, Mar 15, 2021 at 5:12 PM Jay K > wrote: System seems to work with mmap. Surely it must, given the behavior of all? the other systems? Unless mmap is somehow much more random on Linux than any others? Proceed? m3core: Switch Linux/amd64 from sbrk to mmap. by jaykrell ? Pull Request #175 ? modula3/cm3 (github.com) - Jay ________________________________ From: Jay K Sent: Monday, March 15, 2021 8:40 AM To: m3devel > Subject: sbrk vs. mmap Wow. So, you know, I've been diffing platform "object code" ( C :) ). For now RISCV64_LINUX to AMD64_LINUX, to vary the least..and what do I find: readonly RTOSbrk = { % % memory from mmap seemed to be too spread out for the allocator's data structures to describe % "AMD64_LINUX" : 1, % % Several platforms don't define MAP_ANON or MAP_ANONYMOUS or even have Umman, % and sbrk was the historical implementation, but we probably don't care. % see http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/POSIX/RTOS.m3?rev=1.1;content-type=text%2Fplain % % This is very much a guess and should be confirmed if systems are available. % "AIX386" : 1, "IBMR2" : 1, "IBMRT" : 1, "IRIX5" : 1, % Outdated? Seems not. "SPARC" : 1, % SunOS 4.x } I have little direct recollection of this, esp. the AMD64_LINUX part, but.. c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 13) % memory from mmap seemed to be too spread out for the allocator's data structures to describe c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 14) % c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 15) "AMD64_LINUX" : 1, Does anyone believe that? That we need sbrk here? On this one heavily used platform? I mean. I guess, sbrk will just keep incrementing or decrementing. mmap could be kinda random. But this seems kinda crazy/horrible to be using these days. On just one platform (the others there don't count). I both hate to change this and hate to leave it this way. I'll try changing it. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Tue Mar 16 12:15:05 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Tue, 16 Mar 2021 07:15:05 -0400 Subject: [M3devel] sbrk vs. mmap In-Reply-To: References: Message-ID: <20210316111505.xvjrw4t7lsrt4jrp@topoi.pooq.com> On Tue, Mar 16, 2021 at 05:11:28AM +0000, Jay K wrote: > I hadn't read the code either. ? > > But quite possibly an array with an element per page. > > roughly speaking. > > typedef struct PageInfo_t { ... }; > > PageInfo_t pageInfo[]; > > #define PageSize 1MB > > pageInfo[ (highestSeenAddress - LowestSeenAddress) / PageSize] ... > > RTCollector.m3 > > PROCEDURE GrowHeap (pp: INTEGER): BOOLEAN = > > (* determine the new boundaries of the allocated pages *) > IF p0 = Nil THEN > newP0 := firstNewPage; > newP1 := lastNewPage + 1; > ELSIF firstNewPage < p0 THEN > newP0 := firstNewPage; > newP1 := p1; > ELSIF p1 <= lastNewPage THEN > newP0 := p0; > newP1 := lastNewPage + 1; > ELSE > newP0 := p0; > newP1 := p1; > END; > (* extend the side arrays if necessary *) > > so I think it is what I said, and solution is what I said. > We just need to decide how many bits to peel at each level. > 8? > 10? > 16? > 20? > > x86 likes to peel 9 bits at a time (512gb, 1gb, 2mb, 4k), but we > don't have to match, and they have the luxury of deciding the total > number of bits, fewer than 64. > We might want to plan, for something that lasts forever w/o > revisiting. The fewer per bit jump, the smaller the minimum size and > the smaller the waste, but the more jumps to span a large range. > i.e. 16 bits at a time, walk up to 4 levels. Fewer walks if the data > missing, actually 4 for present data. You could leave out the largest levels if they have only one component each. -- hendrik > > - Jay > > ________________________________ > From: Mika Nystrom > Sent: Tuesday, March 16, 2021 4:42 AM > To: Jay K > Cc: m3devel > Subject: Re: [M3devel] sbrk vs. mmap > > I know I could read the code and find out myself, but what is the data structure in question? > > On Mon, Mar 15, 2021 at 8:49 PM Jay K > wrote: > Actually, sorry.. > > Let's leave it for now. > > Let's make sure the GC data structures "scale" > to arbitrarily spread out allocations, > like a page at 0, or at least a page at pagesize, > or some "very small" number, and a page at -pagesize or some "very large" number. > > Modern systems do not yet implement 64 bits of virtual address space decoding > but maybe plan for that. > > Specifically I guess rather than being representing "like" an array > of fixed sized pages from lowest seen to highest seen, there > should be a tree of variably sized ranges, where the sizes > of the ranges are still fixed size, but depend on the > level in the tree. > > Like a page table. > > Alternatively, the sizes can be arbitrary and the structures > even smaller, but that requires a binary search, and I imagine > per pointer reference in a barrier, so too slow. > > I did a bad job earlier with git. > There is pretty good evidence that I did see a problem > and this did workaround it. > > I forced push to restore this. > Let me know if that is really actually painful > and I should in future always revert instead. > > My goal of unified backend output can be met w/o making this change. > So let's split that up. > > Again, sorry. > We should revisit this, but just switching to mmap, w/o fixing the data structures, > with the reason of unified backend output, isn't great. > > https://github.com/modula3/cm3/commit/518f93e67ed8f3291a5cd5cf0a39d1fefbc79969 > https://github.com/modula3/cm3/commit/384b3cc05fcedf4307f077f49cde3d6675b039c6 > > - Jay > > ________________________________ > From: Mika Nystrom > > Sent: Tuesday, March 16, 2021 1:31 AM > To: Jay K > > Cc: m3devel > > Subject: Re: [M3devel] sbrk vs. mmap > > Any idea what the real problem was that caused amd64 to be set to using sbrk in the first place? > > On Mon, Mar 15, 2021 at 5:12 PM Jay K > wrote: > System seems to work with mmap. > Surely it must, given the behavior of all? the other systems? > Unless mmap is somehow much more random on Linux than any others? > Proceed? > m3core: Switch Linux/amd64 from sbrk to mmap. by jaykrell ? Pull Request #175 ? modula3/cm3 (github.com) > > - Jay > > ________________________________ > From: Jay K > Sent: Monday, March 15, 2021 8:40 AM > To: m3devel > > Subject: sbrk vs. mmap > > Wow. So, you know, I've been diffing platform "object code" ( C :) ). > > For now RISCV64_LINUX to AMD64_LINUX, to vary the least..and what do I find: > > readonly RTOSbrk = { > % > % memory from mmap seemed to be too spread out for the allocator's data structures to describe > % > "AMD64_LINUX" : 1, > > % > % Several platforms don't define MAP_ANON or MAP_ANONYMOUS or even have Umman, > % and sbrk was the historical implementation, but we probably don't care. > % see http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/POSIX/RTOS.m3?rev=1.1;content-type=text%2Fplain > % > % This is very much a guess and should be confirmed if systems are available. > % > "AIX386" : 1, > "IBMR2" : 1, > "IBMRT" : 1, > "IRIX5" : 1, % Outdated? Seems not. > "SPARC" : 1, % SunOS 4.x > } > > I have little direct recollection of this, esp. the AMD64_LINUX part, but.. > > c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 13) % memory from mmap seemed to be too spread out for the allocator's data structures to describe > c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 14) % > c8636fb09c (Jay Michael Krell 2008-12-29 10:23:41 +0000 15) "AMD64_LINUX" : 1, > > Does anyone believe that? > That we need sbrk here? On this one heavily used platform? > > I mean. I guess, sbrk will just keep incrementing or decrementing. > mmap could be kinda random. > > But this seems kinda crazy/horrible to be using these days. > On just one platform (the others there don't count). > > I both hate to change this and hate to leave it this way. > I'll try changing it. > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From rodney_bates at lcwb.coop Tue Mar 16 17:55:59 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 16 Mar 2021 11:55:59 -0500 Subject: [M3devel] packedVars broke Win32 record layout In-Reply-To: References: Message-ID: On 3/15/21 2:51 PM, Jay K wrote: > Ok this seems to fix. > I will test a bit more. > > diff --git a/m3-libs/m3core/src/win32/WinBase.i3 b/m3-libs/m3core/src/win32/WinBase.i3 > index a3e471f23..0afb27de8 100644 > --- a/m3-libs/m3core/src/win32/WinBase.i3 > +++ b/m3-libs/m3core/src/win32/WinBase.i3 > @@ -178,8 +178,8 @@ TYPE > ? ?PFILETIME = UNTRACED REF FILETIME; > ? ?LPFILETIME = PFILETIME; (* compat *) > ? ?FILETIME = RECORD > - ? ?dwLowDateTime : UINT32; > - ? ?dwHighDateTime: UINT32; > + ? ?dwLowDateTime : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?dwHighDateTime: BITS BITSIZE(UINT32) FOR UINT32; > ? ?END; > > ?(* System time is represented with the following structure: *) > @@ -1585,16 +1585,16 @@ TYPE > ? ?PBY_HANDLE_FILE_INFORMATION = UNTRACED REF BY_HANDLE_FILE_INFORMATION; > ? ?LPBY_HANDLE_FILE_INFORMATION = UNTRACED REF BY_HANDLE_FILE_INFORMATION; > ? ?BY_HANDLE_FILE_INFORMATION = RECORD > - ? ?dwFileAttributes ? ?: UINT32; > - ? ?ftCreationTime ? ? ?: FILETIME; > - ? ?ftLastAccessTime ? ?: FILETIME; > - ? ?ftLastWriteTime ? ? : FILETIME; > - ? ?dwVolumeSerialNumber: UINT32; > - ? ?nFileSizeHigh ? ? ? : UINT32; > - ? ?nFileSizeLow ? ? ? ?: UINT32; > - ? ?nNumberOfLinks ? ? ?: UINT32; > - ? ?nFileIndexHigh ? ? ?: UINT32; > - ? ?nFileIndexLow ? ? ? : UINT32; > + ? ?dwFileAttributes ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?ftCreationTime ? ? ?: BITS 64 FOR FILETIME; > + ? ?ftLastAccessTime ? ?: BITS 64 FOR FILETIME; > + ? ?ftLastWriteTime ? ? : BITS 64 FOR FILETIME; > + ? ?dwVolumeSerialNumber: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileSizeHigh ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileSizeLow ? ? ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nNumberOfLinks ? ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileIndexHigh ? ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileIndexLow ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > ? ?END; > > ?<*EXTERNAL GetFileInformationByHandle:WINAPI*> > @@ -2280,14 +2280,14 @@ TYPE > ? ?PWIN32_FIND_DATAA = UNTRACED REF WIN32_FIND_DATAA; > ? ?LPWIN32_FIND_DATAA = UNTRACED REF WIN32_FIND_DATAA; > ? ?WIN32_FIND_DATAA = RECORD > - ? ?dwFileAttributes ?: UINT32; > - ? ?ftCreationTime ? ?: FILETIME; > - ? ?ftLastAccessTime ?: FILETIME; > - ? ?ftLastWriteTime ? : FILETIME; > - ? ?nFileSizeHigh ? ? : UINT32; > - ? ?nFileSizeLow ? ? ?: UINT32; > - ? ?dwReserved0 ? ? ? : UINT32; > - ? ?dwReserved1 ? ? ? : UINT32; > + ? ?dwFileAttributes ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?ftCreationTime ? ?: BITS 64 FOR FILETIME; > + ? ?ftLastAccessTime ?: BITS 64 FOR FILETIME; > + ? ?ftLastWriteTime ? : BITS 64 FOR FILETIME; > + ? ?nFileSizeHigh ? ? : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileSizeLow ? ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?dwReserved0 ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?dwReserved1 ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > ? ? ?cFileName ? ? ? ? : ARRAY [0 .. MAX_PATH - 1] OF CHAR; > ? ? ?cAlternateFileName: ARRAY [0 .. 14 - 1] OF CHAR; > ? ?END; > @@ -2295,14 +2295,14 @@ TYPE > ? ?PWIN32_FIND_DATAW = UNTRACED REF WIN32_FIND_DATAW; > ? ?LPWIN32_FIND_DATAW = UNTRACED REF WIN32_FIND_DATAW; > ? ?WIN32_FIND_DATAW = RECORD > - ? ?dwFileAttributes ?: UINT32; > - ? ?ftCreationTime ? ?: FILETIME; > - ? ?ftLastAccessTime ?: FILETIME; > - ? ?ftLastWriteTime ? : FILETIME; > - ? ?nFileSizeHigh ? ? : UINT32; > - ? ?nFileSizeLow ? ? ?: UINT32; > - ? ?dwReserved0 ? ? ? : UINT32; > - ? ?dwReserved1 ? ? ? : UINT32; > + ? ?dwFileAttributes ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?ftCreationTime ? ?: BITS 64 FOR FILETIME; > + ? ?ftLastAccessTime ?: BITS 64 FOR FILETIME; > + ? ?ftLastWriteTime ? : BITS 64 FOR FILETIME; > + ? ?nFileSizeHigh ? ? : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?nFileSizeLow ? ? ?: BITS BITSIZE(UINT32) FOR UINT32; > + ? ?dwReserved0 ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > + ? ?dwReserved1 ? ? ? : BITS BITSIZE(UINT32) FOR UINT32; > ? ? ?cFileName ? ? ? ? : ARRAY [0 .. MAX_PATH - 1] OF WCHAR; > ? ? ?cAlternateFileName: ARRAY [0 .. 14 - 1] OF WCHAR; > ? ?END; > > I'm still a bit unnerved about this though. > > I was going to apply BITS for more broadly, but as I understand, it is not > always needed, even this might be overkill, and I don't really want to vouch for the most of the content > in these files. Header cloning and layout rules changing, not my favorites. > > Maybe fix, and then remove all users in-tree.. > > ?- Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* M3devel on behalf of Jay K > *Sent:* Monday, March 15, 2021 7:01 PM > *To:* rodney.m.bates at acm.org ; m3devel > *Subject:* Re: [M3devel] packedVars broke Win32 record layout > Maybe the Win32 header cloning was always wrong? > I will try putting BITS n FOR all over and see if that helps. ? > > ?- Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Jay K > *Sent:* Monday, March 15, 2021 6:45 PM > *To:* rodney.m.bates at acm.org ; m3devel > *Subject:* packedVars broke Win32 record layout > Rodney, > > Currently, with the packedVars work, cm3 on Win64 doesn't get far. > It tries to allocate too much memory. > > Here: > > 0:000> k > ?# Child-SP ? ? ? ? ?RetAddr ? ? ? ? ? ? ? Call Site > 00 (Inline Function) --------`-------- ? ? KERNELBASE!DebugBreak [minkernel\kernelbase\debug.c @ 143] > 01 000000cf`fe8fe7b8 00007ff7`41a25c73 ? ? KERNELBASE!wil::details::DebugBreak+0x2 [onecore\internal\sdk\inc\wil\opensource\wil\result_macros.h @ 1737] > 02 000000cf`fe8fe7c0 00007ff7`419dee5f ? ? cm3!RTOS__Crash+0x43 [C:\s\cm3\m3-libs\m3core\src\runtime\WIN32\RTOS.m3 @ 26] > 03 000000cf`fe8fe810 00007ff7`41a0d64f ? ? cm3!RTProcess__Crash+0x6f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTProcess.m3 @ 66] > 04 000000cf`fe8fe850 00007ff7`41a0d280 ? ? cm3!RTError__EndError+0x3f [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 119] > 05 000000cf`fe8fe890 00007ff7`41a25b57 ? ? cm3!RTError__MsgS+0xa0 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTError.m3 @ 41] > 06 000000cf`fe8fe8d0 00007ff7`41a2575e ? ? cm3!RTException__Crash+0x317 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 81] > 07 000000cf`fe8fe960 00007ff7`41a2564b ? ? cm3!RTException__DefaultBackstop+0xce [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 39] > 08 000000cf`fe8fe9d0 00007ff7`41a25f1a ? ? cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] > 09 000000cf`fe8fea10 00007ff7`41a2582a ? ? cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] > 0a 000000cf`fe8fea70 00007ff7`41a2564b ? ? cm3!RTException__DefaultBackstop+0x19a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 49] > 0b 000000cf`fe8feae0 00007ff7`41a25f1a ? ? cm3!RTException__InvokeBackstop+0x3b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTException.m3 @ 25] > 0c 000000cf`fe8feb20 00007ff7`419f7a33 ? ? cm3!RTException__Raise+0x9a [C:\s\cm3\m3-libs\m3core\src\runtime\ex_frame\RTExFrame.m3 @ 90] > 0d 000000cf`fe8feb80 00007ff7`41a2726a ? ? cm3!RTHooks__Raise+0xb3 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTHooks.m3 @ 80] > 0e 000000cf`fe8fec10 00007ff7`419df419 ? ? cm3!RuntimeError__Raise+0x6a [C:\s\cm3\m3-libs\m3core\src\runtime\common\RuntimeError.m3 @ 63] > 0f 000000cf`fe8fec60 00007ff7`416eef46 ? ? cm3!RTHooks__AllocateOpenArray+0x59 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTAllocator.m3 @ 146] > 10 000000cf`fe8fecb0 00007ff7`416e6820 ? ? cm3!QScanner__Init+0x206 [C:\s\cm3\m3-sys\m3quake\src\QScanner.m3 @ 48] > 11 000000cf`fe8fef00 00007ff7`416e03bd ? ? cm3!QCompiler__CompileFile+0x290 [C:\s\cm3\m3-sys\m3quake\src\QCompiler.m3 @ 47] > 12 000000cf`fe8ff290 00007ff7`415f5d3a ? ? cm3!Quake__Run+0x9d [C:\s\cm3\m3-sys\m3quake\src\Quake.m3 @ 19] > 13 000000cf`fe8ff2f0 00007ff7`415f5552 ? ? cm3!Main__DoIt+0x75a [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 96] > 14 000000cf`fe8ff6a0 00007ff7`419dda77 ? ? cm3!Main_M3+0x22 [C:\s\cm3\m3-sys\cm3\src\Main.m3 @ 233] > 15 000000cf`fe8ff6e0 00007ff7`419ddcfb ? ? cm3!RTLinker__RunMainBody+0x3e7 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 408] > 16 000000cf`fe8ff780 00007ff7`419dd478 ? ? cm3!RTLinker__AddUnitI+0x11b [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 116] > 17 000000cf`fe8ff7c0 00007ff7`415b1038 ? ? cm3!RTLinker__AddUnit+0x118 [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 125] > 18 000000cf`fe8ff820 00007ff7`41a47554 ? ? cm3!main+0x38 [C:\s\cm3\m3-sys\cm3\AMD64_NTc\_m3main.c @ 23] > 19 (Inline Function) --------`-------- ? ? cm3!invoke_main+0x22 [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] > 1a 000000cf`fe8ff850 00007ffb`74c17034 ? ? cm3!__scrt_common_main_seh+0x10c [d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] > 1b 000000cf`fe8ff890 00007ffb`74fc2651 ? ? KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] > 1c 000000cf`fe8ff8c0 00000000`00000000 ? ? ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] > > > The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. > Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. > ?Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. > ?Perhaps I should. The cloning really is still a bad idea. > > ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** > > But they no longer do. > > Here is the mismatch: > > > cm3!FileWin32__FileStatus+0x2f4: > 00007ff7`419c4194 e847350800 ? ? ?call ? ?cm3!__security_check_cookie (00007ff7`41a476e0) > 0:000> dv > ? _result_L_103 = 0x000000cf`fe8fee30 > ? ? ? ? h_L_104 = 0x000001f3`fb2b9fc8 "p" > ? ? L_166_L_167 = 294960936511.49194336 > ? ? L_168_L_169 = 0n889058230273 > ? ? L_162_L_163 = 0n1 > ? ? L_160_L_161 = 1 > ? ?status_L_101 = struct T53320305 > ? ? ? ffd_L_100 = struct TCCCF928E ?<== This. > ? ? ? ?ft_L_102 = 1 > ? ? ? ? ?_frame = struct FileWin32__FileStatus_Frame_t > ? ? L_164_L_165 = 0x000001f3`fb2f0228 "???" > > 0:000> dx -r1 (*((cm3!TCCCF928E *)0xcffe8fec48)) > (*((cm3!TCCCF928E *)0xcffe8fec48)) ? ? ? ? ? ? ? ? [Type: TCCCF928E] > ? ? [+0x000] dwFileAttributes : 0x20 [Type: unsigned int] > ? ? [+0x004] L_9 ? ? ? ? ? ? ?[Type: unsigned char [4]] <== Modula-3 inserted padded for alignment here > ? ? [+0x008] ftCreationTime ? [Type: T7AF5436C] > ? ? [+0x010] ftLastAccessTime [Type: T7AF5436C] > ? ? [+0x018] ftLastWriteTime ?[Type: T7AF5436C] > ? ? [+0x020] dwVolumeSerialNumber : 0x0 [Type: unsigned int] > ? ? [+0x024] nFileSizeHigh ? ?: 0xcf [Type: unsigned int] > ? ? [+0x028] nFileSizeLow ? ? : 0x1 [Type: unsigned int] > ? ? [+0x02c] nNumberOfLinks ? : 0x190000 [Type: unsigned int] > ? ? [+0x030] nFileIndexHigh ? : 0x30a228 [Type: unsigned int] > ? ? [+0x034] nFileIndexLow ? ?: 0x0 [Type: unsigned int] > 0:000> dt BY_HANDLE_FILE_INFORMATION > ntdll!BY_HANDLE_FILE_INFORMATION > ? ?+0x000 dwFileAttributes : Uint4B > ? ?+0x004 ftCreationTime ? : _FILETIME ?<== So the offset is wrong here and the rest. > ? ?+0x00c ftLastAccessTime : _FILETIME > ? ?+0x014 ftLastWriteTime ?: _FILETIME > ? ?+0x01c dwVolumeSerialNumber : Uint4B > ? ?+0x020 nFileSizeHigh ? ?: Uint4B > ? ?+0x024 nFileSizeLow ? ? : Uint4B > ? ?+0x028 nNumberOfLinks ? : Uint4B > ? ?+0x02c nFileIndexHigh ? : Uint4B > ? ?+0x030 nFileIndexLow ? ?: Uint4B > > So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. I can't reproduce this. I see a record with two UINT32s aligned 32 bits on a 64-bit target. > > I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. > i.e. widening everything to LONGINT or LONGREAL to avoid padding. > But, arguably I should not have to. > > ?- Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Tue Mar 16 18:55:27 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 16 Mar 2021 12:55:27 -0500 Subject: [M3devel] packedVars broke Win32 record layout In-Reply-To: References: Message-ID: <80b16df2-1954-4bdb-3447-1dc2fd653581@lcwb.coop> On 3/16/21 11:55 AM, Rodney M. Bates wrote: > > > On 3/15/21 2:51 PM, Jay K wrote: >>>> I'm still a bit unnerved about this though. >> >> I was going to apply BITS for more broadly, but as I understand, it is not >> always needed, even this might be overkill, and I don't really want to vouch for the most of the content >> in these files. Header cloning and layout rules changing, not my favorites. >> >> Maybe fix, and then remove all users in-tree.. >> >> ??- Jay >> >> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ >> *From:* M3devel on behalf of Jay K >> *Sent:* Monday, March 15, 2021 7:01 PM >> *To:* rodney.m.bates at acm.org ; m3devel >> *Subject:* Re: [M3devel] packedVars broke Win32 record layout >> Maybe the Win32 header cloning was always wrong? >> I will try putting BITS n FOR all over and see if that helps. ? >> >> ??- Jay >> >> >> >> The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. >> Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. >> ??Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. >> ??Perhaps I should. The cloning really is still a bad idea. >> >> ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** >> >> But they no longer do. >> >> So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. > > I can't reproduce this.? I see a record with two UINT32s aligned 32 bits on a > 64-bit target. > With this program: MODULE Main ; TYPE int = [-16_7fffffff-1 .. 16_7fffffff] ; TYPE unsigned_int = [16_0 .. 16_ffffffff] ; TYPE Rec = RECORD F1 : unsigned_int := 1 ; F2 : unsigned_int := 2 END ; TYPE Outer = RECORD O1 : unsigned_int := 11 ; O2 : Rec ; O3 : unsigned_int := 13 END ; VAR GI : int := 9 ; VAR GU : unsigned_int := 7 ; VAR GRec : Rec ; PROCEDURE Work ( ) = VAR OV : Outer ; VAR Stop : INTEGER ; BEGIN OV . O1 := 15 ; OV . O2 . F1 := 17 ; OV . O2 . F2 := 19 ; OV . O3 := 21 ; Stop := 45 END Work ; BEGIN Work ( ) END Main . I get: rodney at allegheny:~/proj/m3/exp/uint32/AMD64_LINUX$ m3gdb prog GNU gdb plus Modula-3 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"...Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". (m3gdb) b Main.m3:29 Breakpoint 1 at 0x4008e7: file ../src/Main.m3, line 29. (m3gdb) run Starting program: /home/rodney/proj/m3/exp/uint32/AMD64_LINUX/prog warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff6fe9f000 Can't disable VM GC. Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 [Thread debugging using libthread_db enabled] [New LWP 22680] ..... Breakpoint 1, Work () at ../src/Main.m3:29 29 ; Stop := 45 Current language: auto; currently Modula-3 (m3gdb) p ADR OV missing opening ( (m3gdb) p ADR (OV) $1 = 16_00007fff6fe5a2d8 (m3gdb) p ADR (OV.O1) $2 = 16_00007fff6fe5a2d8 (m3gdb) p ADR (OV.O2) $3 = 16_00007fff6fe5a2dc (m3gdb) p ADR (OV.O2.F1) $4 = 16_00007fff6fe5a2dc (m3gdb) p ADR (OV.O2.F2) $5 = 16_00007fff6fe5a2e0 (m3gdb) p ADR (OV.O3) $6 = 16_00007fff6fe5a2e4 (m3gdb) Everything goes up by 4 bytes. >> >> I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. >> i.e. widening everything to LONGINT or LONGREAL to avoid padding. >> But, arguably I should not have to. >> Is it possible that C is still doing "the usual conversions" on actual parameters, from the day when you could call a function without defining its formals? I intend and believe that the only things regarding memory layout that have changed are: - Refusing fewer packed cases than before. Those that were previously accepted should work the same, as the language mandates. (Well, OK, newly refusing to pass an actual of packed type as VAR is a new restriction. It was buggy before, though possibly not in every case.) - Giving sets that are not packed, size 8, 16, or 32 if they fit, instead of always one or more full words. There should not be Modula-3 sets passed to C code anyway. >> ??- Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel >> > -- Rodney Bates rodney.m.bates at acm.org From wagner at elegosoft.com Tue Mar 16 20:59:39 2021 From: wagner at elegosoft.com (Olaf Wagner) Date: Tue, 16 Mar 2021 20:59:39 +0100 Subject: [M3devel] Mailing lists and website resources at elegosoft.com Message-ID: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> Hello everybody! There?s much traffic here in recent weeks, and not all of that is caused by meaningful messages from real users. In fact, the m3 mailing lists at elegosoft.com are used increasingly for spamming and we get more and more abuse warnings from our hosting provider. I also notice that we still host the now 20 years old website for cm3 which hasn?t changed for more than 10 years. Due to the COVID-19 pandemic our resources are currently rather limited. System administration would like to shutdown and get rid of as many legacy systems and liabilities as possible. At least we need to close the open subscription interfaces on the mailing lists as they are abused for spamming. Some counter measures have been implemented, but they are not as effective as needed. So what I?m asking is - which of the existing mailing lists and web resources are actually still needed - what can be done to enable a migration for example to github - and who would like to take care of either maintaining or migrating the existing legacy resources? Our system administration will close the open subscription option at the next weekend. We can continue to provide the list services with explicit subscription via member recommendation or something similar if that is wanted and needed. We can of course also just shutdown the complete lists and provide archives of the last 20 years of mailings to import or make available on another platform. I myself still feel somehow attached to m3, but haven?t actively contributed for many years and am not likely to find the time soon to do so. So anybody who would like to step in and take over something we still host is welcome. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dmuysers at hotmail.com Tue Mar 16 21:13:14 2021 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 16 Mar 2021 20:13:14 +0000 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> Message-ID: Sorry, but I don?t see any spam on m3devel. All messages of this month have been technical Sent from my iPhone > On 16 Mar 2021, at 21:00, Olaf Wagner wrote: > > ?Hello everybody! > > There?s much traffic here in recent weeks, and not all of that is caused by > meaningful messages from real users. In fact, the m3 mailing lists at > elegosoft.com are used increasingly for spamming and we get more and more > abuse warnings from our hosting provider. > > I also notice that we still host the now 20 years old website for cm3 > which hasn?t changed for more than 10 years. > > Due to the COVID-19 pandemic our resources are currently rather limited. > System administration would like to shutdown and get rid of as many > legacy systems and liabilities as possible. At least we need to close > the open subscription interfaces on the mailing lists as they are abused > for spamming. Some counter measures have been implemented, but they are > not as effective as needed. > > So what I?m asking is > - which of the existing mailing lists and web resources are actually still > needed > - what can be done to enable a migration for example to github > - and who would like to take care of either maintaining or migrating the > existing legacy resources? > > Our system administration will close the open subscription option at the > next weekend. We can continue to provide the list services with explicit > subscription via member recommendation or something similar if that is > wanted and needed. We can of course also just shutdown the complete lists > and provide archives of the last 20 years of mailings to import or make > available on another platform. > > I myself still feel somehow attached to m3, but haven?t actively contributed > for many years and am not likely to find the time soon to do so. So anybody > who would like to step in and take over something we still host is welcome. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From wagner at elegosoft.com Wed Mar 17 01:08:46 2021 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 17 Mar 2021 01:08:46 +0100 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> Message-ID: <51F74432-7F83-4BBA-B3E3-FF531584203E@elegosoft.com> On 16 Mar 2021, at 21:13, dirk muysers wrote: > Sorry, but I don?t see any spam on m3devel. All messages of this > month have been technical Well yes, my formulation was not completely accurate. The subscription interface of the mailing lists is used to bounce spam messages to arbitrary users, not to the list members. To the recipients of the spam messages it looks like they come from our mailing list administration. Those recipients then complain about us sending spam at our hosting provider. Mail has become one of the most abused services in the internet, and keeping up a reasonably robust open infrastructure gets more and more difficult. It?s rather sad. > Sent from my iPhone > >> On 16 Mar 2021, at 21:00, Olaf Wagner wrote: >> >> ?Hello everybody! >> >> There?s much traffic here in recent weeks, and not all of that is >> caused by >> meaningful messages from real users. In fact, the m3 mailing lists at >> elegosoft.com are used increasingly for spamming and we get more and >> more >> abuse warnings from our hosting provider. >> >> I also notice that we still host the now 20 years old website for cm3 >> which hasn?t changed for more than 10 years. >> >> Due to the COVID-19 pandemic our resources are currently rather >> limited. >> System administration would like to shutdown and get rid of as many >> legacy systems and liabilities as possible. At least we need to close >> the open subscription interfaces on the mailing lists as they are >> abused >> for spamming. Some counter measures have been implemented, but they >> are >> not as effective as needed. >> >> So what I?m asking is >> - which of the existing mailing lists and web resources are actually >> still >> needed >> - what can be done to enable a migration for example to github >> - and who would like to take care of either maintaining or migrating >> the >> existing legacy resources? >> >> Our system administration will close the open subscription option at >> the >> next weekend. We can continue to provide the list services with >> explicit >> subscription via member recommendation or something similar if that >> is >> wanted and needed. We can of course also just shutdown the complete >> lists >> and provide archives of the last 20 years of mailings to import or >> make >> available on another platform. >> >> I myself still feel somehow attached to m3, but haven?t actively >> contributed >> for many years and am not likely to find the time soon to do so. So >> anybody >> who would like to step in and take over something we still host is >> welcome. >> >> Olaf >> -- >> Olaf Wagner -- elego Software Solutions GmbH -- >> http://www.elegosoft.com >> Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 >> 86 95 >> Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: >> DE163214194 >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jayk123 at hotmail.com Wed Mar 17 06:59:23 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 17 Mar 2021 05:59:23 +0000 Subject: [M3devel] packedVars broke Win32 record layout In-Reply-To: <80b16df2-1954-4bdb-3447-1dc2fd653581@lcwb.coop> References: , <80b16df2-1954-4bdb-3447-1dc2fd653581@lcwb.coop> Message-ID: I agree, I hadn't tried earlier, but the simplest case does not repro, i.e. timet = record uint32, uint32 end attrs = record uint32 , /*purported padding here */ timet end has size 4*3=12 as desired but the purported bug makes it 4*4=16 I'll dig more later. I do have the workaround either way, and can strengthen it (a C layer, like we for for Unix stat; it is better motivated for Unix since there are so many varations, vs. "just one" for Windows (unless you consider Windows to be the "n+1th" platform) (Aside I noticed this direction of a thin C layer at the bottom was started not by me but by Anthony. ? ) Thank you, - Jay ________________________________ From: Rodney M. Bates Sent: Tuesday, March 16, 2021 5:55 PM To: Jay K ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] packedVars broke Win32 record layout On 3/16/21 11:55 AM, Rodney M. Bates wrote: > > > On 3/15/21 2:51 PM, Jay K wrote: >>>> I'm still a bit unnerved about this though. >> >> I was going to apply BITS for more broadly, but as I understand, it is not >> always needed, even this might be overkill, and I don't really want to vouch for the most of the content >> in these files. Header cloning and layout rules changing, not my favorites. >> >> Maybe fix, and then remove all users in-tree.. >> >> - Jay >> >> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ >> *From:* M3devel on behalf of Jay K >> *Sent:* Monday, March 15, 2021 7:01 PM >> *To:* rodney.m.bates at acm.org ; m3devel >> *Subject:* Re: [M3devel] packedVars broke Win32 record layout >> Maybe the Win32 header cloning was always wrong? >> I will try putting BITS n FOR all over and see if that helps. ? >> >> - Jay >> >> >> >> The reason is due to a type mismatch between the Windows headers and the Modula-3 clone of them. >> Recall that the work I did to stop cloning /usr/include on Unix, I didn't do much of for Win32. >> Those headers are kinda easier to clone. There aren't n different Win32s like there are n different Unixes. >> Perhaps I should. The cloning really is still a bad idea. >> >> ** Therefore the layout rules in Modula-3 do need to match C on Windows. ** >> >> But they no longer do. >> >> So while the size of FILETIME is correctly 8, its alignment is now incorrectly 8. > > I can't reproduce this. I see a record with two UINT32s aligned 32 bits on a > 64-bit target. > With this program: MODULE Main ; TYPE int = [-16_7fffffff-1 .. 16_7fffffff] ; TYPE unsigned_int = [16_0 .. 16_ffffffff] ; TYPE Rec = RECORD F1 : unsigned_int := 1 ; F2 : unsigned_int := 2 END ; TYPE Outer = RECORD O1 : unsigned_int := 11 ; O2 : Rec ; O3 : unsigned_int := 13 END ; VAR GI : int := 9 ; VAR GU : unsigned_int := 7 ; VAR GRec : Rec ; PROCEDURE Work ( ) = VAR OV : Outer ; VAR Stop : INTEGER ; BEGIN OV . O1 := 15 ; OV . O2 . F1 := 17 ; OV . O2 . F2 := 19 ; OV . O3 := 21 ; Stop := 45 END Work ; BEGIN Work ( ) END Main . I get: rodney at allegheny:~/proj/m3/exp/uint32/AMD64_LINUX$ m3gdb prog GNU gdb plus Modula-3 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"...Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". (m3gdb) b Main.m3:29 Breakpoint 1 at 0x4008e7: file ../src/Main.m3, line 29. (m3gdb) run Starting program: /home/rodney/proj/m3/exp/uint32/AMD64_LINUX/prog warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff6fe9f000 Can't disable VM GC. Signal Stop Print Pass to program Description SIG64 No No Yes Real-time event 64 [Thread debugging using libthread_db enabled] [New LWP 22680] ..... Breakpoint 1, Work () at ../src/Main.m3:29 29 ; Stop := 45 Current language: auto; currently Modula-3 (m3gdb) p ADR OV missing opening ( (m3gdb) p ADR (OV) $1 = 16_00007fff6fe5a2d8 (m3gdb) p ADR (OV.O1) $2 = 16_00007fff6fe5a2d8 (m3gdb) p ADR (OV.O2) $3 = 16_00007fff6fe5a2dc (m3gdb) p ADR (OV.O2.F1) $4 = 16_00007fff6fe5a2dc (m3gdb) p ADR (OV.O2.F2) $5 = 16_00007fff6fe5a2e0 (m3gdb) p ADR (OV.O3) $6 = 16_00007fff6fe5a2e4 (m3gdb) Everything goes up by 4 bytes. >> >> I guess I can fix by loosening the strict Modula-3 <=> C layout coupling. >> i.e. widening everything to LONGINT or LONGREAL to avoid padding. >> But, arguably I should not have to. >> Is it possible that C is still doing "the usual conversions" on actual parameters, from the day when you could call a function without defining its formals? I intend and believe that the only things regarding memory layout that have changed are: - Refusing fewer packed cases than before. Those that were previously accepted should work the same, as the language mandates. (Well, OK, newly refusing to pass an actual of packed type as VAR is a new restriction. It was buggy before, though possibly not in every case.) - Giving sets that are not packed, size 8, 16, or 32 if they fit, instead of always one or more full words. There should not be Modula-3 sets passed to C code anyway. >> - Jay >> >> >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C1f0b3c85e48348128e1208d8e8a4bc3e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637515141518402513%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=FkofXj4OmYgifopRu98q5JJcTUxf7BabQAOow70vyO4%3D&reserved=0 >> > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 17 07:00:03 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 17 Mar 2021 06:00:03 +0000 Subject: [M3devel] sockaddr stuff Message-ID: btw, I expect to be able to revert most or all of the sockaddr changes, and instead have a little copying, or optionally copying layer, in Usocket.c Very much like is done for stat. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Wed Mar 17 07:35:45 2021 From: vvm at tut.by (vvm at tut.by) Date: Wed, 17 Mar 2021 09:35:45 +0300 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: <51F74432-7F83-4BBA-B3E3-FF531584203E@elegosoft.com> References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> <51F74432-7F83-4BBA-B3E3-FF531584203E@elegosoft.com> Message-ID: <25611615962847@mail.yandex.by> An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Wed Mar 17 09:44:54 2021 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 17 Mar 2021 09:44:54 +0100 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: <25611615962847@mail.yandex.by> References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> <51F74432-7F83-4BBA-B3E3-FF531584203E@elegosoft.com> <25611615962847@mail.yandex.by> Message-ID: On 17 Mar 2021, at 7:35, vvm at tut.by wrote: > ? No, HTTP(S) is not involved. The problem are the open SMTP interfaces of the mailing lists themselves. If the lists had an HTPPS interface, we could just put a Captcha on it to restrict the access. There is of course no completely open relay at Elego, just the open lists. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Mar 17 16:29:35 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 17 Mar 2021 10:29:35 -0500 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> Message-ID: On 3/16/21 2:59 PM, Olaf Wagner wrote: > Hello everybody! > > There?s much traffic here in recent weeks, and not all of that is caused by > meaningful messages from real users. In fact, the m3 mailing lists at > elegosoft.com are used increasingly for spamming and we get more and more > abuse warnings from our hosting provider. > > I also notice that we still host the now 20 years old website for cm3 > which hasn?t changed for more than 10 years. On a few occasions, I have started working on some updates to the web pages, but given up trying to reconcile the pages in the repository with what is actually being served. I had presumed the right way was to update the repository, then maybe move things to the server. The lists of file names in the two places have significant differences, and same-named files sometimes have significantly different content. I have felt a bit timid about tampering with them. Maybe I need to get bolder. > > Due to the COVID-19 pandemic our resources are currently rather limited. > System administration would like to shutdown and get rid of as many > legacy systems and liabilities as possible. At least we need to close > the open subscription interfaces on the mailing lists as they are abused > for spamming. Some counter measures have been implemented, but they are > not as effective as needed. > > So what I?m asking is > - which of the existing mailing lists and web resources are actually still > needed I think the 3 lists and the web pages are all needed. M3-announce gets used only occasionally, but that's the idea of an announce list. > - what can be done to enable a migration for example to github > - and who would like to take care of either maintaining or migrating the > existing legacy resources? > > Our system administration will close the open subscription option at the > next weekend. We can continue to provide the list services with explicit > subscription via member recommendation or something similar if that is > wanted and needed. We can of course also just shutdown the complete lists > and provide archives of the last 20 years of mailings to import or make > available on another platform. > > I myself still feel somehow attached to m3, but haven?t actively contributed > for many years and am not likely to find the time soon to do so. So anybody > who would like to step in and take over something we still host is welcome. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Wed Mar 17 17:18:18 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 17 Mar 2021 16:18:18 +0000 Subject: [M3devel] [M3announce] Mailing lists and website resources at elegosoft.com In-Reply-To: References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com>, Message-ID: Oh, are these files live that I edited/renamed? docs: Replace porting instructions with something ? modula3/cm3 at ab4abde (github.com) I agree..it is not easy for me to document..what I know. What order? How to make it hierarchical vs. overly long stream? How to answer multiple/different questions w/o overloading with all the answers? Start writing new documenation/readme. by jaykrell ? Pull Request #152 ? modula3/cm3 (github.com) how to move discussion to github? Slack and Discord are so popular, and I guess Github will compete, but like the archival and search behavior of email. - Jay ________________________________ From: M3announce on behalf of Rodney M. Bates Sent: Wednesday, March 17, 2021 3:29 PM To: Olaf Wagner ; m3announce at elegosoft.com ; m3devel Subject: Re: [M3announce] [M3devel] Mailing lists and website resources at elegosoft.com On 3/16/21 2:59 PM, Olaf Wagner wrote: > Hello everybody! > > There?s much traffic here in recent weeks, and not all of that is caused by > meaningful messages from real users. In fact, the m3 mailing lists at > elegosoft.com are used increasingly for spamming and we get more and more > abuse warnings from our hosting provider. > > I also notice that we still host the now 20 years old website for cm3 > which hasn?t changed for more than 10 years. On a few occasions, I have started working on some updates to the web pages, but given up trying to reconcile the pages in the repository with what is actually being served. I had presumed the right way was to update the repository, then maybe move things to the server. The lists of file names in the two places have significant differences, and same-named files sometimes have significantly different content. I have felt a bit timid about tampering with them. Maybe I need to get bolder. > > Due to the COVID-19 pandemic our resources are currently rather limited. > System administration would like to shutdown and get rid of as many > legacy systems and liabilities as possible. At least we need to close > the open subscription interfaces on the mailing lists as they are abused > for spamming. Some counter measures have been implemented, but they are > not as effective as needed. > > So what I?m asking is > - which of the existing mailing lists and web resources are actually still > needed I think the 3 lists and the web pages are all needed. M3-announce gets used only occasionally, but that's the idea of an announce list. > - what can be done to enable a migration for example to github > - and who would like to take care of either maintaining or migrating the > existing legacy resources? > > Our system administration will close the open subscription option at the > next weekend. We can continue to provide the list services with explicit > subscription via member recommendation or something similar if that is > wanted and needed. We can of course also just shutdown the complete lists > and provide archives of the last 20 years of mailings to import or make > available on another platform. > > I myself still feel somehow attached to m3, but haven?t actively contributed > for many years and am not likely to find the time soon to do so. So anybody > who would like to step in and take over something we still host is welcome. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3announce mailing list M3announce at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3announce&data=04%7C01%7C%7Cf241370a849249c7eec608d8e9598c7e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637515918114357190%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sJHpiA3KiPJw45eM0XNh3jdTFWHTboFqCM1s%2BoLkMbg%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Wed Mar 17 17:21:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 17 Mar 2021 16:21:10 +0000 Subject: [M3devel] sockaddr stuff In-Reply-To: References: Message-ID: I ended up not doing the revert. The {accept,connect,bind}_un functions still seem to clean up the code. Revert is still viable if people like dealing with sockaddr (in Modula3, it is dealt with either way). And more refactoring can be avoided, leveraging just the underlying change. m3core: Change Usocket to copy sockaddr in or out. ? modula3/cm3 at 0584b28 (github.com) We can also add {accept,connect,bind}_in or _ipv4 functions. Though this code isn't right/modern, it ignores ipv6. There is similar code already adapted to ipv6. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Wednesday, March 17, 2021 6:00 AM To: m3devel at elegosoft.com Subject: [M3devel] sockaddr stuff btw, I expect to be able to revert most or all of the sockaddr changes, and instead have a little copying, or optionally copying layer, in Usocket.c Very much like is done for stat. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Mar 17 18:31:47 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 17 Mar 2021 12:31:47 -0500 Subject: [M3devel] The promised doc on sizes, alignments, packed types, and restrictions. Message-ID: CM3, sizes, alignments, packed types, and restrictions. First, some terminology: - Here, "aggregate" means an array or record, including the record portion of an object. - Here, "component" means an element of an array or a field of a record. - In Modula-3, "packed" means declared as BITS n FOR T. - Here, "effectively packed" means has a packed type and is a component. Next, a review of some sometimes missed properties of packed types. These are Modula-3 properties, not just implementation properties. - The term "packed" refers to the component, not to the aggregate that contains it. This is confusing, because, intuitively, "packed array", for example, suggests an array whose elements are squeezed smaller than usual and is often used informally with this meaning. But in the terminology of the Modula-3 definition, a packed array would be something typed BITS FOR ARRAY ... that is a component of a containing aggregate. - BITS n FOR T does nothing to change the value set of T. If n is insufficient for the machine representation of unpacked T, the compiler must give an error. If n is larger than necessary, values stored will still be statically or dynamically checked to lie in the range of unpacked T. - BITS n FOR T affects the memory layout only of components. - No padding is inserted before an effectively packed component. - Regardless of whether a value is a component, T and BITS n FOR T are not equal types. This only makes sense, since effectiveness is not a property of the type, but rather where it is used. There are a few cases where type equality matters even when the packing is not effective. Examples are inside other types and parameters passed VAR. These two types are, however, mutually assignable, which in most cases is what matters. - Arithmetic is always done in full INTEGER or LONGINT, regardless of where values are fetched from or stored into. CM3 implementation layout of values that are not effectively packed: - Scalar values occupy the smallest of 8, 16, 32, or 64 bits that will hold the value. This now includes sets not larger than a word. Larger sets occupy a multiple of full words. For types whose ORD value range lies within [0..LAST(INTEGER)], unsigned interpretation is used. Alignment is the same as the size, but not larger than the word size. - Records and arrays have their components laid out padded as necessary for alignment, for records, in the order declared. The alignment of an array is the alignment of its elements. The alignment of a record is the maximum of the alignments of the fields. These rules apply recursively for aggregates inside of aggregates . CM3 implementation restrictions on packed types: - An actual parameter passed to a VAR formal can not have a packed type, not even if the packing is ineffective or the bit count happens to equal that of the unpacked base type. - An actual parameter that is not at least byte-aligned can not be passed READONLY. - A scalar component can not straddle a word boundary. As a consequence, elements of an open array smaller than byte size are limited to 1, 2, or 4 bits. - The bit size of scalar elements of an open array must evenly divide 64. - An array with multiple open dimensions and smaller than byte-sized elements can not be partially subscripted. I.e., subscripts for all dimensions must be supplied simultaneously. - An array constructor must have at least byte alignment. - Elements of an array constructor whose length is not static must have a byte multiple size. This can arise only in complicated cases where the constructor has multiple open dimensions and all the inner arrays at some same open level are open and non-constructors. (follow that?) - The elements of a SUBARRAY must have at least byte alignment. - The actual parameter to ADR must have at least byte alignment. -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Wed Mar 17 18:41:12 2021 From: jayk123 at hotmail.com (Jay K) Date: Wed, 17 Mar 2021 17:41:12 +0000 Subject: [M3devel] The promised doc on sizes, alignments, packed types, and restrictions. In-Reply-To: References: Message-ID: Aside: Should probably be in an .md or .txt in a PR? ________________________________ From: M3devel on behalf of Rodney M. Bates Sent: Wednesday, March 17, 2021 5:31 PM To: m3devel Subject: [M3devel] The promised doc on sizes, alignments, packed types, and restrictions. CM3, sizes, alignments, packed types, and restrictions. First, some terminology: - Here, "aggregate" means an array or record, including the record portion of an object. - Here, "component" means an element of an array or a field of a record. - In Modula-3, "packed" means declared as BITS n FOR T. - Here, "effectively packed" means has a packed type and is a component. Next, a review of some sometimes missed properties of packed types. These are Modula-3 properties, not just implementation properties. - The term "packed" refers to the component, not to the aggregate that contains it. This is confusing, because, intuitively, "packed array", for example, suggests an array whose elements are squeezed smaller than usual and is often used informally with this meaning. But in the terminology of the Modula-3 definition, a packed array would be something typed BITS FOR ARRAY ... that is a component of a containing aggregate. - BITS n FOR T does nothing to change the value set of T. If n is insufficient for the machine representation of unpacked T, the compiler must give an error. If n is larger than necessary, values stored will still be statically or dynamically checked to lie in the range of unpacked T. - BITS n FOR T affects the memory layout only of components. - No padding is inserted before an effectively packed component. - Regardless of whether a value is a component, T and BITS n FOR T are not equal types. This only makes sense, since effectiveness is not a property of the type, but rather where it is used. There are a few cases where type equality matters even when the packing is not effective. Examples are inside other types and parameters passed VAR. These two types are, however, mutually assignable, which in most cases is what matters. - Arithmetic is always done in full INTEGER or LONGINT, regardless of where values are fetched from or stored into. CM3 implementation layout of values that are not effectively packed: - Scalar values occupy the smallest of 8, 16, 32, or 64 bits that will hold the value. This now includes sets not larger than a word. Larger sets occupy a multiple of full words. For types whose ORD value range lies within [0..LAST(INTEGER)], unsigned interpretation is used. Alignment is the same as the size, but not larger than the word size. - Records and arrays have their components laid out padded as necessary for alignment, for records, in the order declared. The alignment of an array is the alignment of its elements. The alignment of a record is the maximum of the alignments of the fields. These rules apply recursively for aggregates inside of aggregates . CM3 implementation restrictions on packed types: - An actual parameter passed to a VAR formal can not have a packed type, not even if the packing is ineffective or the bit count happens to equal that of the unpacked base type. - An actual parameter that is not at least byte-aligned can not be passed READONLY. - A scalar component can not straddle a word boundary. As a consequence, elements of an open array smaller than byte size are limited to 1, 2, or 4 bits. - The bit size of scalar elements of an open array must evenly divide 64. - An array with multiple open dimensions and smaller than byte-sized elements can not be partially subscripted. I.e., subscripts for all dimensions must be supplied simultaneously. - An array constructor must have at least byte alignment. - Elements of an array constructor whose length is not static must have a byte multiple size. This can arise only in complicated cases where the constructor has multiple open dimensions and all the inner arrays at some same open level are open and non-constructors. (follow that?) - The elements of a SUBARRAY must have at least byte alignment. - The actual parameter to ADR must have at least byte alignment. -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C29e63ebc4e4447cb08d408d8e96a9cea%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637515991399174110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AlVKzU3RySlXhaJimRzfZqPC7U9FtPSrayCJ5I8wGpI%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Wed Mar 17 19:11:36 2021 From: wagner at elegosoft.com (Olaf Wagner) Date: Wed, 17 Mar 2021 19:11:36 +0100 Subject: [M3devel] Mailing lists and website resources at elegosoft.com In-Reply-To: References: <14AE8009-E6BD-4AF1-ABC8-AC921DD22436@elegosoft.com> Message-ID: <734A2A24-E854-4A45-B382-2256B9AB3DC4@elegosoft.com> On 17 Mar 2021, at 16:29, Rodney M. Bates wrote: > On 3/16/21 2:59 PM, Olaf Wagner wrote: >> Hello everybody! >> >> There?s much traffic here in recent weeks, and not all of that is >> caused by >> meaningful messages from real users. In fact, the m3 mailing lists at >> elegosoft.com are used increasingly for spamming and we get more and >> more >> abuse warnings from our hosting provider. >> >> I also notice that we still host the now 20 years old website for cm3 >> which hasn?t changed for more than 10 years. > > On a few occasions, I have started working on some updates to the web > pages, but given up trying to reconcile the pages in the repository > with what is actually being served. I had presumed the right way was > to update the repository, then maybe move things to the server. > > The lists of file names in the two places have significant > differences, > and same-named files sometimes have significantly different content. > I have felt a bit timid about tampering with them. Maybe I need to > get bolder. If you volunteer to work on that, we can give you ssh access to the VM, I guess. It?s quite possible that lots of things have changed in the repositories that have not been transferred to the web server. There used to be some rsync scripts to update the service from checked-out repos. I haven?t run those for years though, so a diff might be called for in order not to overwrite meaningful changes that are accidentally not checked-in. >> Due to the COVID-19 pandemic our resources are currently rather >> limited. >> System administration would like to shutdown and get rid of as many >> legacy systems and liabilities as possible. At least we need to close >> the open subscription interfaces on the mailing lists as they are >> abused >> for spamming. Some counter measures have been implemented, but they >> are >> not as effective as needed. >> >> So what I?m asking is >> - which of the existing mailing lists and web resources are actually >> still >> needed > > I think the 3 lists and the web pages are all needed. M3-announce > gets used > only occasionally, but that's the idea of an announce list. We will keep the lists functional then, but stop the auto-subscription via smtp I think. Probably the whole mailing list system should be replaced by something more modern, but as I said we?re quite busy and haven?t enough human resources to work on these things. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From jayk123 at hotmail.com Thu Mar 18 01:11:17 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 00:11:17 +0000 Subject: [M3devel] order of reveal? Message-ID: I'm compiling RISCV64_LINUX, AMD64_LINUX, and ALPHA_LINUX. M3core. All Linux, 64bit, little endian. Everything is the same in the IR (sequence of CG calls and their parameters), except two things: 1. The order of reveal_opaque varies. That isn't terrible, in that, all the C backend does for this is issue comments. I'm not sure there is anything else imaginable for it to do. This is separate from declare_opaque. Maybe I should issue typedefs? But everything compiles asis so I just removed the comments. 2. But then more of similar, there are some init_int calls, whose values are opaque ids, and they vary in order too. This is actual output data. Not ignorable..assuming the data is referenced, which is not clear actually, 'cause I'd expect some correlating other differences, like to read the data (it isn't just the order that varies, what happens is the offsets of the initialization is the same but the values are jumbled, the resulting data is different). Any ideas? I'll try to figure it out. #1 is a common problem, #2 only occurs in one file in m3core ThreadPThread.m3. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 01:38:46 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 00:38:46 +0000 Subject: [M3devel] order of reveal? In-Reply-To: References: Message-ID: I think there is no required ordering of this data: C:\s\cm3\m3-sys\m3front\src\values\Revelation.m3 PROCEDURE GenList (s: Set; cnt: INTEGER; eq: BOOLEAN): INTEGER = VAR base := Module.Allocate (cnt * M3RT.RV_SIZE + Target.Integer.size, Target.Integer.align, TRUE, "revelations"); offs := base; iter : Iterator; l : List; BEGIN InitIterator (s, iter); WHILE Iterate (iter) DO l := iter.cur; IF (l.local) AND (l.ident.equal = eq) THEN CG.Init_intt (offs + M3RT.RV_lhs_id, Target.Integer.size, Type.GlobalUID (l.ident.lhs), TRUE); CG.Init_intt (offs + M3RT.RV_rhs_id, Target.Integer.size, Type.GlobalUID (l.ident.rhs), TRUE); INC (offs, M3RT.RV_SIZE); END; END; RETURN base; END GenList; and it can/should be sorted. And I guess it is outputing in a hashed order otherwise, which is something to avoid. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 18, 2021 12:11 AM To: m3devel at elegosoft.com Subject: [M3devel] order of reveal? I'm compiling RISCV64_LINUX, AMD64_LINUX, and ALPHA_LINUX. M3core. All Linux, 64bit, little endian. Everything is the same in the IR (sequence of CG calls and their parameters), except two things: 1. The order of reveal_opaque varies. That isn't terrible, in that, all the C backend does for this is issue comments. I'm not sure there is anything else imaginable for it to do. This is separate from declare_opaque. Maybe I should issue typedefs? But everything compiles asis so I just removed the comments. 2. But then more of similar, there are some init_int calls, whose values are opaque ids, and they vary in order too. This is actual output data. Not ignorable..assuming the data is referenced, which is not clear actually, 'cause I'd expect some correlating other differences, like to read the data (it isn't just the order that varies, what happens is the offsets of the initialization is the same but the values are jumbled, the resulting data is different). Any ideas? I'll try to figure it out. #1 is a common problem, #2 only occurs in one file in m3core ThreadPThread.m3. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 01:50:21 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 00:50:21 +0000 Subject: [M3devel] heap zero init? Message-ID: s.hash := NEW (HashTable, n_buckets); FOR i := 0 TO n_buckets - 1 DO s.hash[i] := NIL END; Doesn't Modula-3 zero initialized upon heap allocation, traced or not? So this loop is not needed? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 01:54:10 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 00:54:10 +0000 Subject: [M3devel] order of reveal? In-Reply-To: References: , Message-ID: Indeed, this hack stabilized the output: ff --git a/m3-sys/m3front/src/values/Revelation.m3 b/m3-sys/m3front/src/values/Revelation.m3 index e778149ad..504bedd23 100644 --- a/m3-sys/m3front/src/values/Revelation.m3 +++ b/m3-sys/m3front/src/values/Revelation.m3 @@ -157,8 +157,13 @@ PROCEDURE TypeCheck (s: Set) = save := Scanner.offset; (* allocate and initialize the hash table *) - <*ASSERT s.hash = NIL*> (* otherwise we've been checked twice!? *) - n_buckets := 2 * s.count; + <*ASSERT s.hash = NIL*> (* otherwise we've been checked twice!? *) + + (* TODO n_buckets is increased here + * to encourage output stability. + * Output should be sorted reliably independent of hash order. + *) + n_buckets := MAX(65536, 2 * s.count); s.hash := NEW (HashTable, n_buckets); FOR i := 0 TO n_buckets - 1 DO s.hash[i] := NIL END; It could be that some platform or another has more/fewer opaque types, perturbing the hashtable. i.e. the input is quite identical and the impact on output is disproportionate. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 18, 2021 12:38 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] order of reveal? I think there is no required ordering of this data: C:\s\cm3\m3-sys\m3front\src\values\Revelation.m3 PROCEDURE GenList (s: Set; cnt: INTEGER; eq: BOOLEAN): INTEGER = VAR base := Module.Allocate (cnt * M3RT.RV_SIZE + Target.Integer.size, Target.Integer.align, TRUE, "revelations"); offs := base; iter : Iterator; l : List; BEGIN InitIterator (s, iter); WHILE Iterate (iter) DO l := iter.cur; IF (l.local) AND (l.ident.equal = eq) THEN CG.Init_intt (offs + M3RT.RV_lhs_id, Target.Integer.size, Type.GlobalUID (l.ident.lhs), TRUE); CG.Init_intt (offs + M3RT.RV_rhs_id, Target.Integer.size, Type.GlobalUID (l.ident.rhs), TRUE); INC (offs, M3RT.RV_SIZE); END; END; RETURN base; END GenList; and it can/should be sorted. And I guess it is outputing in a hashed order otherwise, which is something to avoid. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 18, 2021 12:11 AM To: m3devel at elegosoft.com Subject: [M3devel] order of reveal? I'm compiling RISCV64_LINUX, AMD64_LINUX, and ALPHA_LINUX. M3core. All Linux, 64bit, little endian. Everything is the same in the IR (sequence of CG calls and their parameters), except two things: 1. The order of reveal_opaque varies. That isn't terrible, in that, all the C backend does for this is issue comments. I'm not sure there is anything else imaginable for it to do. This is separate from declare_opaque. Maybe I should issue typedefs? But everything compiles asis so I just removed the comments. 2. But then more of similar, there are some init_int calls, whose values are opaque ids, and they vary in order too. This is actual output data. Not ignorable..assuming the data is referenced, which is not clear actually, 'cause I'd expect some correlating other differences, like to read the data (it isn't just the order that varies, what happens is the offsets of the initialization is the same but the values are jumbled, the resulting data is different). Any ideas? I'll try to figure it out. #1 is a common problem, #2 only occurs in one file in m3core ThreadPThread.m3. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 05:34:51 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 04:34:51 +0000 Subject: [M3devel] order of reveal? In-Reply-To: References: , , Message-ID: There seems to be two problems. One is the hashorder output in GenList, for initialization. But there is also still the declare_opaque reordering. - Jay ________________________________ From: Jay K Sent: Thursday, March 18, 2021 12:54 AM To: m3devel at elegosoft.com Subject: Re: order of reveal? Indeed, this hack stabilized the output: ff --git a/m3-sys/m3front/src/values/Revelation.m3 b/m3-sys/m3front/src/values/Revelation.m3 index e778149ad..504bedd23 100644 --- a/m3-sys/m3front/src/values/Revelation.m3 +++ b/m3-sys/m3front/src/values/Revelation.m3 @@ -157,8 +157,13 @@ PROCEDURE TypeCheck (s: Set) = save := Scanner.offset; (* allocate and initialize the hash table *) - <*ASSERT s.hash = NIL*> (* otherwise we've been checked twice!? *) - n_buckets := 2 * s.count; + <*ASSERT s.hash = NIL*> (* otherwise we've been checked twice!? *) + + (* TODO n_buckets is increased here + * to encourage output stability. + * Output should be sorted reliably independent of hash order. + *) + n_buckets := MAX(65536, 2 * s.count); s.hash := NEW (HashTable, n_buckets); FOR i := 0 TO n_buckets - 1 DO s.hash[i] := NIL END; It could be that some platform or another has more/fewer opaque types, perturbing the hashtable. i.e. the input is quite identical and the impact on output is disproportionate. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 18, 2021 12:38 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] order of reveal? I think there is no required ordering of this data: C:\s\cm3\m3-sys\m3front\src\values\Revelation.m3 PROCEDURE GenList (s: Set; cnt: INTEGER; eq: BOOLEAN): INTEGER = VAR base := Module.Allocate (cnt * M3RT.RV_SIZE + Target.Integer.size, Target.Integer.align, TRUE, "revelations"); offs := base; iter : Iterator; l : List; BEGIN InitIterator (s, iter); WHILE Iterate (iter) DO l := iter.cur; IF (l.local) AND (l.ident.equal = eq) THEN CG.Init_intt (offs + M3RT.RV_lhs_id, Target.Integer.size, Type.GlobalUID (l.ident.lhs), TRUE); CG.Init_intt (offs + M3RT.RV_rhs_id, Target.Integer.size, Type.GlobalUID (l.ident.rhs), TRUE); INC (offs, M3RT.RV_SIZE); END; END; RETURN base; END GenList; and it can/should be sorted. And I guess it is outputing in a hashed order otherwise, which is something to avoid. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Thursday, March 18, 2021 12:11 AM To: m3devel at elegosoft.com Subject: [M3devel] order of reveal? I'm compiling RISCV64_LINUX, AMD64_LINUX, and ALPHA_LINUX. M3core. All Linux, 64bit, little endian. Everything is the same in the IR (sequence of CG calls and their parameters), except two things: 1. The order of reveal_opaque varies. That isn't terrible, in that, all the C backend does for this is issue comments. I'm not sure there is anything else imaginable for it to do. This is separate from declare_opaque. Maybe I should issue typedefs? But everything compiles asis so I just removed the comments. 2. But then more of similar, there are some init_int calls, whose values are opaque ids, and they vary in order too. This is actual output data. Not ignorable..assuming the data is referenced, which is not clear actually, 'cause I'd expect some correlating other differences, like to read the data (it isn't just the order that varies, what happens is the offsets of the initialization is the same but the values are jumbled, the resulting data is different). Any ideas? I'll try to figure it out. #1 is a common problem, #2 only occurs in one file in m3core ThreadPThread.m3. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 06:06:36 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 05:06:36 +0000 Subject: [M3devel] win32 vs. new packing rules? In-Reply-To: References: <520141615711185@mail.yandex.by> <80dace68-3cda-e1f0-0271-0bba13b1527e@lcwb.coop> , Message-ID: > Hope this helps. I don''t know if any of this has been lowered more > than ideal for producing C code, but I would kind of doubt it. I > think naive implementation of the operators in M3CG_Ops.i3 would work, > and this would just be to help understand what the front end is doing. You are right that from a C backend point of view, m3front handles these details and I don't have to care. It has worked for a long time/forever. Though m3front/m3back could have a different division of labor. m3back could handle pass/return structs by value..easy in C using idiomatic C89. I think Target.i3 already has provision for that. But my question was really from the point of view of C interop. e.g. the sockaddr changes. I would perhaps like to declare C parameters as VAR or READONLY for idiom and posisble safety in caller, but I need to use ADR for certainty. I need to write the same function prototypes in Modula-3 and C and be sure they match. Perhaps m3front could write suggested declarations to somewhere, esp. for functions marked <*external*>. Arguably ADR is better, because, you know, Rust has this rule that all calls to foreign functions are unsafe. Using ADR kinda makes that the rule here, partially. You know, we have that bug where small structs returned by value disagree between Modula-3 and C. WinUser.i3 works around this. Like the POINT or RECT structs. - Jay ________________________________ From: Rodney M. Bates Sent: Monday, March 15, 2021 6:54 PM To: Jay K ; vvm at tut.by ; m3devel ; rodney.m.bates at acm.org Subject: Re: [M3devel] win32 vs. new packing rules? On 3/14/21 3:18 PM, Jay K wrote: > Thank you. > > I understand it might be complicated. > > My naive idea is "run the logic twice both ways > and see if it the same". > > But "the logic" might devolve to "the entire compiler". > Not a good answer. > > Perhaps, and I am guessing, the logic could be: > > Run the logic just once, with the stated BITS for, and > notice if there is any padding inserted in between any > fields, or at the end of the record, and if no to both, > then it would be same either way? > I'm not sure I understand what you are saying, but From 2.2.5: A declaration of a packed type has the form: TYPE T = BITS n FOR Base where Base is a type and n is an integer-valued constant expression. The values of type T are the same as the values of type \verb|Base|, but variables of type T that occur in records, objects, or arrays will occupy exactly n bits and be packed adjacent to the ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ preceding field or element. ^^^^^^^^^^^^^^^^^^^^^^^^^^^ So if it's a field or element and packed, no padding is inserted. If it's not packed, the implementation is free to pad, and the only reasonable way is to pad to the alignment of the type, which is what cm3 does. No two-time run here. > In terms of documentation, I would like clarity > on ABI. Aka clarity on "what does look like in C". > i.e. clarify on pass by value vs. pointer. > e.g. what does "READONLY" mean? Perhaps it means > "by reference like VAR but readonly"?? > > I understand it might be complicated, because of > passing/returning records by semantic value, and sets. > > I understand there is some desire to separate semantics ("mutable reference or value") > from implementations details ("pointer or not"). > > But I cannot avoid wanting to know. > The abstract view is that VALUE mode in Modula-3, known generally as pass-by-value, means the formal is effectively a local variable of the callee, initialized during the call process to a copy of the actual. Modula-3 VAR, generally knows as pass-by-reference, means the formal is an alias of the actual. This is invariably implemented by passing a pointer to the actual. Compiler-generated code at the call site takes the address of the actual and passes the address. In the callee, every reference to the formal starts with a compiler-generated dereference of the pointer. This requires that the actual be something that is in memory, and Modula-3 requires that it be a designator, specifically defined to include only things that would certainly be so in any reasonable implementation. It excludes some that, in general, might not, e.g. result of a function call. In C, the only parameter mode is by value. So to have a parameter that the callee can alter and have the alterations seen by the caller, the source programmer has to manually create the same mechanism by explicitly coding the address-taking and the referencing. The actual will have to be an l-value, the rough equivalent of a designator. The C way creates some possible patterns that a language-provided by-reference mechanism, as in Modula-3 and lots of others, will prevent: A nil pointer, an uninitialized pointer, a misaligned pointer. Also, if the actual is also a formal of its containing function, or an array, which C almost always implicitly takes the address of, there can be some tricky counting of levels of indirection that need to be done by the programmer. As for READONLY, From 2.3.2: A READONLY formal is treated as a VAR formal if the actual is a designator and the type of the actual is the same as the type of the formal (or an array type that is assignable to the type of the formal); otherwise it is treated as a VALUE formal. When you get down to the machine code level (actually, the output of the front end), it gets more complicated. Since what READONLY does depends on properties of the actual, different calls can use different passing modes. The only reasonable way to implement this is to always pass by reference at the point of call, and have the call site code, when appropriate, make the copy into a local temporary, before passing its address. Then the callee can treat it the same, regardless. Notice that the conditions under which it's abstractly passed by reference correspond to when pass-by-reference would be legal. The separately-enforced prohibition against callee making assignments to the formal rules out some potential problems with this. It's a safe speculation that the original motive for READONLY was so a caller can have confidence his actual won't get stepped-on, while still enjoying the sometimes significant performance benefits of passing large objects without copying. The front end has quite a few variations on what to do at the call site, before actually passing, depending on the mode, the type, runtime assignability checks, and the conditions of READONLY. It is even sometimes possible to avoid making a copy, if the actual is something that is anonymous and could not be accessed any other way, for example, the result of a function call. These things are taken care of by the front end prior to the actual final parameter passing and needn't concern a back end, other than, obviously, translating all the IR operators. If you want to examine it, most will be found in Formal.m3, in several different passes: Check, CheckArg, PrepArg, and EmitArg and the things they call. There is one other Kink. For a formal of open array type, passed VALUE, the copy is made at the prolog of the callee, rather than at the call site. Presumably this is because the space needed is dynamic and the copy code is identical for all actuals, so needs not to be duplicated at every call. Hope this helps. I don''t know if any of this has been lowered more than ideal for producing C code, but I would kind of doubt it. I think naive implementation of the operators in M3CG_Ops.i3 would work, and this would just be to help understand what the front end is doing. > Sorry can you repeat though, what are the rules (I guess > you are going to write it up) and maybe the rationale? > More coming, but the main rule is that scalar values don't cross word boundaries, with various implications. > I am not sure "the language spec says so" is always the best > rationale. > > In particular, if you look at C (sorry, but this is a good idea, I will always believe), > you can see a few things: > > - the official standard is very vague about padding and alignment > > - In actual implementations, there is a lot of flexibility and a lot works. > In particular, you can align many things in many ways, and no matter what, > you can pass/return records/structs by value or pointer. Alignment/padding/packing > does not affect what you can pass. Well, at least in terms of entire records. > There are generally restrictions around member data/fields crossing alignment boundaries, > and you cannot..and maybe this is the crux of it..you cannot take the address of a bitfield. > > So I *gather* what Modula-3 is doing is trying to capture that last thing. > Inability to address bitfields. You can always access them directly, i.e. Record.Field or Array[Subscript]. But not pass by reference, where you would need a pointer to a bit, not just a byte. > You might quibble, that sometimes bitfields end up reasonable to address. > But C just bars it entirely. > > Is that what is going on here? Teasing out some VAR-ability of bitfields, if they > end up nicely aligned? > > But then, there should be no impact on the overall RECORD, right? > > Or RECORDs might themselves kinda look like bitfields? > > Can I say this: > TYPE OneBit = BITS 1 FOR RECORD > bool: BITS 1 FOR BOOLEAN; > END; > > TWO TwoBits = BITS 2 FOR RECORD > a,b: BITS 1 FOR OneBit; > END; > > Some redundancy there? > > TYPE Byte = ARRAY [0..3] OF TwoBits; > > <*ASSERT BYTESIZE(OneBit) = 1*> > <*ASSERT BYTESIZE(TwoBits) = 1*> > <*ASSERT BYTESIZE(Byte) = 1*> > > b: Byte; > > b[1].b.bool > > ? > > Is that what these rules enable? Yes. > Which admittedly you cannot do in C. > > And I even had a use case for a few years ago. > Encoding fonts in constant data. > C bitfields aren't great for that. > > In C structs and array elements have BITSIZE MOD 8 == 0. > (On some ARM targets there is apparently a notion of worse.) > > Except one time though the byte-size granularity > of C structs and array elements has seemed fine. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Rodney M. Bates > *Sent:* Sunday, March 14, 2021 7:51 PM > *To:* Jay K ; vvm at tut.by ; m3devel > *Subject:* Re: [M3devel] win32 vs. new packing rules? > Yes, removing the BITS 64 FOR is the right fix here. And yes, it > doesn't change anything the code does. > > There is a subtle language and implementation issue here: > > Since a formal is never a record field or array element, BITS ... FOR > will never affect its size or alignment. These will be selected by > the compiler as usual. > > But a corresponding actual of type BITS n FOR.., even with the same n, > could be a field or element and have an alignment of only one bit. > And different formals in different calls could differ from each otner. > This would mean pass by reference would have to be via a variable bit > pointer, and all the uses in the procedure would have to handle it. > This is not a reasonable implementation. > > It could be accepted in specific cases, this case in point, if the > actual that had the same alignment as the natural alignment as the > formal. But excessively conditional rules like this in a language is > bad language design and a start down the slippery slope that leads to > programming languages with 300, 400, or 500 pages of definition that > almost nobody understands. > > So when I encountered problems with this in the existing compiler, I > fixed it with the unconditional rule that a perameter passed VAR > cannot be BITS FOR. No such restriction on VAR or READONLY. > > This is within Modula-3's allowing of restrictions on packed types. I > have started writing a statement of all the implementation > restrictions regarding them, which belongs in implementation > documentation. > > > On 3/14/21 3:53 AM, Jay K wrote: >> Looks like we can just remove the BITS FOR statement. >> Which makes the new rules seem, kinda pointless at least in this case. >> BITS FOR didn't actually change anything, but validity. In this case. >> >> - Jay >> >> > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Thu Mar 18 06:43:18 2021 From: jayk123 at hotmail.com (Jay K) Date: Thu, 18 Mar 2021 05:43:18 +0000 Subject: [M3devel] Passing Text to Win32 wide/W/unicode functions? Message-ID: We have this common pattern: fname := M3toC.SharedTtoS(p); handle := WinBase.CreateFile(fname, ...); . . . M3toC.FreeSharedS(p, fname); Presumably we are supposed to now "rewrite" them all, like: fname := M3toC.SharedTtoW(p); handle := WinBase.CreateFileW(fname, ...); . . . M3toC.FreeSharedW(p, fname); ? or, um, maybe not quite. For the shared-not-writable case, and the string has all bytes < 0x80, this causes, um, an unnecessary heap allocation and copy. So you kinda want something like: fname := M3toC.SharedTtoX(p); (* X is for unknown or varying *) IF fname.wide # NIL THEN handle := WinBase.CreateFileW(fname.wide, ..); (* or "unicode"? or just "W"? *) ELSE handle := WinBase.CreateFile(fname.narrow, ..); (* or "ascii"? or just "A"? *) END; but: 1. of course this is cumbersome 2. the next layer down is going to pay the same cost anyway, so might as well go with my first suggestion? That is, CreateFileA calls CreateFileW anyway. Someone has to pay the cost and it is basically the same either way. Though on some systems the heap is avoided via a thread local. Our free function might defer to GC, might not. CreateFileA of course does not. And you know, I'm assuming not Windows 95 here. Perhaps WinBase.CreateFile could be some kind of wrapper that does this work. Perhaps SharedTtoS could silently change to this new dynamic thing. Or just continue to ignore it for a while long? Or maybe UTF8? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Thu Mar 18 19:39:48 2021 From: vvm at tut.by (vvm at tut.by) Date: Thu, 18 Mar 2021 21:39:48 +0300 Subject: [M3devel] Passing Text to Win32 wide/W/unicode functions? In-Reply-To: References: Message-ID: <471231616091954@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 03:17:40 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 02:17:40 +0000 Subject: [M3devel] quake mystery solved Message-ID: I finally figured out "the problem" with quake. Most of the quake using code is "object oriented". By that, I mean passing around state. Not using globals. "this", "self", etc. i.e. the local variable/parameter mach: QMachine.T or s.mach, etc. But, the exception, is MxConfig. MxConfig provides roughly two functionalities: 1. find the path to cm3.cfg remember it in a global, under a lock 2. query that cm3.cfg for definitions This is done via global MxConfig.mach: QMachine.T. Functionality 1 is used by cm3. Functionality 2 appears to be mostly legacy, for cm3ide. However functionality 2 is used in one place by cm3. So when you see "mach", sometimes it is a properly scoped local, that flows through the code. Sometimes it is a global, that does not share the initialization of the local. The initialization of the local includes several things, including the processing of -Dfoo=bar on the command line. So when you say cm3 -DTARGET=whatever and it prints "default to native target other"..huh?.. that is because, there is one use of functionality 2 that snuck in. Where cm3.cfg is evaluated in a separate "clean" context from the command line. That is the unicode work: Unicode_WIDECHAR_opt := MxConfig.Get("Unicode_WIDECHAR"); Basically, MxConfix.Get should never be used and this line confuses thing. Or it should be changed to take in a QMachine.T parameter. Not use a global. It is fairly side-effect free, but it prints about the default target, and it is why scripts have to set ROOT/CM3_ROOT in environment, the command line does not work. So also we were evaluating cm3.cfg twice. Yuck. Again, fairly side-effect free. Defines functions to compile and linke, but does not run them. I will fix shortly. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 03:28:51 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 02:28:51 +0000 Subject: [M3devel] quake mystery solved In-Reply-To: References: Message-ID: All of these should probably change: Searching for 'mxconfig.get'... C:\s\cm3\m3-sys\cm3\src\Makefile.m3(634): Msg.Out (" target: ", MxConfig.Get("TARGET"), Wr.EOL); C:\s\cm3\m3-sys\cm3\src\Utils.m3(344): Unicode_WIDECHAR_opt := MxConfig.Get("Unicode_WIDECHAR"); C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(35): build_dir := MxConfig.Get ("BUILD_DIR"); C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(36): system_root := MxConfig.Get ("PKG_USE"); (* note: system_root is the public package root *) C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(37): doc_root := MxConfig.Get ("DOC_INSTALL"); C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(38): initial_browser := MxConfig.Get ("INITIAL_CM3_IDE_BROWSER"); C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(39): initial_editor := MxConfig.Get ("INITIAL_CM3_IDE_EDITOR"); C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(51): WITH p = MxConfig.Get ("TARGET") DO C:\s\cm3\m3-sys\cm3ide\src\misc\Default.m3(60): WITH p = MxConfig.Get ("INSTALL_ROOT") DO C:\s\cm3\m3-sys\llvm\llvm3.6.1\src\Main.m3(260): machine := MxConfig.Get ("TARGET"); C:\s\cm3\m3-sys\llvm\llvm5\src\Main.m3(260): machine := MxConfig.Get ("TARGET"); C:\s\cm3\m3-sys\llvm\llvm9\src\Main.m3(348): machine := MxConfig.Get ("TARGET"); C:\s\cm3\m3-sys\m3cgcat\src\Main.m3(148): machine := MxConfig.Get ("TARGET"); C:\s\cm3\m3-sys\m3tools\src\M3Parse.m3(110): sys := MxConfig.Get ("TARGET"); None of them handle -DTARGET=xx on the command line. That does not necessarily matter, i.e. for cases that lack such command line processing. And I suppose the environment variable M3CONFIG is just as well. They do handle that. - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Friday, March 19, 2021 2:17 AM To: m3devel at elegosoft.com Subject: [M3devel] quake mystery solved I finally figured out "the problem" with quake. Most of the quake using code is "object oriented". By that, I mean passing around state. Not using globals. "this", "self", etc. i.e. the local variable/parameter mach: QMachine.T or s.mach, etc. But, the exception, is MxConfig. MxConfig provides roughly two functionalities: 1. find the path to cm3.cfg remember it in a global, under a lock 2. query that cm3.cfg for definitions This is done via global MxConfig.mach: QMachine.T. Functionality 1 is used by cm3. Functionality 2 appears to be mostly legacy, for cm3ide. However functionality 2 is used in one place by cm3. So when you see "mach", sometimes it is a properly scoped local, that flows through the code. Sometimes it is a global, that does not share the initialization of the local. The initialization of the local includes several things, including the processing of -Dfoo=bar on the command line. So when you say cm3 -DTARGET=whatever and it prints "default to native target other"..huh?.. that is because, there is one use of functionality 2 that snuck in. Where cm3.cfg is evaluated in a separate "clean" context from the command line. That is the unicode work: Unicode_WIDECHAR_opt := MxConfig.Get("Unicode_WIDECHAR"); Basically, MxConfix.Get should never be used and this line confuses thing. Or it should be changed to take in a QMachine.T parameter. Not use a global. It is fairly side-effect free, but it prints about the default target, and it is why scripts have to set ROOT/CM3_ROOT in environment, the command line does not work. So also we were evaluating cm3.cfg twice. Yuck. Again, fairly side-effect free. Defines functions to compile and linke, but does not run them. I will fix shortly. Related aside: I am not sure these are good places to look for cm3.cfg: (* try the current directory *) IF TryConfig (".", Filename) THEN RETURN END; (* try the immediate source directory *) IF TryConfig ("src", Filename) THEN RETURN END; (* try the sibling source directory *) IF TryConfig ("..", "src", Filename) THEN RETURN END; Can we remove those? Other parts of PROCEDURE FindConfig () I don't love. In practise it is usually the "next to current executable" one. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 05:29:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 04:29:00 +0000 Subject: [M3devel] m3front vs. middle vs. back vs. cm3 Message-ID: If you find yourself in cm3 setting globals in m3front/back/linker, put them in m3middle instead. m3middle is where you can share between m3front and m3back. This way you can have just one global. I'm not generally aware of m3linker, granted. (Regarding widechar 16 vs. 32 configuration... but it applies more generally.) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 08:32:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 07:32:00 +0000 Subject: [M3devel] integer with constrained set of values? Message-ID: I'd really like a type like this: TYPE WordSizeType = [32, 64]; PROCEDURE SetWordSize(w: WordSizes); SetWordSize(1); // compile or runtime error w: WordSizeType bytesInWord : INTEGER := w / 8; (* can be mixed with integer, ok *) Is this feasable? I gave up and used INTEGER. I know there are enums but they do not fit great here I think. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 11:39:52 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 10:39:52 +0000 Subject: [M3devel] supporting old systems and tools Message-ID: I have through time introduced a lot of workarounds for old systems -- old C compilers, bootstrapping from old cm3, running on old OSes. I think I/we should adjust a bit and drop old stuff. Perhaps: - Drop support for pre-Windows Vista. - Drop support for pre-Visual C++ 2008 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) I did try building with Visual C++ 2.0 tonight and ran into IPv6 being a problem. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Fri Mar 19 12:14:23 2021 From: vvm at tut.by (vvm at tut.by) Date: Fri, 19 Mar 2021 14:14:23 +0300 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: References: Message-ID: <688611616152261@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 13:03:52 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 12:03:52 +0000 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: <688611616152261@mail.yandex.by> References: , <688611616152261@mail.yandex.by> Message-ID: Vista is "much better" than Server 2003, because of declspec(thread) and condition variables. But ok for a bit longer I suppose. I don't think Visual C++ compiler runs on it either, but you can still build your binaries elsewhere. I've never run ReactOS but perhaps it just works? Even Windows 7 is going out of support, e.g. from Python. Many other projects only go down to Windows 7, like Chrome and CoreCLR. - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 19, 2021 11:14 AM To: Jay K ; m3devel at elegosoft.com Subject: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! Please, if it's possible, continue support of Windows Server 2003 ( a-ka "NT52"). And don't drop support for ReactOs. Thank! Best regards, Victor Miasnikov 19.03.2021, 13:40, "Jay K" : I have through time introduced a lot of workarounds for old systems -- old C compilers, bootstrapping from old cm3, running on old OSes. I think I/we should adjust a bit and drop old stuff. Perhaps: - Drop support for pre-Windows Vista. - Drop support for pre-Visual C++ 2008 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) I did try building with Visual C++ 2.0 tonight and ran into IPv6 being a problem. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Fri Mar 19 13:40:22 2021 From: vvm at tut.by (vvm at tut.by) Date: Fri, 19 Mar 2021 15:40:22 +0300 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: References: , <688611616152261@mail.yandex.by> Message-ID: <154161616157070@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 18:01:50 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 17:01:50 +0000 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: <154161616157070@mail.yandex.by> References: , <688611616152261@mail.yandex.by> , <154161616157070@mail.yandex.by> Message-ID: You don't need 32bit Server to run 32bit stuff and I don't understand "MS ISA Server 2006 with OSPF routing." Very little software under active development runs pre-Vista or pre-Windows7. - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 19, 2021 12:40 PM To: Jay K ; m3devel at elegosoft.com Subject: Re: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! ReactOs is "free Windows". This project doesn't related with Microsoft corporation. Today ReactOs is comptible with Windows Server 2003. Yes, ReactOs works. -- Mini FAQ. Q: What features old Windows can't be replaced? A: "Vista Server" Windows Server 2008 "R0" x86 is last 32bit Windows with server options. Active Directory for example. It haven't another advantage. Main advantage of Windows Server 2003 is possiblity to running MS ISA Server 2006 with OSPF routing. M.a-ge Windows Server 2008 R2 is p-ty to r-ing MS TMG Server and IAG. Best regards, Victor Miasnikov 19.03.2021, 15:04, "Jay K" : Vista is "much better" than Server 2003, because of declspec(thread) and condition variables. But ok for a bit longer I suppose. I don't think Visual C++ compiler runs on it either, but you can still build your binaries elsewhere. I've never run ReactOS but perhaps it just works? Even Windows 7 is going out of support, e.g. from Python. Many other projects only go down to Windows 7, like Chrome and CoreCLR. - Jay ________________________________ From: vvm at tut.by > Sent: Friday, March 19, 2021 11:14 AM To: Jay K >; m3devel at elegosoft.com > Subject: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! Please, if it's possible, continue support of Windows Server 2003 ( a-ka "NT52"). And don't drop support for ReactOs. Thank! Best regards, Victor Miasnikov 19.03.2021, 13:40, "Jay K" >: I have through time introduced a lot of workarounds for old systems -- old C compilers, bootstrapping from old cm3, running on old OSes. I think I/we should adjust a bit and drop old stuff. Perhaps: - Drop support for pre-Windows Vista. - Drop support for pre-Visual C++ 2008 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) I did try building with Visual C++ 2.0 tonight and ran into IPv6 being a problem. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Fri Mar 19 20:24:56 2021 From: vvm at tut.by (vvm at tut.by) Date: Fri, 19 Mar 2021 22:24:56 +0300 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: References: , <688611616152261@mail.yandex.by> , <154161616157070@mail.yandex.by> Message-ID: <754431616181378@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Fri Mar 19 20:47:36 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 19:47:36 +0000 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: <754431616181378@mail.yandex.by> References: , <688611616152261@mail.yandex.by> , <154161616157070@mail.yandex.by> , <754431616181378@mail.yandex.by> Message-ID: Ah, and you have to run Modula3 binaries on these same very old systems? These were very good releases but it is probably long past time to move on. Anyway I have other stuff to do so maybe leave this alone "for now". We could leave the code with compile time or runtime conditionals, perhaps, though they have their problems. - Jay ________________________________ From: vvm at tut.by Sent: Friday, March 19, 2021 7:24 PM To: Jay K ; m3devel at elegosoft.com Subject: Re: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! It's firewall and web-proxy. It's contain driver. I.e. must be installed exactly on Win Srv 2003. OSFP is routing protocol. OSPF doesn't work on Win Srv 2008 R2 with another MS firewall ( TMG Server), but works on 2003+ISA. All this about "real world". Youre theory simply isn't true, becase there exist many even more exotic software that MS firewalls. Best regards, Victor Miasnikov 19.03.2021, 20:02, "Jay K" : You don't need 32bit Server to run 32bit stuff and I don't understand "MS ISA Server 2006 with OSPF routing." Very little software under active development runs pre-Vista or pre-Windows7. - Jay ________________________________ From: vvm at tut.by > Sent: Friday, March 19, 2021 12:40 PM To: Jay K >; m3devel at elegosoft.com > Subject: Re: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! ReactOs is "free Windows". This project doesn't related with Microsoft corporation. Today ReactOs is comptible with Windows Server 2003. Yes, ReactOs works. -- Mini FAQ. Q: What features old Windows can't be replaced? A: "Vista Server" Windows Server 2008 "R0" x86 is last 32bit Windows with server options. Active Directory for example. It haven't another advantage. Main advantage of Windows Server 2003 is possiblity to running MS ISA Server 2006 with OSPF routing. M.a-ge Windows Server 2008 R2 is p-ty to r-ing MS TMG Server and IAG. Best regards, Victor Miasnikov 19.03.2021, 15:04, "Jay K" >: Vista is "much better" than Server 2003, because of declspec(thread) and condition variables. But ok for a bit longer I suppose. I don't think Visual C++ compiler runs on it either, but you can still build your binaries elsewhere. I've never run ReactOS but perhaps it just works? Even Windows 7 is going out of support, e.g. from Python. Many other projects only go down to Windows 7, like Chrome and CoreCLR. - Jay ________________________________ From: vvm at tut.by > Sent: Friday, March 19, 2021 11:14 AM To: Jay K >; m3devel at elegosoft.com > Subject: Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: [M3devel] supporting old systems and tools Hi! Please, if it's possible, continue support of Windows Server 2003 ( a-ka "NT52"). And don't drop support for ReactOs. Thank! Best regards, Victor Miasnikov 19.03.2021, 13:40, "Jay K" >: I have through time introduced a lot of workarounds for old systems -- old C compilers, bootstrapping from old cm3, running on old OSes. I think I/we should adjust a bit and drop old stuff. Perhaps: - Drop support for pre-Windows Vista. - Drop support for pre-Visual C++ 2008 (https://www.microsoft.com/en-us/download/details.aspx?id=44266) I did try building with Visual C++ 2.0 tonight and ran into IPv6 being a problem. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 00:57:22 2021 From: jayk123 at hotmail.com (Jay K) Date: Fri, 19 Mar 2021 23:57:22 +0000 Subject: [M3devel] module segments vs. debugging Message-ID: When there is code like: // global VAR traceInit := FALSE PROCEDURE TraceModule(s: TEXT; m: RT0.ModulePtr) = BEGIN IF NOT traceInit THEN RETURN END; <== RTIO.PutText(s); OutModuleName(m); RTIO.PutText("\r\n"); RTIO.Flush(); END TraceModule; we get code like: 0:000:x86> u cm3!RTLinker__TraceModule cm3!RTLinker__TraceModule [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 519]: 00ab19d0 55 push ebp 00ab19d1 8bec mov ebp,esp 00ab19d3 51 push ecx 00ab19d4 8d45fc lea eax,[ebp-4] 00ab19d7 8945fc mov dword ptr [ebp-4],eax 00ab19da 0fb60d34f2ba00 movzx ecx,byte ptr [cm3!M_RTLinker_L_48+0x34 (00baf234)] <== 00ab19e1 85c9 test ecx,ecx 00ab19e3 7402 je cm3!RTLinker__TraceModule+0x17 (00ab19e7) but in any other programming environment, this would be more like: 00ab19da 0fb60d34f2ba00 movzx ecx,byte ptr [cm3!RTLinker_traceInit] <== or 00ab19da 0fb60d34f2ba00 movzx ecx,byte ptr [cm3!traceInit] <== i.e. better symbols and debugging. (maybe m3gdb knows about this? But m3gdb is missing on many platforms: NT, Darwin, HPUX). The thing is, cm3 puts all of a module's globals in two segments, readonly and writable, and then access them via offsets from the start, like a struct. There might be reasons for this. - To the extent that the variables are traced references, maybe the offsets are recorded somewhere so garbage collector can move and fixup? This could be replaced with pointers, albeit that'd be more/double the data. - It gives a sort of fast portable linking semantic. You get more all-or-nothing type linking behavior. But that also means the linker cannot remove unused data. So you get larger binaries. - It obviates the need to discern file-level static from extern, as the number of symbols is reduced. They are basically all static. I propose that at least to the extent that the variables are not traced references, they be moved into their own "segments". That extern vs. static be managed and informed to the backends. Presumably this is representable to them. It could be for C backend only if there is a problem, or really, so I don't have to investigate. And do people know answers to the implied questions: Why was it done this way? Do we use the offsets otherwise? I should be able to figure that out. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 05:28:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 04:28:00 +0000 Subject: [M3devel] wintrestle breaks Message-ID: wintrestle breaks "..\src\MGPaintOp.m3", line 60: No value specified for record constructor field (2.6.8). (xrgb) "..\src\MGPaintOp.m3", line 97: No value specified for record constructor field (2.6.8). (xrgb) 2 errors encountered There are more. One I can fix. Another is due to the packing changes. Someone needs to try their hand at Github actions or something. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 05:41:22 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 04:41:22 +0000 Subject: [M3devel] wintrestle breaks In-Reply-To: References: Message-ID: I could for XRGB.T for Win32 vs. Posix or Win32 vs. other but I'm reluctant to rule out that X clients can easily run on Windows. It's just network I/O. ? But we do have: if equal (OS_TYPE, "WIN32") include_dir ("winvbt") else include_dir ("xvbt") end so i guess that is the way. Or initialize to zero or something? - Jay ________________________________ From: M3devel on behalf of Jay K Sent: Saturday, March 20, 2021 4:28 AM To: m3devel Subject: [M3devel] wintrestle breaks wintrestle breaks "..\src\MGPaintOp.m3", line 60: No value specified for record constructor field (2.6.8). (xrgb) "..\src\MGPaintOp.m3", line 97: No value specified for record constructor field (2.6.8). (xrgb) 2 errors encountered There are more. One I can fix. Another is due to the packing changes. Someone needs to try their hand at Github actions or something. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sat Mar 20 07:54:21 2021 From: vvm at tut.by (vvm at tut.by) Date: Sat, 20 Mar 2021 09:54:21 +0300 Subject: [M3devel] Continue support of Windows Server 2003 ( a-ka "NT52") and ReactOs Re: supporting old systems and tools In-Reply-To: References: , <688611616152261@mail.yandex.by> , <154161616157070@mail.yandex.by> , <754431616181378@mail.yandex.by> Message-ID: <738941616222117@mail.yandex.by> An HTML attachment was scrubbed... URL: From vvm at tut.by Sat Mar 20 07:57:37 2021 From: vvm at tut.by (vvm at tut.by) Date: Sat, 20 Mar 2021 09:57:37 +0300 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: Message-ID: <703131616223358@mail.yandex.by> An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Sat Mar 20 08:16:26 2021 From: peter.mckinna at gmail.com (Peter McKinna) Date: Sat, 20 Mar 2021 18:16:26 +1100 Subject: [M3devel] module segments vs. debugging In-Reply-To: <703131616223358@mail.yandex.by> References: <703131616223358@mail.yandex.by> Message-ID: It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio as well as all the normal optimisations.. The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) Regards Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 08:26:34 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 07:26:34 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by>, Message-ID: This is a function of the IR not any limit of the C backend. The C backend produces full native debuginfo, CodeView, Dwarf, also. LLVM won't be able to do better with the same IR. That is the problem, the IR. m3front is what needs work, to split up the segments into individual variables. I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. - Jay ________________________________ From: Peter McKinna Sent: Saturday, March 20, 2021 7:16 AM To: vvm at tut.by Cc: Jay K ; m3devel Subject: Re: [M3devel] module segments vs. debugging It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio as well as all the normal optimisations.. The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) Regards Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sat Mar 20 09:09:27 2021 From: vvm at tut.by (vvm at tut.by) Date: Sat, 20 Mar 2021 11:09:27 +0300 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> Message-ID: <757611616227684@mail.yandex.by> An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat Mar 20 16:43:43 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 20 Mar 2021 10:43:43 -0500 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: Message-ID: On 3/19/21 6:57 PM, Jay K wrote: > When there is code like: > > // global > VAR > ? traceInit ? := FALSE > > > PROCEDURE TraceModule(s: TEXT; m: RT0.ModulePtr) = > ? BEGIN > ? ? IF NOT traceInit THEN RETURN END; <== > ? ? RTIO.PutText(s); > ? ? OutModuleName(m); > ? ? RTIO.PutText("\r\n"); > ? ? RTIO.Flush(); > ? END TraceModule; > > > we get code like: > > 0:000:x86> u cm3!RTLinker__TraceModule > cm3!RTLinker__TraceModule [C:\s\cm3\m3-libs\m3core\src\runtime\common\RTLinker.m3 @ 519]: > 00ab19d0 55 ? ? ? ? ? ? ?push ? ?ebp > 00ab19d1 8bec ? ? ? ? ? ?mov ? ? ebp,esp > 00ab19d3 51 ? ? ? ? ? ? ?push ? ?ecx > 00ab19d4 8d45fc ? ? ? ? ?lea ? ? eax,[ebp-4] > 00ab19d7 8945fc ? ? ? ? ?mov ? ? dword ptr [ebp-4],eax > 00ab19da 0fb60d34f2ba00 ?movzx ? ecx,byte ptr [cm3!M_RTLinker_L_48+0x34 (00baf234)] <== > 00ab19e1 85c9 ? ? ? ? ? ?test ? ?ecx,ecx > 00ab19e3 7402 ? ? ? ? ? ?je ? ? ?cm3!RTLinker__TraceModule+0x17 (00ab19e7) > > > but in any other programming environment, this would be more like: > > 00ab19da 0fb60d34f2ba00 ?movzx ? ecx,byte ptr [cm3!RTLinker_traceInit] <== > or > 00ab19da 0fb60d34f2ba00 ?movzx ? ecx,byte ptr [cm3!traceInit] <== > > i.e. better symbols and debugging. > > (maybe m3gdb knows about this? But m3gdb is missing on many platforms: NT, Darwin, HPUX). > > The thing is, cm3 puts all of a module's globals in two segments, readonly and writable, > and then access them via offsets from the start, like a struct. > > There might be reasons for this. > > ?- To the extent that the variables are traced references, maybe the offsets are recorded somewhere > ? ?so garbage collector can move and fixup? > ? ?This could be replaced with pointers, albeit that'd be more/double the data. > > ?- It gives a sort of fast portable linking semantic. You get more all-or-nothing type linking behavior. > ? ?But that also means the linker cannot remove unused data. So you get larger binaries. > > ?- It obviates the need to discern file-level static from extern, as the number of symbols > ? ?is reduced. They are basically all static. > > I propose that at least to the extent that the variables are not traced references, > they be moved into their own "segments". That extern vs. static be managed and informed to the backends. > Presumably this is representable to them. > > It could be for C backend only if there is a problem, or really, so I don't have to investigate. > > And do people know answers to the implied questions: > Why was it done this way? > Do we use the offsets otherwise? I should be able to figure that out. I don't know why it was done that way, but debug info coming out of the front end in IR reflects it and at least m3cc pass it on through. m3gdb knows about it and handles it. Does the C backend remove the record around the readonly and readwrite globals? > > ? - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 20 17:03:44 2021 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 20 Mar 2021 11:03:44 -0500 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> Message-ID: <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop> On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > ?- Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > *Cc:* Jay K ; m3devel > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From jayk123 at hotmail.com Sat Mar 20 17:30:20 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 16:30:20 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop> References: <703131616223358@mail.yandex.by> , <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop> Message-ID: Hm, so there is declare_global, but the globals are all then stored in a struct with unnamed fields, and they are accessed through that. So yeah the information is on the side, but the codegen is uncooperative. I would have to "rewrite" all those struct+offset accesses to access a global instead. I suppose I could, but why is the IR like that? Why not access globals? Why stuff them all into large structs? That is surely not the IR any other language would chose? Structs are similarly bad. They should be field accesses, not base + offset. Well, in this case, you can start to understand, that m3front lowers things quite far, which is good and bad. The C backend works much better than you suggest. It really works well and the result is quite debuggable. It is more debuggable than Darwin and NT386 at least, that lack m3gdb. The Modula-3 stuff just needs debugger extensions, like if you really want to view all the kinds of TEXTS. But some of them are trivial today, "Text8Short" or such. Just like C++ debuggers need, like to view std::string, which also views horribly by default in debuggers. You know, structs need "interpretation" sometimes. This is inevitable in all systems. C is no exception. Pointers may or may not be profitable to dereference by default. etc. - Jay ________________________________ From: Rodney M. Bates Sent: Saturday, March 20, 2021 4:03 PM To: Jay K ; Peter McKinna ; vvm at tut.by Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > *Cc:* Jay K ; m3devel > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5e206ef3ff7b45686c8608d8ebb9c882%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518530448347398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ucE8LY%2BgjWujIbDPAb2vTkGDRpsMvhSfLhRDW8fOWWQ%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 18:54:38 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 17:54:38 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> , <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop>, Message-ID: sorry to repeat itself but I didn't reply to everything.... > insurmountable obstacle is going through a standard C compiler afterwards. It works already. > No C compiler will never handle any of the Modula-3-specific stuff. Like what? Sets, enums, and texts? Enums we should be able to quite well. In C++ you can specify the base type of an enum and qualify the names. So while currently I just have defines and ints, we could have: Modula3: type Color = {Orange, Blue, Yellow"; c := Color.Yellow C++ enum class Color : uint8 { Orange, Blue, Yellow }; Color c:= Color::Yellow and debugger would show it well. Presently, admitted, I generate: typedef uint8 color #define color_Orange 0 uint8 c = color_orange or maybe 0 -- IR probably needs raising here too Texts would require debugger extensions, just like std::string. That is, user written add-ins. Which people often write. Sets could probably be a struct with single bit bitfields, maybe unioned with an integer or array of integers for codegen convenience. i.e. so I wouldn't have to reraise the IR. I didn't try. For objects you'd want to fish out the typecode and/or vtable to decide what the type is, downcast, and show fields. Windbg is already terrible with C++ here, it doesn't do like that, but should. Something might be possible here but I've learned to live with the already poor C++ support. It is really just a specific case of special casing certain structs though, just that objects are common. > unless you do a lot of modification to the C compiler itself. Of course that is not the idea. Exactly the opposite. I think we are agreeing that the present IR is hostile to standard debuggers. The representation of globals at least is poor. I mean, what if you had no fork of gdb to maintain, then what? Then m3cc would be similar, maybe worse, than m3c, right? Really the C backend already works, works well, and is as debuggable or more debuggable than other systems. And I believe crucially, is very portable without having an unending maintenance cost. There isn't the forking or keeping up with gcc/llvm, and it automatically supports new systems. ARM64 and RISCV64 required no backend work at all. It is repeatedly said that it can't or won't work, or won't work well, but it does work and does work well. Yes compilation is noticably slower, but tolerable. I restored multithreaded backend for NT. I am going to try "unit at a time" to improve that. I can make the text smaller. Yes we'll never get GC root-in-register info. No backends have that anyway. But it still works quite well. - Jay ________________________________ From: Jay K Sent: Saturday, March 20, 2021 4:30 PM To: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging Hm, so there is declare_global, but the globals are all then stored in a struct with unnamed fields, and they are accessed through that. So yeah the information is on the side, but the codegen is uncooperative. I would have to "rewrite" all those struct+offset accesses to access a global instead. I suppose I could, but why is the IR like that? Why not access globals? Why stuff them all into large structs? That is surely not the IR any other language would chose? Structs are similarly bad. They should be field accesses, not base + offset. Well, in this case, you can start to understand, that m3front lowers things quite far, which is good and bad. The C backend works much better than you suggest. It really works well and the result is quite debuggable. It is more debuggable than Darwin and NT386 at least, that lack m3gdb. The Modula-3 stuff just needs debugger extensions, like if you really want to view all the kinds of TEXTS. But some of them are trivial today, "Text8Short" or such. Just like C++ debuggers need, like to view std::string, which also views horribly by default in debuggers. You know, structs need "interpretation" sometimes. This is inevitable in all systems. C is no exception. Pointers may or may not be profitable to dereference by default. etc. - Jay ________________________________ From: Rodney M. Bates Sent: Saturday, March 20, 2021 4:03 PM To: Jay K ; Peter McKinna ; vvm at tut.by Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > *Cc:* Jay K ; m3devel > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5e206ef3ff7b45686c8608d8ebb9c882%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518530448347398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ucE8LY%2BgjWujIbDPAb2vTkGDRpsMvhSfLhRDW8fOWWQ%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 19:31:18 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 18:31:18 +0000 Subject: [M3devel] refactoring cm3 into just one package? Message-ID: I was thinking.. sysutils m3scanner m3middle m3objfile m3linker m3back m3quake m3front cm3 Might as well we combine these into just one or two packages? I understand the need to break software up into coherent reusable pieces, and I have no intention to change the graph among these, but is this more than necessary splitting? Plus, you know, linkers can split up static libs. I say two because: Move it all into a lib, and then link executables to it. So some of it can be reused outside of cm3, some is. We'd have cm3lib and cm3, or cm3 and cm3exe. m3tools cm3ide etc. would import the library. The executable would just be one line that calls library. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From antony.hosking at anu.edu.au Sat Mar 20 22:53:28 2021 From: antony.hosking at anu.edu.au (Tony Hosking) Date: Sat, 20 Mar 2021 21:53:28 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> , <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop>, , Message-ID: I have been working lately with LLVM stack maps and there is provision for GC roots in registers. It may be the case that its backend does not currently generate them, but the stack map encoding allows it. It also has support for global (static) GC roots. Indeed, having m3front generate straight to LLVM IR instead of m3cg would be ideal in allowing a whole range of symbolic naming to flow through. That is a big project though. -Tony Get Outlook for iOS ________________________________ From: M3devel on behalf of Jay K Sent: Sunday, March 21, 2021 4:54:38 AM To: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging sorry to repeat itself but I didn't reply to everything.... > insurmountable obstacle is going through a standard C compiler afterwards. It works already. > No C compiler will never handle any of the Modula-3-specific stuff. Like what? Sets, enums, and texts? Enums we should be able to quite well. In C++ you can specify the base type of an enum and qualify the names. So while currently I just have defines and ints, we could have: Modula3: type Color = {Orange, Blue, Yellow"; c := Color.Yellow C++ enum class Color : uint8 { Orange, Blue, Yellow }; Color c:= Color::Yellow and debugger would show it well. Presently, admitted, I generate: typedef uint8 color #define color_Orange 0 uint8 c = color_orange or maybe 0 -- IR probably needs raising here too Texts would require debugger extensions, just like std::string. That is, user written add-ins. Which people often write. Sets could probably be a struct with single bit bitfields, maybe unioned with an integer or array of integers for codegen convenience. i.e. so I wouldn't have to reraise the IR. I didn't try. For objects you'd want to fish out the typecode and/or vtable to decide what the type is, downcast, and show fields. Windbg is already terrible with C++ here, it doesn't do like that, but should. Something might be possible here but I've learned to live with the already poor C++ support. It is really just a specific case of special casing certain structs though, just that objects are common. > unless you do a lot of modification to the C compiler itself. Of course that is not the idea. Exactly the opposite. I think we are agreeing that the present IR is hostile to standard debuggers. The representation of globals at least is poor. I mean, what if you had no fork of gdb to maintain, then what? Then m3cc would be similar, maybe worse, than m3c, right? Really the C backend already works, works well, and is as debuggable or more debuggable than other systems. And I believe crucially, is very portable without having an unending maintenance cost. There isn't the forking or keeping up with gcc/llvm, and it automatically supports new systems. ARM64 and RISCV64 required no backend work at all. It is repeatedly said that it can't or won't work, or won't work well, but it does work and does work well. Yes compilation is noticably slower, but tolerable. I restored multithreaded backend for NT. I am going to try "unit at a time" to improve that. I can make the text smaller. Yes we'll never get GC root-in-register info. No backends have that anyway. But it still works quite well. - Jay ________________________________ From: Jay K Sent: Saturday, March 20, 2021 4:30 PM To: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging Hm, so there is declare_global, but the globals are all then stored in a struct with unnamed fields, and they are accessed through that. So yeah the information is on the side, but the codegen is uncooperative. I would have to "rewrite" all those struct+offset accesses to access a global instead. I suppose I could, but why is the IR like that? Why not access globals? Why stuff them all into large structs? That is surely not the IR any other language would chose? Structs are similarly bad. They should be field accesses, not base + offset. Well, in this case, you can start to understand, that m3front lowers things quite far, which is good and bad. The C backend works much better than you suggest. It really works well and the result is quite debuggable. It is more debuggable than Darwin and NT386 at least, that lack m3gdb. The Modula-3 stuff just needs debugger extensions, like if you really want to view all the kinds of TEXTS. But some of them are trivial today, "Text8Short" or such. Just like C++ debuggers need, like to view std::string, which also views horribly by default in debuggers. You know, structs need "interpretation" sometimes. This is inevitable in all systems. C is no exception. Pointers may or may not be profitable to dereference by default. etc. - Jay ________________________________ From: Rodney M. Bates Sent: Saturday, March 20, 2021 4:03 PM To: Jay K ; Peter McKinna ; vvm at tut.by Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > *Cc:* Jay K ; m3devel > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5e206ef3ff7b45686c8608d8ebb9c882%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518530448347398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ucE8LY%2BgjWujIbDPAb2vTkGDRpsMvhSfLhRDW8fOWWQ%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 23:36:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 22:36:00 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> , <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop>, , , Message-ID: Good you are still here. ? > Indeed, having m3front generate straight to LLVM IR in instead of m3cg Do you think, maybe, we can have it both ways? I have some vague ideas. Target.BackendDoesX := TRUE; Target.BackendDoesY := TRUE; For example "field access". m3front: if Target.BackendDoesFieldAccess THEN m3cg.access_field(...) ELSE existing code or maybe m3front: cg.access_field(base var, field offset, field name, type) and backend will use either the offset or the name? I realize this is vague. i.e. something incremental that does not break the existing backends? Or at worst, sends the existing backends more parameters, that they can ignore? Or even gradually notice. Granted also, going to LLVM does not preclude updating integrated backend, or maybe deleting it, or writing an LLVM to M3CG translator. Or LLVM to C. That exists. Another perspective is like, field layout would be its own layer, split out from m3front, and optionally between m3front and m3back. Something like that. C backend pipeline could omit that layer producing slightly less ugly. Exception handling is another example. Optionally m3front continues to lower to setjmp/longjmp, or optionally leave to the next level down. You can view things as a gradual lowering, passing through multiple optional steps. If m3back can lower exception handling, then let it see it. If not, pass it through the portable inefficient EH to setjmp pass first. Then you need a common in-memory format. Which I *sort of* "invented". In the C backend, I first build up all the m3cg calls into a direct representation. Then I loop over it a few times. I don't think I transform it though. And I certainly didn't design it. It just the calls and their parameters. I just needed to know stuff in varying orders, multiple visits resolves that. But I think you can imagine transforms like I said, such as lowering EH or field access. Of course first we need higher level representation of those. - Jay ________________________________ From: Tony Hosking Sent: Saturday, March 20, 2021 9:53 PM To: Jay K ; Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging I have been working lately with LLVM stack maps and there is provision for GC roots in registers. It may be the case that its backend does not currently generate them, but the stack map encoding allows it. It also has support for global (static) GC roots. Indeed, having m3front generate straight to LLVM IR instead of m3cg would be ideal in allowing a whole range of symbolic naming to flow through. That is a big project though. ?Tony Get Outlook for iOS ________________________________ From: M3devel on behalf of Jay K Sent: Sunday, March 21, 2021 4:54:38 AM To: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging sorry to repeat itself but I didn't reply to everything.... > insurmountable obstacle is going through a standard C compiler afterwards. It works already. > No C compiler will never handle any of the Modula-3-specific stuff. Like what? Sets, enums, and texts? Enums we should be able to quite well. In C++ you can specify the base type of an enum and qualify the names. So while currently I just have defines and ints, we could have: Modula3: type Color = {Orange, Blue, Yellow"; c := Color.Yellow C++ enum class Color : uint8 { Orange, Blue, Yellow }; Color c:= Color::Yellow and debugger would show it well. Presently, admitted, I generate: typedef uint8 color #define color_Orange 0 uint8 c = color_orange or maybe 0 -- IR probably needs raising here too Texts would require debugger extensions, just like std::string. That is, user written add-ins. Which people often write. Sets could probably be a struct with single bit bitfields, maybe unioned with an integer or array of integers for codegen convenience. i.e. so I wouldn't have to reraise the IR. I didn't try. For objects you'd want to fish out the typecode and/or vtable to decide what the type is, downcast, and show fields. Windbg is already terrible with C++ here, it doesn't do like that, but should. Something might be possible here but I've learned to live with the already poor C++ support. It is really just a specific case of special casing certain structs though, just that objects are common. > unless you do a lot of modification to the C compiler itself. Of course that is not the idea. Exactly the opposite. I think we are agreeing that the present IR is hostile to standard debuggers. The representation of globals at least is poor. I mean, what if you had no fork of gdb to maintain, then what? Then m3cc would be similar, maybe worse, than m3c, right? Really the C backend already works, works well, and is as debuggable or more debuggable than other systems. And I believe crucially, is very portable without having an unending maintenance cost. There isn't the forking or keeping up with gcc/llvm, and it automatically supports new systems. ARM64 and RISCV64 required no backend work at all. It is repeatedly said that it can't or won't work, or won't work well, but it does work and does work well. Yes compilation is noticably slower, but tolerable. I restored multithreaded backend for NT. I am going to try "unit at a time" to improve that. I can make the text smaller. Yes we'll never get GC root-in-register info. No backends have that anyway. But it still works quite well. - Jay ________________________________ From: Jay K Sent: Saturday, March 20, 2021 4:30 PM To: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging Hm, so there is declare_global, but the globals are all then stored in a struct with unnamed fields, and they are accessed through that. So yeah the information is on the side, but the codegen is uncooperative. I would have to "rewrite" all those struct+offset accesses to access a global instead. I suppose I could, but why is the IR like that? Why not access globals? Why stuff them all into large structs? That is surely not the IR any other language would chose? Structs are similarly bad. They should be field accesses, not base + offset. Well, in this case, you can start to understand, that m3front lowers things quite far, which is good and bad. The C backend works much better than you suggest. It really works well and the result is quite debuggable. It is more debuggable than Darwin and NT386 at least, that lack m3gdb. The Modula-3 stuff just needs debugger extensions, like if you really want to view all the kinds of TEXTS. But some of them are trivial today, "Text8Short" or such. Just like C++ debuggers need, like to view std::string, which also views horribly by default in debuggers. You know, structs need "interpretation" sometimes. This is inevitable in all systems. C is no exception. Pointers may or may not be profitable to dereference by default. etc. - Jay ________________________________ From: Rodney M. Bates Sent: Saturday, March 20, 2021 4:03 PM To: Jay K ; Peter McKinna ; vvm at tut.by Cc: m3devel Subject: Re: [M3devel] module segments vs. debugging On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > *Cc:* Jay K ; m3devel > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5e206ef3ff7b45686c8608d8ebb9c882%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518530448347398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ucE8LY%2BgjWujIbDPAb2vTkGDRpsMvhSfLhRDW8fOWWQ%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 23:38:53 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 22:38:53 +0000 Subject: [M3devel] module segments vs. debugging In-Reply-To: References: <703131616223358@mail.yandex.by> <19f427cb-3fd2-f215-6261-d2b34354b99f@lcwb.coop> , Message-ID: I think your best best, actually, is use the ability for debuggers to call into the code they are running. There multiple approaches and tradeoffs, granted. Calling into the program is easy, but doesn't work in all cases, like kernel debugging. But it is also the most portable solution. I personally have never written a windbg extension and I rarely call into running program since I use kernel debuggers so I kinda trained not to. But it is a popular technique. And then you just need to be sure compiler/linker do not throw out the seemingly unused functions. - Jay ________________________________ From: Mika Nystrom Sent: Saturday, March 20, 2021 7:05 PM To: Jay K Cc: Peter McKinna ; vvm at tut.by ; rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] module segments vs. debugging Any chance of writing these debugger extensions themselves in modula-3? On Sat, Mar 20, 2021, 10:55 AM Jay K > wrote: sorry to repeat itself but I didn't reply to everything.... > insurmountable obstacle is going through a standard C compiler afterwards. It works already. > No C compiler will never handle any of the Modula-3-specific stuff. Like what? Sets, enums, and texts? Enums we should be able to quite well. In C++ you can specify the base type of an enum and qualify the names. So while currently I just have defines and ints, we could have: Modula3: type Color = {Orange, Blue, Yellow"; c := Color.Yellow C++ enum class Color : uint8 { Orange, Blue, Yellow }; Color c:= Color::Yellow and debugger would show it well. Presently, admitted, I generate: typedef uint8 color #define color_Orange 0 uint8 c = color_orange or maybe 0 -- IR probably needs raising here too Texts would require debugger extensions, just like std::string. That is, user written add-ins. Which people often write. Sets could probably be a struct with single bit bitfields, maybe unioned with an integer or array of integers for codegen convenience. i.e. so I wouldn't have to reraise the IR. I didn't try. For objects you'd want to fish out the typecode and/or vtable to decide what the type is, downcast, and show fields. Windbg is already terrible with C++ here, it doesn't do like that, but should. Something might be possible here but I've learned to live with the already poor C++ support. It is really just a specific case of special casing certain structs though, just that objects are common. > unless you do a lot of modification to the C compiler itself. Of course that is not the idea. Exactly the opposite. I think we are agreeing that the present IR is hostile to standard debuggers. The representation of globals at least is poor. I mean, what if you had no fork of gdb to maintain, then what? Then m3cc would be similar, maybe worse, than m3c, right? Really the C backend already works, works well, and is as debuggable or more debuggable than other systems. And I believe crucially, is very portable without having an unending maintenance cost. There isn't the forking or keeping up with gcc/llvm, and it automatically supports new systems. ARM64 and RISCV64 required no backend work at all. It is repeatedly said that it can't or won't work, or won't work well, but it does work and does work well. Yes compilation is noticably slower, but tolerable. I restored multithreaded backend for NT. I am going to try "unit at a time" to improve that. I can make the text smaller. Yes we'll never get GC root-in-register info. No backends have that anyway. But it still works quite well. - Jay ________________________________ From: Jay K > Sent: Saturday, March 20, 2021 4:30 PM To: Peter McKinna >; vvm at tut.by >; rodney.m.bates at acm.org > Cc: m3devel > Subject: Re: [M3devel] module segments vs. debugging Hm, so there is declare_global, but the globals are all then stored in a struct with unnamed fields, and they are accessed through that. So yeah the information is on the side, but the codegen is uncooperative. I would have to "rewrite" all those struct+offset accesses to access a global instead. I suppose I could, but why is the IR like that? Why not access globals? Why stuff them all into large structs? That is surely not the IR any other language would chose? Structs are similarly bad. They should be field accesses, not base + offset. Well, in this case, you can start to understand, that m3front lowers things quite far, which is good and bad. The C backend works much better than you suggest. It really works well and the result is quite debuggable. It is more debuggable than Darwin and NT386 at least, that lack m3gdb. The Modula-3 stuff just needs debugger extensions, like if you really want to view all the kinds of TEXTS. But some of them are trivial today, "Text8Short" or such. Just like C++ debuggers need, like to view std::string, which also views horribly by default in debuggers. You know, structs need "interpretation" sometimes. This is inevitable in all systems. C is no exception. Pointers may or may not be profitable to dereference by default. etc. - Jay ________________________________ From: Rodney M. Bates > Sent: Saturday, March 20, 2021 4:03 PM To: Jay K >; Peter McKinna >; vvm at tut.by > Cc: m3devel > Subject: Re: [M3devel] module segments vs. debugging On 3/20/21 2:26 AM, Jay K wrote: > This is a function of the IR not any limit of the C backend. Not true. The IR has lots of debug info specific to both the Modula-3 language and the CM3 implementation of it. It needs even more, but that is tangential. m3cc passes this all through. So does the integrated back end. I think the llvm backend does some of it, but llvm itself, despite what its documentation seems clearly to say, fails to handle but a small subset of dwarf, apparently just sufficient for C and C++. While a C backend itself is not limited in what it could do, the insurmountable obstacle is going through a standard C compiler afterwards. No C compiler will never handle any of the Modula-3-specific stuff. That is, unless you do a lot of modification to the C compiler itself. But that would just be a reimplementation of m3cc, though possibly in significantly different ways. It would still suffer the problem that m3cc does of remaining static or constantly being remerged with new versions of the base C compiler. That is, unless you write a whole new C compiler from scratch, in which case, it would be a reimplementation of the integrated backend, though possibly in significantly different ways. > The C backend produces full native debuginfo, CodeView, Dwarf, also. > LLVM won't be able to do better with the same IR. > That is the problem, the IR. No, the problem is the C compiler. > m3front is what needs work, to split up the segments into individual variables. > Although I have no great love for the way globals are handled, it presents no obstacle to debugging, as m3cc/m3gdb show. Such a redesign would solve only a small subset of the problems getting debug info through a C compiler. > I am curious if we can replace m3cc and possibly m3c with LLVM, across the board. > LLVM might be a possibility, but llvm itself would have to be modified, thus again the same 'ol same 'ol of remerging with newer versions of LLVM. > - Jay > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* Peter McKinna > > *Sent:* Saturday, March 20, 2021 7:16 AM > *To:* vvm at tut.by > > *Cc:* Jay K >; m3devel > > *Subject:* Re: [M3devel] module segments vs. debugging > It's not that well known but the llvm backend runs on windows. At least the 32 bit version did. > I tested it with llvm 9 last year and it generates codeview debug output for use with visual studio > as well as all the normal optimisations.. > > The codeview debug output isn't as mature as the dwarf stuff but its getting there slowly > plus, of course, our generation of what is debuggable is limited to what llvm provides. (no support for sets for example) > > Regards Peter > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7C5e206ef3ff7b45686c8608d8ebb9c882%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518530448347398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ucE8LY%2BgjWujIbDPAb2vTkGDRpsMvhSfLhRDW8fOWWQ%3D&reserved=0 > -- Rodney Bates rodney.m.bates at acm.org _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sat Mar 20 23:52:14 2021 From: jayk123 at hotmail.com (Jay K) Date: Sat, 20 Mar 2021 22:52:14 +0000 Subject: [M3devel] cm3 build order Message-ID: So we have this thing that confuses people, when you first rebuild the compiler, you do not rebuild m3core and libm3. I noticed in old notes the notion that this is, really, "optional". Sometimes building m3core + libm3 first is ok, or even required. So I thought I'd explain my understanding. Why the confusing ordering? 1. cm3 outputs calls to m3core. So cm3 is tightly coupled with m3core. For example NEW calls the allocator. See RT0.i3. But this part rarely changes. The last change was probably Tony adding the barriers. 2. m3core is written in Modula-3. Sometimes the language changes and newer compiler is needed to compile m3core. Again this is rare and the last change was LONGINT addition. 3. It used to be the case, used to, that the compiler and either m3core or libm3, had a long list (enum) of target systems. They had to be in sync. Any time you added a target, you had to use that build ordering. You could not even just add to the end a new system, because of the type hashing. But I removed both copies of the enum long ago. (Basically, aside, as I ported, I kept judging the steps to port not really justified and gradually removed them.) On the flip side, very simple, sometimes you might add something to m3core or libm3 that you want to call from cm3. In that case, you do need to rebuild m3core/libm3 first. So I think, given the rarity of the first two things, we should not get *too* hung up on avoiding the last, and maybe consider the more mainstream process is to build m3core/libm3 first. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.mckinna at gmail.com Sun Mar 21 00:12:20 2021 From: peter.mckinna at gmail.com (Peter McKinna) Date: Sun, 21 Mar 2021 10:12:20 +1100 Subject: [M3devel] cm3 build order In-Reply-To: References: Message-ID: I'm not sure it's relevant, but I thought the restriction was partly meant to prevent you from building a broken cm3 caused by adding a bug to the libraries. I know, in the past, I have mucked around with m3core and built a cm3 that was useless and I had to find an old copy somewhere or download the boot distribution and start over. Maybe, before we start the rebuild we make a backup copy of cm3 and cm3cg and whatever other backend you like? On Sun, 21 Mar 2021 at 09:52, Jay K wrote: > So we have this thing that confuses people, > when you first rebuild the compiler, you do not rebuild m3core and libm3. > > I noticed in old notes the notion that this is, really, "optional". > Sometimes building m3core + libm3 first is ok, or even required. > > So I thought I'd explain my understanding. > Why the confusing ordering? > > 1. cm3 outputs calls to m3core. > So cm3 is tightly coupled with m3core. > For example NEW calls the allocator. > See RT0.i3. > But this part rarely changes. > The last change was probably Tony adding the barriers. > > 2. m3core is written in Modula-3. > Sometimes the language changes and newer compiler > is needed to compile m3core. > Again this is rare and the last change was LONGINT addition. > > 3. It used to be the case, used to, that the compiler and either > m3core or libm3, had a long list (enum) of target systems. They had > to be in sync. > Any time you added a target, you had to use that build ordering. > You could not even just add to the end a new system, because of the type > hashing. > But I removed both copies of the enum long ago. > (Basically, aside, as I ported, I kept judging the steps to port > not really justified and gradually removed them.) > > On the flip side, very simple, sometimes you might add > something to m3core or libm3 that you want to call from cm3. > In that case, you do need to rebuild m3core/libm3 first. > > So I think, given the rarity of the first two things, > we should not get *too* hung up on avoiding the last, > and maybe consider the more mainstream process > is to build m3core/libm3 first. > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 21 01:55:56 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 00:55:56 +0000 Subject: [M3devel] cm3 build order In-Reply-To: References: , Message-ID: Yes to both, but you can also put the bugs in the other parts. I like how make-dist.py works in this regard: it never really updates in place. And then, after that, just recursive copy. Upgrade.py should maybe take a similar approach. It should be (is?) possible to also output or ship directly into a new parallel structure. Make-dist.py kinda cheats, it does that by copying cm3 into the new structure and running it from there. Another approach would be run the unshipped compiler. Don?t ship it until it works. Make a copy in its output directory so it can link itself, approximately. - Jay ________________________________ From: Peter McKinna Sent: Saturday, March 20, 2021 4:12:20 PM To: Jay K Cc: m3devel Subject: Re: [M3devel] cm3 build order I'm not sure it's relevant, but I thought the restriction was partly meant to prevent you from building a broken cm3 caused by adding a bug to the libraries. I know, in the past, I have mucked around with m3core and built a cm3 that was useless and I had to find an old copy somewhere or download the boot distribution and start over. Maybe, before we start the rebuild we make a backup copy of cm3 and cm3cg and whatever other backend you like? On Sun, 21 Mar 2021 at 09:52, Jay K > wrote: So we have this thing that confuses people, when you first rebuild the compiler, you do not rebuild m3core and libm3. I noticed in old notes the notion that this is, really, "optional". Sometimes building m3core + libm3 first is ok, or even required. So I thought I'd explain my understanding. Why the confusing ordering? 1. cm3 outputs calls to m3core. So cm3 is tightly coupled with m3core. For example NEW calls the allocator. See RT0.i3. But this part rarely changes. The last change was probably Tony adding the barriers. 2. m3core is written in Modula-3. Sometimes the language changes and newer compiler is needed to compile m3core. Again this is rare and the last change was LONGINT addition. 3. It used to be the case, used to, that the compiler and either m3core or libm3, had a long list (enum) of target systems. They had to be in sync. Any time you added a target, you had to use that build ordering. You could not even just add to the end a new system, because of the type hashing. But I removed both copies of the enum long ago. (Basically, aside, as I ported, I kept judging the steps to port not really justified and gradually removed them.) On the flip side, very simple, sometimes you might add something to m3core or libm3 that you want to call from cm3. In that case, you do need to rebuild m3core/libm3 first. So I think, given the rarity of the first two things, we should not get *too* hung up on avoiding the last, and maybe consider the more mainstream process is to build m3core/libm3 first. - Jay _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Mar 21 02:42:09 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sat, 20 Mar 2021 21:42:09 -0400 Subject: [M3devel] cm3 build order In-Reply-To: References: Message-ID: <20210321014209.uie77fd3yj4dfhhg@topoi.pooq.com> On Sun, Mar 21, 2021 at 12:55:56AM +0000, Jay K wrote: > Yes to both, but you can also put the bugs in the other parts. > > I like how make-dist.py works in this regard: it never really updates in place. > And then, after that, just recursive copy. > Upgrade.py should maybe take a similar approach. > > It should be (is?) possible to also output or ship directly into a new parallel structure. The way the Cambridge Algol 68C compiler was bootstrapped from one version to another: (1) Compile the new source code with the old compiler. (2) Compile the new source code using the result of (1). (3) Compile the new source code using the result of (2). The compiler is considered OK if the result of (2) is bit-for-bit identical with the resut of (2). If not identical, it says that old compiler produces code that behaved sifferently from the code generated by the new one for the same source code, so there must be a bug. They did keep old versions around somewhere, just in case. And they did sometimes discover they had to go back several versions to deal with a problem. -- hendrik > > Make-dist.py kinda cheats, it does that by copying cm3 into the new structure and running it from there. > > Another approach would be run the unshipped compiler. Don?t ship it until it works. Make a copy in its output directory so it can link itself, approximately. > > - Jay > ________________________________ > From: Peter McKinna > Sent: Saturday, March 20, 2021 4:12:20 PM > To: Jay K > Cc: m3devel > Subject: Re: [M3devel] cm3 build order > > I'm not sure it's relevant, but I thought the restriction > was partly meant to prevent you from building a broken cm3 > caused by adding a bug to the libraries. > > I know, in the past, I have mucked around with m3core and built > a cm3 that was useless and I had to find an old copy somewhere or > download the boot distribution and start over. > > Maybe, before we start the rebuild we make a backup copy of cm3 > and cm3cg and whatever other backend you like? > > On Sun, 21 Mar 2021 at 09:52, Jay K > wrote: > So we have this thing that confuses people, > when you first rebuild the compiler, you do not rebuild m3core and libm3. > > I noticed in old notes the notion that this is, really, "optional". > Sometimes building m3core + libm3 first is ok, or even required. > > So I thought I'd explain my understanding. > Why the confusing ordering? > > 1. cm3 outputs calls to m3core. > So cm3 is tightly coupled with m3core. > For example NEW calls the allocator. > See RT0.i3. > But this part rarely changes. > The last change was probably Tony adding the barriers. > > 2. m3core is written in Modula-3. > Sometimes the language changes and newer compiler > is needed to compile m3core. > Again this is rare and the last change was LONGINT addition. > > 3. It used to be the case, used to, that the compiler and either > m3core or libm3, had a long list (enum) of target systems. They had > to be in sync. > Any time you added a target, you had to use that build ordering. > You could not even just add to the end a new system, because of the type hashing. > But I removed both copies of the enum long ago. > (Basically, aside, as I ported, I kept judging the steps to port > not really justified and gradually removed them.) > > On the flip side, very simple, sometimes you might add > something to m3core or libm3 that you want to call from cm3. > In that case, you do need to rebuild m3core/libm3 first. > > So I think, given the rarity of the first two things, > we should not get *too* hung up on avoiding the last, > and maybe consider the more mainstream process > is to build m3core/libm3 first. > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From jayk123 at hotmail.com Sun Mar 21 04:25:00 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 03:25:00 +0000 Subject: [M3devel] cm3 build order In-Reply-To: <20210321014209.uie77fd3yj4dfhhg@topoi.pooq.com> References: , <20210321014209.uie77fd3yj4dfhhg@topoi.pooq.com> Message-ID: We do the "build multiple times and it works" thing but not (yet) the memcmp check. gcc's build has the build twice and memcmp but sometimes e.g. debug info gets in the way. There is this thing that the inputs are sometimes more than it might seem. For example the full path to the source. Maybe the username. And maybe the time. (int main() { printf("%s\n", TIMESTAMP);} compiles differently every time. also the operating systems are now deliberately random: int main() { printf("%p\", main); } runs differently every time. So you have to be sure not to do those things. They look obvious but there are subtle variations also. Deterministic builds are a "new" thing, or, well, um, recently rediscovered. It is a shame. They should have been the thing all along. But timestamps, version numbers, guids, creep in. I was involved with a project where "patches" where produced by comparing updated builds with older builds. So I got to discover surprises. Things like, add a function declaration to a header. Change nothing else. Do not implement or call that function, no other change. And you'd get different results. The input did in fact change, but not in a way that should matter. I'm fighting this now in fact. In cm3 building AMD64_LINUX vs. ALPHA_LINUX is almost identical input and almost identical output, but something gets reordered. Possibly those very strings get into hash tables and then an output occurs in hash order (for example: m3front: Output revelations in sorted order instead of hash order. ? modula3/cm3 at 505aaae (github.com) does that matter? It was deterministic before, but slight perturbations in input had disproportionate impact on output. Now those same input perturbations have no impact on output.) Debian started doing deterministic builds a few years ago, but not all packages. Visual C++ only recently got a non-default option to be deterministic. In my view it should be the only mode. cm3 is guilty of this itself. It likes to put the current time in itself. And developers do have that fascination with ever increasing version numbers. The time is the best, because it always increases. Here: cm3: Move version from C object to C source, using ? modula3/cm3 at ae2770f (github.com) cm3 never ever builds the same twice. Deliberately. It isn't my doing. I preserved the old behavior. See ReproducibleBuilds - Debian Wiki Reproducible, right, I always forget that name. The key is that others can reproduce the same build even. That the source can be verified to be the actual source. An introduction to deterministic builds with C/C++ (conan.io) Getting to Deterministic Builds on Windows - Nikhil's blog (nikhilism.com) Windows, C++: investigate '/experimental:deterministic' compiler flag ? Issue #9466 ? bazelbuild/bazel (github.com) etc. - Jay ________________________________ From: M3devel on behalf of Hendrik Boom Sent: Sunday, March 21, 2021 1:42 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] cm3 build order On Sun, Mar 21, 2021 at 12:55:56AM +0000, Jay K wrote: > Yes to both, but you can also put the bugs in the other parts. > > I like how make-dist.py works in this regard: it never really updates in place. > And then, after that, just recursive copy. > Upgrade.py should maybe take a similar approach. > > It should be (is?) possible to also output or ship directly into a new parallel structure. The way the Cambridge Algol 68C compiler was bootstrapped from one version to another: (1) Compile the new source code with the old compiler. (2) Compile the new source code using the result of (1). (3) Compile the new source code using the result of (2). The compiler is considered OK if the result of (2) is bit-for-bit identical with the resut of (2). If not identical, it says that old compiler produces code that behaved sifferently from the code generated by the new one for the same source code, so there must be a bug. They did keep old versions around somewhere, just in case. And they did sometimes discover they had to go back several versions to deal with a problem. -- hendrik > > Make-dist.py kinda cheats, it does that by copying cm3 into the new structure and running it from there. > > Another approach would be run the unshipped compiler. Don?t ship it until it works. Make a copy in its output directory so it can link itself, approximately. > > - Jay > ________________________________ > From: Peter McKinna > Sent: Saturday, March 20, 2021 4:12:20 PM > To: Jay K > Cc: m3devel > Subject: Re: [M3devel] cm3 build order > > I'm not sure it's relevant, but I thought the restriction > was partly meant to prevent you from building a broken cm3 > caused by adding a bug to the libraries. > > I know, in the past, I have mucked around with m3core and built > a cm3 that was useless and I had to find an old copy somewhere or > download the boot distribution and start over. > > Maybe, before we start the rebuild we make a backup copy of cm3 > and cm3cg and whatever other backend you like? > > On Sun, 21 Mar 2021 at 09:52, Jay K > wrote: > So we have this thing that confuses people, > when you first rebuild the compiler, you do not rebuild m3core and libm3. > > I noticed in old notes the notion that this is, really, "optional". > Sometimes building m3core + libm3 first is ok, or even required. > > So I thought I'd explain my understanding. > Why the confusing ordering? > > 1. cm3 outputs calls to m3core. > So cm3 is tightly coupled with m3core. > For example NEW calls the allocator. > See RT0.i3. > But this part rarely changes. > The last change was probably Tony adding the barriers. > > 2. m3core is written in Modula-3. > Sometimes the language changes and newer compiler > is needed to compile m3core. > Again this is rare and the last change was LONGINT addition. > > 3. It used to be the case, used to, that the compiler and either > m3core or libm3, had a long list (enum) of target systems. They had > to be in sync. > Any time you added a target, you had to use that build ordering. > You could not even just add to the end a new system, because of the type hashing. > But I removed both copies of the enum long ago. > (Basically, aside, as I ported, I kept judging the steps to port > not really justified and gradually removed them.) > > On the flip side, very simple, sometimes you might add > something to m3core or libm3 that you want to call from cm3. > In that case, you do need to rebuild m3core/libm3 first. > > So I think, given the rarity of the first two things, > we should not get *too* hung up on avoiding the last, > and maybe consider the more mainstream process > is to build m3core/libm3 first. > > - Jay > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Ced1669a8eada4668abdf08d8ec0a980a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518877544610580%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N2aVbz%2F0Uo7gCT7SzR6WFe8Bn1LAAKJoQ1oXBmow8JY%3D&reserved=0 > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Ced1669a8eada4668abdf08d8ec0a980a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518877544630572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=u1BRxXXSJ719CX7mf7CCd%2BVpJIcjAwy35ImNFNlZIAk%3D&reserved=0 _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fm3lists.elegosoft.com%2Fmailman%2Flistinfo%2Fm3devel&data=04%7C01%7C%7Ced1669a8eada4668abdf08d8ec0a980a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637518877544630572%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=u1BRxXXSJ719CX7mf7CCd%2BVpJIcjAwy35ImNFNlZIAk%3D&reserved=0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 21 07:43:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 06:43:06 +0000 Subject: [M3devel] IR to defined records or not? Message-ID: C:\s\cm3\m3-ui\ui\src\trestle\InstalledVBT.m3 PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); (* line 41 *) j.trueChild := ch; EVAL j.init(child) END InitChild; The init call has some default parameters. It is actually: C:\s\cm3\m3-ui\ui\src\split\HighlightVBT.i3(31): init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T The C backend produces: #line 41 "../src/trestle/InstalledVBT.m3" (*(ADDRESS*)(&L_72_L_73))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_78_L_79)( ((ADDRESS)(grandChild_L_52)), ((ADDRESS)(ch_L_55)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(1000)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1024)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((ADDRESS)(&L_76_L_77)))))); See the third and fourth parameters are defaulted, to the addresses of globals. T79A5AEBB and T23E73F69 are record types. Being passed by address, I understand. Question is, should the IR declare those types? It used to. With the packedVar work, it does not. So this no longer compiles. The C backend issues an error for use of undefined types. But, I understand, it is only a pointer, so the type definition is arguably optional and optimize it away. Though more generally, a higher level IR would hopefully read: M_InstalledVBT.PaintOp->TransparentSwap or import_PaintOp->TransparentSwap passed by value and C compiler would need the type definition. It doesn't work that way currently of course. I suspect this change but don't fully understand it. ommit 0d6ffe0e98909412f820f3842c05f9e3d2e8e574 Author: Rodney Bates Date: Thu Oct 24 16:46:45 2019 -0500 Eliminate redundant record constructor temporary. A static record constructor was building two redundant copies of the record value: A: one built at RT into a stack temp (not used) and B: one built at CT into the static global area (used). Something like, call to Type.Compile is missing? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 21 07:50:58 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 21 Mar 2021 09:50:58 +0300 Subject: [M3devel] refactoring cm3 into just one package? In-Reply-To: References: Message-ID: <801831616309048@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 21 08:34:08 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 07:34:08 +0000 Subject: [M3devel] refactoring cm3 into just one package? In-Reply-To: <801831616309048@mail.yandex.by> References: , <801831616309048@mail.yandex.by> Message-ID: > Using .cmd scripts require using MS VS 2012 ( VC110). > Analogs of some .sh scriprs rewrited to .py aren't found. To answer some of it: Olaf presumably wrote the .sh files. I was on Windows. I rewrote them in .cmd. I recall someone wrote similar or additional ones. Then I was on Mac. I had multiple frustrations: cmd is definitely a horrible language. It is "builtin" yes, but otherwise is horrible. Really. Avoid it at much cost. Please let's not argue this. I haved use it a ton. Trust me. I suspect sh is a horrible language too, but I am less experienced with it. I am a little more open to discussing this, educate me, but this is not likely the right forum. In particular it seems too string based, not enough typing. I wanted *one* portable workflow, that I could understand and maintain and hopefully not just me. (Who among the .cmd maintainers also would maintain the .sh, or vice versa?) Those are both fuzzy things. sh is "portable" to Windows, just install Cygwin (or WSL). Heck, it is portable to MS-DOS (djgpp). I should? be able to maintain sh, but, if I have/had freedom on the matter, I chose Python. At least back then. I later became somewhat disenchanted with Python, but maybe here it is ok. It is certainly preferable to .cmd in any case. So I rewrote again in Python and have been using these ever since on Unix and Windows. I would really prefer to delete *.sh and *.cmd but people always protest that they like them. If you find something missing and desired in the *.py let me know, or try. I admit pylib.py is a bit monolithic ("easier to reuse") but I still think it isn't horrible. Or maybe I got coded into a corner and another rewrite is needed. I do think the parameter handling is probably backwards -- it should take out what it knows is for the Python and pass through everything else by default to cm3, instead of the nickel and dime maintanence where I keep adding pass throughs. I was slowed down (years?) getting to that realization though by the unicode size problem which I just fixed. We should probably try to use Quake and/or Modula-3 here more instead, really. I think Quake is up to this task actually. I spent a fair amount of time redoing all the config files also. ? (Esp. the "Cartesian refactoring" into architecture + OS, all the ".common" files). Quake may not be general purpose but it isn't horrible either. (The variable look ups are slow though, alas, I think multiple hash table lookups, one per nested scope..but it is fast enough). For building the entire system, approximately, drop m3makefile's that connect all the packages. For iterating over specific packages/groups, maybe trivial also. Granted, Quake is super obscure and time spent learning it will not reap much benefit. Python is at least "more useful" in the wide world. > How turn on "wide char" / 21bit UNICODE in 4 bytes feature on Win 32? Look here and you can figure it out: Rework widechar size initialization. ? modula3/cm3 at c408284 (github.com) There is no perfect answer here, as to the size of widechar. 1, 2, and 4 all make sense. 1 represents a rebellion against widechar, and the world has mostly rebelled against it or never accepted it 2 represents Windows widespread use, and Java, and .NET. This is both not-huge-like-4, and perhaps the victim of early adoption, when 2 seemed like eough. 4 represents Unix "widespread" use, or at least the typedef that nobody uses, and perhaps what Windows should use. As well, portability would be good. C and C++ already lack portability here, due to the windows:2 vs. unix:4 schism. C++ now has char16 and char32. Perhaps Modula-3 should too? - Jay ________________________________ From: vvm at tut.by Sent: Sunday, March 21, 2021 6:50 AM To: Jay K ; m3devel Subject: Re: [M3devel] refactoring cm3 into just one package? Hi! We can compare how we do some things today and will be do in future. As example: How turn on "wide char" / 21bit UNICODE in 4 bytes feature on Win 32? In correlated ReadMe described only Unix / Cygwin / MinGW way with .sh files. Using .cmd scripts require using MS VS 2012 ( VC110). Analogs of some .sh scriprs rewrited to .py aren't found. Best regards, Victor Miasnikov 20.03.2021, 21:31, "Jay K" : I was thinking.. sysutils m3scanner m3middle m3objfile m3linker m3back m3quake m3front cm3 Might as well we combine these into just one or two packages? I understand the need to break software up into coherent reusable pieces, and I have no intention to change the graph among these, but is this more than necessary splitting? Plus, you know, linkers can split up static libs. I say two because: Move it all into a lib, and then link executables to it. So some of it can be reused outside of cm3, some is. We'd have cm3lib and cm3, or cm3 and cm3exe. m3tools cm3ide etc. would import the library. The executable would just be one line that calls library. - Jay , _______________________________________________ M3devel mailing list M3devel at elegosoft.com https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 21 10:06:26 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 21 Mar 2021 12:06:26 +0300 Subject: [M3devel] refactoring cm3 into just one package? In-Reply-To: References: , <801831616309048@mail.yandex.by> Message-ID: <781721616317274@mail.yandex.by> An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 21 10:34:06 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 09:34:06 +0000 Subject: [M3devel] IR to defined records or not? In-Reply-To: References: Message-ID: Before packedVar the records were declared here: 0:000> k12 # Child-SP RetAddr Call Site 00 00000086`b66fd600 00007ff6`1b4b9312 cm3!CG__Declare_record+0x12 [C:\s\cm3\m3-sys\m3front\src\misc\CG.m3 @ 268] 01 00000086`b66fd650 00007ff6`1b425806 cm3!RecordType__Compiler+0x252 [C:\s\cm3\m3-sys\m3front\src\types\RecordType.m3 @ 325] 02 00000086`b66fd6d0 00007ff6`1b5a9b58 cm3!Type__Compile+0x126 [C:\s\cm3\m3-sys\m3front\src\types\Type.m3 @ 627] 03 00000086`b66fd750 00007ff6`1b5a5b8d cm3!Formal__GenRecord+0x1b8 [C:\s\cm3\m3-sys\m3front\src\values\Formal.m3 @ 628] 04 00000086`b66fd7f0 00007ff6`1b49b95d cm3!Formal__EmitArg+0x17d [C:\s\cm3\m3-sys\m3front\src\values\Formal.m3 @ 465] 05 00000086`b66fd840 00007ff6`1b5b1d33 cm3!UserProc__Prep+0x93d [C:\s\cm3\m3-sys\m3front\src\types\UserProc.m3 @ 132] 06 00000086`b66fd9c0 00007ff6`1b59d176 cm3!CallExpr__Prep+0xe3 [C:\s\cm3\m3-sys\m3front\src\exprs\CallExpr.m3 @ 343] 07 00000086`b66fda20 00007ff6`1b5c831a cm3!Expr__Prep+0x96 [C:\s\cm3\m3-sys\m3front\src\exprs\Expr.m3 @ 200] 08 00000086`b66fda70 00007ff6`1b4d13ac cm3!EvalStmt__Compile+0x8a [C:\s\cm3\m3-sys\m3front\src\stmts\EvalStmt.m3 @ 43] 09 00000086`b66fdac0 00007ff6`1b4d06cf cm3!Stmt__Compile+0xac [C:\s\cm3\m3-sys\m3front\src\stmts\Stmt.m3 @ 115] 0a 00000086`b66fdb20 00007ff6`1b4d13ac cm3!BlockStmt__Compile+0x40f [C:\s\cm3\m3-sys\m3front\src\stmts\BlockStmt.m3 @ 103] 0b 00000086`b66fdb80 00007ff6`1b46967f cm3!Stmt__Compile+0xac [C:\s\cm3\m3-sys\m3front\src\stmts\Stmt.m3 @ 115] 0c 00000086`b66fdbe0 00007ff6`1b46909e cm3!Procedure__GenBody+0x5cf [C:\s\cm3\m3-sys\m3front\src\values\Procedure.m3 @ 598] 0d 00000086`b66fdc70 00007ff6`1b4588b3 cm3!Procedure__EmitBody+0xae [C:\s\cm3\m3-sys\m3front\src\values\Procedure.m3 @ 563] 0e 00000086`b66fdcc0 00007ff6`1b4576e5 cm3!ProcBody__EmitBody+0x303 [C:\s\cm3\m3-sys\m3front\src\misc\ProcBody.m3 @ 171] 0f 00000086`b66fdd40 00007ff6`1b43607e cm3!ProcBody__EmitAll+0x135 [C:\s\cm3\m3-sys\m3front\src\misc\ProcBody.m3 @ 65] 10 00000086`b66fddd0 00007ff6`1b430d78 cm3!Module__CompileModule+0x29e [C:\s\cm3\m3-sys\m3front\src\values\Module.m3 @ 992] 11 00000086`b66fde60 00007ff6`1b4172ab cm3!Module__Compile+0x398 [C:\s\cm3\m3-sys\m3front\src\values\Module.m3 @ 923] This was lost it appears here: commit f39c54e1003a737cf4765ba63972ee8bac4af31b Author: Rodney Bates Date: Mon Apr 27 20:47:23 2020 -0500 Initial cut at passing packed parameters. Including some misc fixes and infrastructure additions. git difftool -dy f39c54e1003a737cf4765ba63972ee8bac4af31b~1..f39c54e1003a737cf4765ba63972ee8bac4af31b GenRecord became GenStruct and the Type.Compile call was lost before pop_struct. Formal.m3: ... -(* TODO^ Suppress generating code if Expr.Use returns TRUE? *) - Expr.Compile (actual); - EVAL Type.CheckInfo (t.tipe, info); - Type.Compile (t.tipe); - CG.Pop_struct (Type.GlobalUID (t.tipe), info.size, info.alignment); ... - Jay ________________________________ From: Jay K Sent: Sunday, March 21, 2021 6:43 AM To: m3devel ; rodney.m.bates at acm.org Subject: IR to defined records or not? C:\s\cm3\m3-ui\ui\src\trestle\InstalledVBT.m3 PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); (* line 41 *) j.trueChild := ch; EVAL j.init(child) END InitChild; The init call has some default parameters. It is actually: C:\s\cm3\m3-ui\ui\src\split\HighlightVBT.i3(31): init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T The C backend produces: #line 41 "../src/trestle/InstalledVBT.m3" (*(ADDRESS*)(&L_72_L_73))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_78_L_79)( ((ADDRESS)(grandChild_L_52)), ((ADDRESS)(ch_L_55)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(1000)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1024)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((ADDRESS)(&L_76_L_77)))))); See the third and fourth parameters are defaulted, to the addresses of globals. T79A5AEBB and T23E73F69 are record types. Being passed by address, I understand. Question is, should the IR declare those types? It used to. With the packedVar work, it does not. So this no longer compiles. The C backend issues an error for use of undefined types. But, I understand, it is only a pointer, so the type definition is arguably optional and optimize it away. Though more generally, a higher level IR would hopefully read: M_InstalledVBT.PaintOp->TransparentSwap or import_PaintOp->TransparentSwap passed by value and C compiler would need the type definition. It doesn't work that way currently of course. I suspect this change but don't fully understand it. ommit 0d6ffe0e98909412f820f3842c05f9e3d2e8e574 Author: Rodney Bates Date: Thu Oct 24 16:46:45 2019 -0500 Eliminate redundant record constructor temporary. A static record constructor was building two redundant copies of the record value: A: one built at RT into a stack temp (not used) and B: one built at CT into the static global area (used). Something like, call to Type.Compile is missing? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayk123 at hotmail.com Sun Mar 21 11:19:02 2021 From: jayk123 at hotmail.com (Jay K) Date: Sun, 21 Mar 2021 10:19:02 +0000 Subject: [M3devel] IR to defined records or not? In-Reply-To: References: , Message-ID: m3front: Restore declaration of record types. ? modula3/cm3 at cc6eed0 (github.com) ________________________________ From: Jay K Sent: Sunday, March 21, 2021 9:34 AM To: m3devel ; rodney.m.bates at acm.org Subject: Re: IR to defined records or not? Before packedVar the records were declared here: 0:000> k12 # Child-SP RetAddr Call Site 00 00000086`b66fd600 00007ff6`1b4b9312 cm3!CG__Declare_record+0x12 [C:\s\cm3\m3-sys\m3front\src\misc\CG.m3 @ 268] 01 00000086`b66fd650 00007ff6`1b425806 cm3!RecordType__Compiler+0x252 [C:\s\cm3\m3-sys\m3front\src\types\RecordType.m3 @ 325] 02 00000086`b66fd6d0 00007ff6`1b5a9b58 cm3!Type__Compile+0x126 [C:\s\cm3\m3-sys\m3front\src\types\Type.m3 @ 627] 03 00000086`b66fd750 00007ff6`1b5a5b8d cm3!Formal__GenRecord+0x1b8 [C:\s\cm3\m3-sys\m3front\src\values\Formal.m3 @ 628] 04 00000086`b66fd7f0 00007ff6`1b49b95d cm3!Formal__EmitArg+0x17d [C:\s\cm3\m3-sys\m3front\src\values\Formal.m3 @ 465] 05 00000086`b66fd840 00007ff6`1b5b1d33 cm3!UserProc__Prep+0x93d [C:\s\cm3\m3-sys\m3front\src\types\UserProc.m3 @ 132] 06 00000086`b66fd9c0 00007ff6`1b59d176 cm3!CallExpr__Prep+0xe3 [C:\s\cm3\m3-sys\m3front\src\exprs\CallExpr.m3 @ 343] 07 00000086`b66fda20 00007ff6`1b5c831a cm3!Expr__Prep+0x96 [C:\s\cm3\m3-sys\m3front\src\exprs\Expr.m3 @ 200] 08 00000086`b66fda70 00007ff6`1b4d13ac cm3!EvalStmt__Compile+0x8a [C:\s\cm3\m3-sys\m3front\src\stmts\EvalStmt.m3 @ 43] 09 00000086`b66fdac0 00007ff6`1b4d06cf cm3!Stmt__Compile+0xac [C:\s\cm3\m3-sys\m3front\src\stmts\Stmt.m3 @ 115] 0a 00000086`b66fdb20 00007ff6`1b4d13ac cm3!BlockStmt__Compile+0x40f [C:\s\cm3\m3-sys\m3front\src\stmts\BlockStmt.m3 @ 103] 0b 00000086`b66fdb80 00007ff6`1b46967f cm3!Stmt__Compile+0xac [C:\s\cm3\m3-sys\m3front\src\stmts\Stmt.m3 @ 115] 0c 00000086`b66fdbe0 00007ff6`1b46909e cm3!Procedure__GenBody+0x5cf [C:\s\cm3\m3-sys\m3front\src\values\Procedure.m3 @ 598] 0d 00000086`b66fdc70 00007ff6`1b4588b3 cm3!Procedure__EmitBody+0xae [C:\s\cm3\m3-sys\m3front\src\values\Procedure.m3 @ 563] 0e 00000086`b66fdcc0 00007ff6`1b4576e5 cm3!ProcBody__EmitBody+0x303 [C:\s\cm3\m3-sys\m3front\src\misc\ProcBody.m3 @ 171] 0f 00000086`b66fdd40 00007ff6`1b43607e cm3!ProcBody__EmitAll+0x135 [C:\s\cm3\m3-sys\m3front\src\misc\ProcBody.m3 @ 65] 10 00000086`b66fddd0 00007ff6`1b430d78 cm3!Module__CompileModule+0x29e [C:\s\cm3\m3-sys\m3front\src\values\Module.m3 @ 992] 11 00000086`b66fde60 00007ff6`1b4172ab cm3!Module__Compile+0x398 [C:\s\cm3\m3-sys\m3front\src\values\Module.m3 @ 923] This was lost it appears here: commit f39c54e1003a737cf4765ba63972ee8bac4af31b Author: Rodney Bates Date: Mon Apr 27 20:47:23 2020 -0500 Initial cut at passing packed parameters. Including some misc fixes and infrastructure additions. git difftool -dy f39c54e1003a737cf4765ba63972ee8bac4af31b~1..f39c54e1003a737cf4765ba63972ee8bac4af31b GenRecord became GenStruct and the Type.Compile call was lost before pop_struct. Formal.m3: ... -(* TODO^ Suppress generating code if Expr.Use returns TRUE? *) - Expr.Compile (actual); - EVAL Type.CheckInfo (t.tipe, info); - Type.Compile (t.tipe); - CG.Pop_struct (Type.GlobalUID (t.tipe), info.size, info.alignment); ... - Jay ________________________________ From: Jay K Sent: Sunday, March 21, 2021 6:43 AM To: m3devel ; rodney.m.bates at acm.org Subject: IR to defined records or not? C:\s\cm3\m3-ui\ui\src\trestle\InstalledVBT.m3 PROCEDURE InitChild (j: Join; ch: VBT.T; p: DeleteProc)= VAR grandChild := NEW(GrandChild, proc := p); child := StableVBT.New(grandChild); BEGIN EVAL grandChild.init(ch); (* line 41 *) j.trueChild := ch; EVAL j.init(child) END InitChild; The init call has some default parameters. It is actually: C:\s\cm3\m3-ui\ui\src\split\HighlightVBT.i3(31): init(ch: VBT.T; op: PaintOp.T := PaintOp.TransparentSwap; txt: Pixmap.T := Pixmap.Gray; READONLY delta := Point.T{h := 0, v := 1}): T The C backend produces: #line 41 "../src/trestle/InstalledVBT.m3" (*(ADDRESS*)(&L_72_L_73))=(ADDRESS)(((ADDRESS)(((ADDRESS (__cdecl*)(DOTDOTDOT))L_78_L_79)( ((ADDRESS)(grandChild_L_52)), ((ADDRESS)(ch_L_55)), ((T79A5AEBB*)(*((ADDRESS*)(INT64_(232)+((ADDRESS)(*((ADDRESS*)(INT64_(1000)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((T23E73F69*)(*((ADDRESS*)(INT64_(120)+((ADDRESS)(*((ADDRESS*)(INT64_(1024)+((ADDRESS)(&M_InstalledVBT_L_44)))))))))), ((ADDRESS)(&L_76_L_77)))))); See the third and fourth parameters are defaulted, to the addresses of globals. T79A5AEBB and T23E73F69 are record types. Being passed by address, I understand. Question is, should the IR declare those types? It used to. With the packedVar work, it does not. So this no longer compiles. The C backend issues an error for use of undefined types. But, I understand, it is only a pointer, so the type definition is arguably optional and optimize it away. Though more generally, a higher level IR would hopefully read: M_InstalledVBT.PaintOp->TransparentSwap or import_PaintOp->TransparentSwap passed by value and C compiler would need the type definition. It doesn't work that way currently of course. I suspect this change but don't fully understand it. ommit 0d6ffe0e98909412f820f3842c05f9e3d2e8e574 Author: Rodney Bates Date: Thu Oct 24 16:46:45 2019 -0500 Eliminate redundant record constructor temporary. A static record constructor was building two redundant copies of the record value: A: one built at RT into a stack temp (not used) and B: one built at CT into the static global area (used). Something like, call to Type.Compile is missing? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From vvm at tut.by Sun Mar 21 11:39:17 2021 From: vvm at tut.by (vvm at tut.by) Date: Sun, 21 Mar 2021 13:39:17 +0300 Subject: [M3devel] Packed strings as 3 21bit WIDECHARs in 8 bytes Re: refactoring cm3 into just one package? In-Reply-To: <781721616317274@mail.yandex.by> References: , <801831616309048@mail.yandex.by> <781721616317274@mail.yandex.by> Message-ID: <652131616322707@mail.yandex.by> An HTML attachment was scrubbed... URL: From hendrik at topoi.pooq.com Sun Mar 21 13:18:35 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 21 Mar 2021 08:18:35 -0400 Subject: [M3devel] refactoring cm3 into just one package? In-Reply-To: References: <801831616309048@mail.yandex.by> Message-ID: <20210321121835.brsfui3qi7gy32j3@topoi.pooq.com> On Sun, Mar 21, 2021 at 07:34:08AM +0000, Jay K wrote: > > Using .cmd scripts require using MS VS 2012 ( VC110). > > Analogs of some .sh scriprs rewrited to .py aren't found. > > To answer some of it: > Olaf presumably wrote the .sh files. > I was on Windows. I rewrote them in .cmd. I recall someone wrote similar or additional ones. > Then I was on Mac. > I had multiple frustrations: > cmd is definitely a horrible language. It is "builtin" yes, but otherwise is horrible. > Really. Avoid it at much cost. Please let's not argue this. I haved use it a ton. Trust me. > > I suspect sh is a horrible language too, but I am less experienced with it. > I am a little more open to discussing this, educate me, but this is not likely the right forum. > In particular it seems too string based, not enough typing. > > I wanted *one* portable workflow, that I could understand and maintain and hopefully not just me. > (Who among the .cmd maintainers also would maintain the .sh, or vice versa?) > Those are both fuzzy things. sh is "portable" to Windows, just install Cygwin (or WSL). > Heck, it is portable to MS-DOS (djgpp). > I should? be able to maintain sh, but, if I have/had freedom on the matter, I chose Python. At least back then. > I later became somewhat disenchanted with Python, but maybe here it is ok. > It is certainly preferable to .cmd in any case. > > > So I rewrote again in Python and have been using these ever since on Unix and Windows. > > I would really prefer to delete *.sh and *.cmd but people always protest that they like them. > > If you find something missing and desired in the *.py let me know, or try. > I admit pylib.py is a bit monolithic ("easier to reuse") but I still think it isn't horrible. Or maybe I got coded into a corner and another rewrite is needed. I do think the parameter handling is probably backwards -- it should take out what it knows is for the Python and pass through everything else by default to cm3, instead of the nickel and dime maintanence where I keep adding pass throughs. I was slowed down (years?) getting to that realization though by the unicode size problem which I just fixed. > > We should probably try to use Quake and/or Modula-3 here more instead, really. > I think Quake is up to this task actually. I spent a fair amount of time redoing all the config files also. ? > (Esp. the "Cartesian refactoring" into architecture + OS, all the ".common" files). > > Quake may not be general purpose but it isn't horrible either. (The variable look ups are slow though, alas, I think multiple hash table lookups, one per nested scope..but it is fast enough). You seem to be fishing for a programming language to use as a bootstrapping tool. Have you considerd Scheme? There are implementations of it that run somewhat indifferently on Linux, Windows, and Mac. I use Racket myself on Linux, and it claims to be compatible across Windows Mac, and Linux; but Gnu's own Guile seems to be a smaller system and is designed to be a scripting extension language for other programs. -- hendrik > > For building the entire system, approximately, drop m3makefile's that connect all the packages. > > For iterating over specific packages/groups, maybe trivial also. > > Granted, Quake is super obscure and time spent learning it will not reap much benefit. > Python is at least "more useful" in the wide world. > > > How turn on "wide char" / 21bit UNICODE in 4 bytes feature on Win 32? > > Look here and you can figure it out: > > Rework widechar size initialization. ? modula3/cm3 at c408284 (github.com) > > There is no perfect answer here, as to the size of widechar. 1, 2, and 4 all make sense. > 1 represents a rebellion against widechar, and the world has mostly rebelled against it or never accepted it > 2 represents Windows widespread use, and Java, and .NET. > This is both not-huge-like-4, and perhaps the victim of early adoption, when 2 seemed like eough. > 4 represents Unix "widespread" use, or at least the typedef that nobody uses, and perhaps what Windows should use. > As well, portability would be good. C and C++ already lack portability here, due to the windows:2 vs. unix:4 schism. > C++ now has char16 and char32. Perhaps Modula-3 should too? > > - Jay > > ________________________________ > From: vvm at tut.by > Sent: Sunday, March 21, 2021 6:50 AM > To: Jay K ; m3devel > Subject: Re: [M3devel] refactoring cm3 into just one package? > > Hi! > > > We can compare how we do some things today and will be do in future. > > As example: > > How turn on "wide char" / 21bit UNICODE in 4 bytes feature on Win 32? > > In correlated ReadMe described only Unix / Cygwin / MinGW way with .sh files. > > Using .cmd scripts require using MS VS 2012 ( VC110). > > Analogs of some .sh scriprs rewrited to .py aren't found. > > > Best regards, Victor Miasnikov > > 20.03.2021, 21:31, "Jay K" : > I was thinking.. > > sysutils m3scanner m3middle m3objfile m3linker m3back m3quake m3front cm3 > > Might as well we combine these into just one or two packages? > I understand the need to break software up into coherent > reusable pieces, and I have no intention to change the graph > among these, but is this more than necessary splitting? > > Plus, you know, linkers can split up static libs. > > I say two because: Move it all into a lib, and then link executables to it. > So some of it can be reused outside of cm3, some is. > We'd have cm3lib and cm3, or cm3 and cm3exe. > m3tools cm3ide etc. would import the library. > The executable would just be one line that calls library. > > - Jay > , > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel From hendrik at topoi.pooq.com Sun Mar 21 13:26:09 2021 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 21 Mar 2021 08:26:09 -0400 Subject: [M3devel] cm3 build order In-Reply-To: References: <20210321014209.uie77fd3yj4dfhhg@topoi.pooq.com> Message-ID: <20210321122609.lwcs6p5wn2br4g2h@topoi.pooq.com> On Sun, Mar 21, 2021 at 03:25:00AM +0000, Jay K wrote: > We do the "build multiple times and it works" thing but not (yet) the memcmp check. > > gcc's build has the build twice and memcmp but sometimes e.g. debug info gets in the way. > There is this thing that the inputs are sometimes more than it might seem. > For example the full path to the source. Maybe the username. And maybe the time. > (int main() { printf("%s\n", TIMESTAMP);} > compiles differently every time. > > also the operating systems are now deliberately random: > int main() { printf("%p\", main); } > runs differently every time. > > So you have to be sure not to do those things. They look obvious but there are subtle variations also. > > Deterministic builds are a "new" thing, or, well, um, recently rediscovered. > It is a shame. They should have been the thing all along. But timestamps, version numbers, guids, creep in. > > I was involved with a project where "patches" where produced by comparing updated builds with older builds. > So I got to discover surprises. Things like, add a function declaration to a header. Change nothing else. Do not implement or call that function, no other change. And you'd get different results. The input did in fact change, but not in a way that should matter. I'm fighting this now in fact. In cm3 building AMD64_LINUX vs. ALPHA_LINUX is almost identical input and almost identical output, but something gets reordered. Possibly those very strings get into hash tables and then an output occurs in hash order (for example: m3front: Output revelations in sorted order instead of hash order. ? modula3/cm3 at 505aaae (github.com) does that matter? It was deterministic before, but slight perturbations in input had disproportionate impact on output. Now those same input perturbations have no impact on output.) Which is why the Algol 68 C reproducibiity was a three-phase compilation sequence instead of a two-phase one. > > Debian started doing deterministic builds a few years ago, but not all packages. > Visual C++ only recently got a non-default option to be deterministic. > In my view it should be the only mode. > > cm3 is guilty of this itself. > It likes to put the current time in itself. And that's a killer. Maybe a compile-time option to lie about the time? Or maybe an object-file diff program that finds and ignores the timestamp? > And developers do have that fascination with ever increasing version numbers. > The time is the best, because it always increases. > Here: > cm3: Move version from C object to C source, using ? modula3/cm3 at ae2770f (github.com) > > cm3 never ever builds the same twice. Deliberately. It isn't my doing. I preserved the old behavior. > > See ReproducibleBuilds - Debian Wiki Yes, that's what you would need for this to work. > > Reproducible, right, I always forget that name. > The key is that others can reproduce the same build even. > That the source can be verified to be the actual source. > > An introduction to deterministic builds with C/C++ (conan.io) > Getting to Deterministic Builds on Windows - Nikhil's blog (nikhilism.com) > > Windows, C++: investigate '/experimental:deterministic' compiler flag ? Issue #9466 ? bazelbuild/bazel (github.com) > etc. > > > - Jay > > ________________________________ > From: M3devel on behalf of Hendrik Boom > Sent: Sunday, March 21, 2021 1:42 AM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] cm3 build order > > On Sun, Mar 21, 2021 at 12:55:56AM +0000, Jay K wrote: > > Yes to both, but you can also put the bugs in the other parts. > > > > I like how make-dist.py works in this regard: it never really updates in place. > > And then, after that, just recursive copy. > > Upgrade.py should maybe take a similar approach. > > > > It should be (is?) possible to also output or ship directly into a new parallel structure. > > The way the Cambridge Algol 68C compiler was bootstrapped from one version to another: > > (1) Compile the new source code with the old compiler. > (2) Compile the new source code using the result of (1). > (3) Compile the new source code using the result of (2). > > The compiler is considered OK if the result of (2) is bit-for-bit > identical with the resut of (2). > > If not identical, it says that old compiler produces code that > behaved sifferently from the code generated by the new one for the > same source code, so there must be a bug. > > They did keep old versions around somewhere, just in case. > And they did sometimes discover they had to go back several versions to > deal with a problem. > > -- hendrik > > > > > Make-dist.py kinda cheats, it does that by copying cm3 into the new structure and running it from there. > > > > Another approach would be run the unshipped compiler. Don?t ship it until it works. Make a copy in its output directory so it can link itself, approximately. > > > > - Jay > > ________________________________ > > From: Peter McKinna > > Sent: Saturday, March 20, 2021 4:12:20 PM > > To: Jay K > > Cc: m3devel > > Subject: Re: [M3devel] cm3 build order > > > > I'm not sure it's relevant, but I thought the restriction > > was partly meant to prevent you from building a broken cm3 > > caused by adding a bug to the libraries. > > > > I know, in the past, I have mucked around with m3core and built > > a cm3 that was useless and I had to find an old copy somewhere or > > download the boot distribution and start over. > > > > Maybe, before we start the rebuild we make a backup copy of cm3 > > and cm3cg and whatever other backend you like? > > > > On Sun, 21 Mar 2021 at 09:52, Jay K > wrote: > > So we have this thing that confuses people, > > when you first rebuild the compiler, you do not rebuild m3core and libm3. > > > > I noticed in old notes the notion that this is, really, "optional". > > Sometimes building m3core + libm3 first is ok, or even required. > > > > So I thought I'd explain my understanding. > > Why the confusing ordering? > > > > 1. cm3 outputs calls to m3core. > > So cm3 is tightly coupled with m3core. > > For example NEW calls the allocator. > > See RT0.i3. > > But this part rarely changes. > > The last change was probably Tony adding the barriers. > > > > 2. m3core is written in Modula-3. > > Sometimes the language changes and newer compiler > > is needed to compile m3core. > > Again this is rare and the last change was LONGINT addition. > > > > 3. It used to be the case, used to, that the compiler and either > > m3core or libm3, had a long list (enum) of target systems. They had > > to be in sync. > > Any time you added a target, you had to use that build ordering. > > You could not even just add to the end a new system, because of the type hashing. > > But I removed both copies of the enum long ago. > > (Basically, aside, as I ported, I kept judging the steps to port > > not really justified and gradually removed them.) > > > > On the flip side, very simple, sometimes you might add > > something to m3core or libm3 that