From rodney_bates at lcwb.coop Mon Mar 2 00:26:19 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 01 Mar 2015 17:26:19 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> Message-ID: <54F3A01B.8000207@lcwb.coop> On 02/27/2015 03:46 PM, Olaf Wagner wrote: > On Fri, 27 Feb 2015 12:55:26 -0600 > "Rodney M. Bates" wrote: > >> I have quite a list of little (and some not so little) maintenance things to >> do to the Modula-3 distribution. I am learning to use git and github, but >> some involve other things than just the repository. > > I'm just in the last stages of migrating one of our customer projects > from Bazaar to Git, including a full-featured automated testing and > deployment system. We've already invested more than 400 hours for > that. Automation of building, testing and deploying software needs > a lot of maintenance and work, which nobody has done yet for the > M3 Github transition. I put a plan of things that would need to be > done into the old Trac Wiki, but I doubt that more than 10% of them > have been achieved. > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. I have asked michael? to give it to me. >> 1) Olaf, are you intending to continue hosting the web pages? > > Elego can do still do that if no one else volunteers. I cannot > guarantee more that a couple of minutes administration time per > week though. We're rather short of ressources currently. > >> 2) How can I update these? There appear to be copies of some/many of them >> in the cm3 repository, but these are not served from there as web pages, >> AFAIK, > > There used to be packages names www and doc, and some ship scripts, > which were used to transfer the contents to our web server. I assume > they're still in the Github repo in the same form, so you could use > them. You only need ssh access for birch.elegosoft.com, if Mike > hasn't moved that to another VM. I just tried and can login there. > There is an account > rodney:x:6011:6011:Rodney M. Bates:/home/rodney:/bin/bash > > You have quite a few keys installed there: > > % sudo ls /home/rodney/.ssh/ > allegheny.id_dsa.pub authorized_keys.1 authorized_keys.3 corliss.id_dsa.pub known_hosts selkirk.id_dsa.pub > authorized_keys authorized_keys2 authorized_keys.3~ garratt.id_dsa.pub rbates.id_dsa.pub yellowstone.id_dsa.pub > authorized_keys.0 authorized_keys.2 authorized_keys.4 id_dsa.pub runnymede.id_dsa.pub > > If you still have one of them, you have ssh access. If not, Mike > (manderson at elegosoft.com) will store another for you. I can log in to birch. Poking around, I find that birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net are the same machine. I find I also have a password on this machine as well. Is this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch to allow me to access the same machine would obviate needing this password. > > The shipping scripts can (could?) be found in the packages: > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > I also found copies of these is a number of places, most significanly, /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... The scripts are the same, but presumably other stuff is more up-to-date here. There are a _lot_ of differences in this directory that are not shipped to the web directory. I don't feel like shipping any of it without careful looking, to be sure it is what we want on the web. > The machine still is in use: > > elegos-MacBook-Pro:tools.git wagner$ host www.opencm3.net > www.opencm3.net is an alias for birch.elego.de. > birch.elego.de has address 46.4.219.187 > > (elego.de and elegosoft.com are mostly equivalent) > >> 3) I have never understood the automated build system on various platforms. >> What is its status? Can run these builds manually, or will they run >> automatically if I put source updates in the right place? What targets >> do we have? (I have LINUXLIBC6 and AMD64_LINUX myself.) > > I'm afraid that the Hudson installation on our server is out of order > for quiet a long time. I don't know of any adaptation even for the > source checkout from Github. > > I think some other public server should be used now for builds and > tests. The old scripts could be adapted, but most people prefer to > do these things in their own way. I used shell in 2010, but many > scripts have been rewritten in python after that. > Does anybody have a server to suggest here? I think a couple of my own machines would have enough storage, RAM, CPU, and possibly network capacity for the builds themselves if run only in overnight hours. But I definitely do not have the networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe LINUXLIBC6. >> 4) I have also never understood how to run all the many test cases that are >> in the repository. Has it been done at all recently? Are there scripts >> to do it? Can I add cases to them? Are there various targets I can get >> them to run on? > > I haven't done that recently, and I don't really know if anybody > else has. > I would really like to do it and set up a working CI system for > M3 again -- I still think this is the best programming language > for large systems I have encountered so far -- but nobody has ever > paid Elego one Euro for any M3 work, so it's just not business- > compatible ;-) I've got to care for my company, especially since > we've had some hard times during the last two years. > We all greatly appreciate all the support you have given to Modula3. I hope at least its benefits have helped you in other, revenue-producing projects. > I think the shell and python scripts could be adapted to set up > a working automated build and test system with some weeks of > effort. It's nothing that can be done by changing a few lines > I'm afraid. > > I just tried to access the Hudson CI system, and it's still > working, though nothing has been running there for some time. > You'll find it at http://hudson.modula3.com:8080. Some > build servers seem to be still online: master (Linux), > luthien (FreeBSD 8), and the Solaris buildfarm (opencsw). > The last successful builds seem to be 1 year and 4 months ago, > for example > http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-AMD64_LINUX/ > > You could transfer these jobs to another Hudson or Jenkins system > (the config.xml files should suffice for that), adapt the source > checkout to the Github repository and try to get some builds > running. If you really plan to do that, I offer to help with > all the advice and knowledge I have by answering your questions. > With some help, I could take this on. I have looked briefly at hudson/jenkins. It looks like there is enough info there to get things going. > I hope this hasn't been too discouraging, The most daunting part for me is mediocre network access. My latency has recently improved enough so that command-line character echo has gone from just about unusable to marginally usable. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From wagner at elegosoft.com Thu Mar 5 15:58:05 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 5 Mar 2015 15:58:05 +0100 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <54F3A01B.8000207@lcwb.coop> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> Message-ID: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> On Sun, 01 Mar 2015 17:26:19 -0600 "Rodney M. Bates" wrote: > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. > I have asked michael? to give it to me. OK. I spoke with Michael about you and M3 at Elego, but I haven't seen an answer from him, so I don't know if he got back to you. > >> 1) Olaf, are you intending to continue hosting the web pages? > > > > Elego can do still do that if no one else volunteers. I cannot > > guarantee more that a couple of minutes administration time per > > week though. We're rather short of ressources currently. > > > >> 2) How can I update these? There appear to be copies of some/many of them > >> in the cm3 repository, but these are not served from there as web pages, > >> AFAIK, [...] > I can log in to birch. Poking around, I find that > birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net > are the same machine. I find I also have a password on this machine as well. Is > this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It > is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch > to allow me to access the same machine would obviate needing this password. Yes, it's all the same VM currently, but it will probably be migrated to other servers and services separated. Yes, you can install any new key you like there. > > The shipping scripts can (could?) be found in the packages: > > > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > > I also found copies of these is a number of places, most significanly, > > /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... > > The scripts are the same, but presumably other stuff is more up-to-date here. > > There are a _lot_ of differences in this directory that are not shipped to the web > directory. I don't feel like shipping any of it without careful looking, to be > sure it is what we want on the web. Hm, I cannot really explain these offhand. Perhaps its best to ship to a sibling directory and then do a diff -r -b -B old new. > > I'm afraid that the Hudson installation on our server is out of order > > for quiet a long time. I don't know of any adaptation even for the > > source checkout from Github. > > > > I think some other public server should be used now for builds and > > tests. The old scripts could be adapted, but most people prefer to > > do these things in their own way. I used shell in 2010, but many > > scripts have been rewritten in python after that. > > Does anybody have a server to suggest here? I think a couple of my own machines > would have enough storage, RAM, CPU, and possibly network capacity for the builds > themselves if run only in overnight hours. But I definitely do not have the > networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe > LINUXLIBC6. I don't really know one offhand now. As far as I know Github does not have it's own build service. But Mike has suggested that if you don't find a suitable service, we can continue to host the CI server. But he wants to abandon the old and outdated Hudson and install a current Jenkins system. That would be much easier to maintain, too. > > You could transfer these jobs to another Hudson or Jenkins system > > (the config.xml files should suffice for that), adapt the source > > checkout to the Github repository and try to get some builds > > running. If you really plan to do that, I offer to help with > > all the advice and knowledge I have by answering your questions. > > With some help, I could take this on. I have looked briefly at hudson/jenkins. > It looks like there is enough info there to get things going. Look at Jenkins, this is the current free open source variant. > The most daunting part for me is mediocre network access. My latency has recently > improved enough so that command-line character echo has gone from just about > unusable to marginally usable. That's annoying of course. Excuse my curiosity: where are you located? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri Mar 6 22:38:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 06 Mar 2015 15:38:06 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> Message-ID: <54FA1E3E.8020804@lcwb.coop> On 03/05/2015 08:58 AM, Olaf Wagner wrote: > On Sun, 01 Mar 2015 17:26:19 -0600 > "Rodney M. Bates" wrote: > >> I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. >> I have asked michael? to give it to me. > > OK. I spoke with Michael about you and M3 at Elego, but I haven't > seen an answer from him, so I don't know if he got back to you. > Not yet. >>>> 1) Olaf, are you intending to continue hosting the web pages? >>> >>> Elego can do still do that if no one else volunteers. I cannot >>> guarantee more that a couple of minutes administration time per >>> week though. We're rather short of ressources currently. >>> >>>> 2) How can I update these? There appear to be copies of some/many of them >>>> in the cm3 repository, but these are not served from there as web pages, >>>> AFAIK, > [...] > >> I can log in to birch. Poking around, I find that >> birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net >> are the same machine. I find I also have a password on this machine as well. Is >> this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It >> is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch >> to allow me to access the same machine would obviate needing this password. > > Yes, it's all the same VM currently, but it will probably be migrated > to other servers and services separated. > > Yes, you can install any new key you like there. Done. I also changed the password. Do you want me to delete password login altogether? > >>> The shipping scripts can (could?) be found in the packages: >>> >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc >> >> I also found copies of these is a number of places, most significanly, >> >> /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... >> >> The scripts are the same, but presumably other stuff is more up-to-date here. >> >> There are a _lot_ of differences in this directory that are not shipped to the web >> directory. I don't feel like shipping any of it without careful looking, to be >> sure it is what we want on the web. > > Hm, I cannot really explain these offhand. Perhaps its best to ship > to a sibling directory and then do a diff -r -b -B old new. > There is not enough disk space to create a sibling directory, but I have been diffing directly between the likely place to ship from and the shipped web files. Looking at just the first file that differs (about-cm3.html), I find the web version is CVS version 1.14.2.2, in branch release_CM3_5_8_6, and dated 2010-04-29, whereas the head and just about everywhere I look is version 1.14, dated 2007-05-15. So the release branch versions are not merged back to the head. I will work on figuring out how many files this applies to and get them merged into the git head, as appropriate. >>> I'm afraid that the Hudson installation on our server is out of order >>> for quiet a long time. I don't know of any adaptation even for the >>> source checkout from Github. >>> >>> I think some other public server should be used now for builds and >>> tests. The old scripts could be adapted, but most people prefer to >>> do these things in their own way. I used shell in 2010, but many >>> scripts have been rewritten in python after that. >> >> Does anybody have a server to suggest here? I think a couple of my own machines >> would have enough storage, RAM, CPU, and possibly network capacity for the builds >> themselves if run only in overnight hours. But I definitely do not have the >> networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe >> LINUXLIBC6. > > I don't really know one offhand now. As far as I know Github does not > have it's own build service. But Mike has suggested that if you don't > find a suitable service, we can continue to host the CI server. But > he wants to abandon the old and outdated Hudson and install a current > Jenkins system. That would be much easier to maintain, too. > >>> You could transfer these jobs to another Hudson or Jenkins system >>> (the config.xml files should suffice for that), adapt the source >>> checkout to the Github repository and try to get some builds >>> running. If you really plan to do that, I offer to help with >>> all the advice and knowledge I have by answering your questions. >> >> With some help, I could take this on. I have looked briefly at hudson/jenkins. >> It looks like there is enough info there to get things going. > > Look at Jenkins, this is the current free open source variant. > I've started looking at it. I has some hudson->jenkins conversion instructions. >> The most daunting part for me is mediocre network access. My latency has recently >> improved enough so that command-line character echo has gone from just about >> unusable to marginally usable. > > That's annoying of course. Excuse my curiosity: where are you located? Tallgrass prairie, in the flint hills of eastern Kansas, where I have wanted to live for decades. But I am just barely out of reach of two land-line and one ground-level radio internet services, in one case, by just 200 feet, and by regulatory, not technical limitations. Fortunately, my satellite ISP has recently improved the round-trip latency from ~1.5 sec to ~.8 sec. which is just marginally usable for command-line access. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Mar 18 08:06:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 18 Mar 2015 07:06:49 +0000 Subject: [M3devel] access to github/modula3/cm3? Message-ID: Do I have write access to github modula3/cm3? I don't think so. Can I be added? i.e. we are allow direct writes, w/o going through pull requests? I have small changes to get started: 1) fix m3-demo/mentor/src/sorting/m3makefile for bootstrapping from last release 2) fix m3-sys/mklib/main.m3 similarly, by cloning everything it uses from m3core/winnt.i3 3) copy targets.txt to .gitignore everywhere Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Mar 21 01:10:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 21 Mar 2015 00:10:44 +0000 Subject: [M3devel] propose "output root" to replace shipping Message-ID: Building the current system fails in caltech-parser\parserlib\parserlib\test. It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. But even if shipped exists, it likely shouldn't be using it. This is a general problem we have been through somewhat before. There are multiple parts to our current partial solution: They can be seen here: C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl And those various "build tools" use build_standalone() This could imho be abstracted. However, I propose where today we have: source_root/m3-foo/bar/target/abc.obj source_root/m3-foo/bar/target/bar.exe source_root/m3-foo/bar/src/abc.m3 we instead have: output_root/obj/bar/target/abc.obj output_root/bin/target/bar.exe output_root/bin/bar.exe link or stub to target/abc.exe output_root/pkg/bar/... or: output_root_obj/bar/target/abc.obj output_root_install/bin/target/bar.exe output_root_install/bin/bar.exe link or stub to target/abc.exe output_root_install/pkg/bar/... and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it just "script", not anything in cm3, and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't use standalone for this, and on systems that don't support "origin", still use standalone and then shipping goes away, as a package operation. you can instead ship an output root. and then this override stuff goes away also; You bootstrap by copying in a few working files. Like make-dist.py does. The system is reduced in size, as build_standalone falls away -- you can still use it, but it won't be used usually. The larger source tree would become readonly, as it should be. Thoughts? I believe Modula-3 was originally ahead of its time in having the readonly src directories, but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which might as well not say "src", but preserve that instead of rearranging everything. I know, I've heard, having the separate buildlocal / buildglobal is useful. This doesn't elimine it that. It replaces it with "multiple buildglobal". Instead of having just the two options -- local and global -- you can create an arbitrary number of "global" scopes by creating a new output_root, copying it from a preexisting one. As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. This has several nice properties. readonly source tree less file copying possibly (depending on if final scripted ship is directory copy or move) cm3 can stop implementing ship no need to use build_standalone on most systems, including for cm3 itself (but it still works) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Mar 22 17:57:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 22 Mar 2015 11:57:35 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: Message-ID: <550EF47F.7080006@lcwb.coop> I haven't had time to digest your proposal yet, but I have often been uncomfortable about the current build system when building/bootstrapping itself. It is probably just lack of understanding of the right procedure, but I usually cringe before attempting to bootstrap. I have always been unclear about the sequence of when the newly built components overlay the preexisting ones, so I would welcome some attention to this. I would like to put in a vote for a way to build everything without overlaying any existing components, so a bootstrapping operation can be easily restarted back where it started. Right now, for the compiler itself, I am meticulously saving side copies of the compiler executables in /usr/local/cm3/bin every time. On 03/20/2015 07:10 PM, Jay K wrote: > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > But even if shipped exists, it likely shouldn't be using it. > > > > This is a general problem we have been through somewhat before. > > > > There are multiple parts to our current partial solution: > > > They can be seen here: > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > And those various "build tools" use build_standalone() > > > This could imho be abstracted. > > > However, I propose where today we have: > source_root/m3-foo/bar/target/abc.obj > source_root/m3-foo/bar/target/bar.exe > source_root/m3-foo/bar/src/abc.m3 > > > we instead have: > output_root/obj/bar/target/abc.obj > output_root/bin/target/bar.exe > output_root/bin/bar.exe link or stub to target/abc.exe > output_root/pkg/bar/... > > > or: > output_root_obj/bar/target/abc.obj > output_root_install/bin/target/bar.exe > output_root_install/bin/bar.exe link or stub to target/abc.exe > output_root_install/pkg/bar/... > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > just "script", not anything in cm3, > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > use standalone for this, and on systems that don't support "origin", still use standalone > > > and then shipping goes away, as a package operation. > you can instead ship an output root. > > > and then this override stuff goes away also; > You bootstrap by copying in a few working files. Like make-dist.py does. > The system is reduced in size, as build_standalone falls away -- you can still use it, > but it won't be used usually. > > > The larger source tree would become readonly, as it should be. > > > Thoughts? > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > might as well not say "src", but preserve that instead of rearranging everything. > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > This doesn't elimine it that. > It replaces it with "multiple buildglobal". > Instead of having just the two options -- local and global -- you can create an arbitrary number of > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > This has several nice properties. > readonly source tree > less file copying possibly (depending on if final scripted ship is directory copy or move) > cm3 can stop implementing ship > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Mar 24 09:19:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:19:06 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <550EF47F.7080006@lcwb.coop> References: , <550EF47F.7080006@lcwb.coop> Message-ID: The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Mar 24 09:29:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:29:19 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Mar 24 10:01:39 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 10:01:39 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: Has anybody ever considered to use Zero Install for shipping and distribution? From: Jay K Sent: Tuesday, March 24, 2015 9:29 AM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] propose "output root" to replace shipping There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay -------------------------------------------------------------------------------- From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 11:22:01 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 10:22:01 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> Message-ID: <20150324102200.GA2982@zoho.com> On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From dmuysers at hotmail.com Tue Mar 24 12:08:13 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 12:08:13 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <20150324102200.GA2982@zoho.com> References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: I doubt that you haven't given a look at 0install, because it cares well about security concerns. One of the problems with Windows 7 is that, if you install cm3 into the programs folder (where it should naturally belong to), you can't even create package directories or modify your code because of the very strict UAC that enforces immutability for everything belonging to a program. -----Original Message----- From: microcode at zoho.com Sent: Tuesday, March 24, 2015 11:22 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] propose "output root" to replace shipping On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and > distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From estellnb at elstel.org Tue Mar 24 18:25:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 24 Mar 2015 18:25:07 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: <55119DF3.8000803@elstel.org> Am 24.03.15 um 09:29 schrieb Jay K: > There is a problem here. > Something I also want to do is stop building things standalone, > on NT and systems that support "origin", which is pretty much all of them. > The problem .. I'm alluding to two solutions. > > > 1) "run from" and "install to" are the same > 2) or different > > > But we also want "atomic" update of cm3 and its dependencies > (m3core/libm3). > #1 only works if the old and new are compatible-enough. > #1 is more automatic > > > I think the answer is, if you are rebuilding m3core/libm3/cm3, and the > old and new are not compatible-enough, you must set the output to be a > new empty directory, and "run from" and "install to" must be > different. If you are building almost anything else, you can safely > update in place w/o breaking the compiler. > Dear Jay K. Yes, I had been thinking the same way as you those times when I hacked pm3 and cm3. A solution for it needed to be still around somewhere on my hard disk. However I remember it was not so easy to achieve this as there are mutual dependencies between the front end and the core library. If there are incompatibilities between both you may need an intermediate code version that ships b but still uses a, followed by a version that uses b and ships b. Nonetheless the approach works fine as long as no such dependencies are given. Unfortunately I currently do not have the time to dig it out for you; next month or more likely just in two month I can perhaps do that if you are interested. It would be great to have such a feature in the main tree (i.e ship to B and compile from A) though getting the changes there will be somewhat beyond my scope. Cheers, Elmar -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 19:37:23 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 18:37:23 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: <20150324183723.GA3046@zoho.com> I already said I don't care what happens with Windows. That platform is and has always been riddled with security holes like Swiss cheese and 99.9% of Windows victims need an installer anyway. It's not whether they need one it's just which one is least bad. Perhaps you are not aware that every application you add to your system is another vector for vulnerabilities and bugs. Perhaps you are also not aware people running UNIX or UNIX-like OS really are not interested and in most cases will simply not use anything that has any installation requirements beyond a normal build or untarring a tarball. I don't tolerate installers running as root downloading random crap from the web. I download what I want myself from the correct site and I check the signatures or checksums if provided. Then I build the application from source. Then I check it. If it looks ok then I install it. I don't think it's appropriate to burden non-Windows users with potential malware just for the sake of Windows users. We are not accustomed to keep clicking OK until the system reboots. On Tue, Mar 24, 2015 at 12:08:13PM +0100, dirk muysers wrote: > I doubt that you haven't given a look at 0install, because it cares well > about security concerns. One of the problems with Windows 7 is that, > if you install cm3 into the programs folder (where it should naturally > belong to), you can't even create package directories or modify your code > because of the very strict UAC that enforces immutability for everything > belonging to a program. > > -----Original Message----- From: microcode at zoho.com > Sent: Tuesday, March 24, 2015 11:22 AM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > >Has anybody ever considered to use Zero Install for shipping and > >distribution? > > Requiring infrastructure to "install" something is a good way to kill off > any remining interest. Please, keep the requirements to an absolute minimum > and don't expect people to add tools or security holes to their OS except > for Windows where that isn't a concern. Other platforms mostly don't expect > plug and play installations nor do they want them. > > -- From rodney_bates at lcwb.coop Sat Mar 28 16:12:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:12:34 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C4E2.9010701@lcwb.coop> This has suddenly reminded me of a problem and solution from the past. I once did a lot of work in and on a package containing several source code generators for components of compilers and such (Cocktail, if anybody cares). It was highly cyclic, several of the generators being used by several of themselves to generate parts of their own code. One day, after introducing a simple bug, I ended up with no working executable needed to regenerate itself after I fixed the bug. I don't remember how I got out of this, but it took a lot of work and it was tedious and very hacky. Afterwards, I spent some time setting up a new build environment. There could be any number of what I called "views" (named after a different development system from mars), each a complete, more-or-less copy of everything: hand edited source, generated source, compiled objects, executables. The build scripts used a symlink, found at a fixed place within a view, to access executables to be used in building. The view-cloning script set up this link in the newly cloned view to point to the executables in the old view. So each view was built using unmodified executables from the older view it was cloned from. The use process was to always keep a view named "stable", well tested to be able to rebuild itself. I would clone a new view named "working" and do development in that. Periodically, I would clone "phase1" from working, build in phase1 (which used executables in working), clone "phase2" from phase1, and build in phase2. If the sources in working could regenerate and rebuild themselves twice in this way, I considered it good evidence to make phase2 become the new stable. I don't remember the details, but I also had some scripted diffs between equivalent generated source files in different views, to show the regeneration process had converged to a fixed point. As I recall, there were often legitimate differences that had to be manually reviewed, but that was just a good test of the most recent changes. After that things worked very smoothly. It did take up quite a bit of disk space, with multiple copies of everything, but it was still feasible, even in days when 6 GB was a big disk. We don't have as much circularity in Modula-3, but there is some, and even a tiny bit can cause a lot of grief. So I suggest something like this. Installing from a view to a system-wide installation location would be a separate step. It would be greatly simplified if, as Jay suggested, the as-built and as-installed subdirectory structures could be made the same. On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay >-- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:15:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:15:03 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C577.1000503@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. I really like this idea. Things might need to be slightly different when it comes to installed copies of library interfaces. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:59:02 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:59:02 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516CFC6.6040504@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? I think this is still the list to use. I found, with a bit of difficulty, how to get git to send emails about git events, and set it up to send to me. Perhaps we would want to set these to send to m3commit? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > >-- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Tue Mar 31 15:40:38 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 31 Mar 2015 15:40:38 +0200 Subject: [M3devel] existant backup data against losses from server crash Message-ID: <551AA3D6.1090805@elstel.org> Dear Modula-3 maintainers and developers; dear Elegosoft, Last year I have downloaded the complete modula3.elegosoft.com/cm3 page including several source and compiled versions of CM3. Today I have visited your page again and saw the message that many links were broken on your page due to data losses on a server crash. I believe that I still have the web image of your page somewhere around as downloaded by wget -R. It should allow the recovery of all lost release data which was available to the public. If you are interested I will burn some blue rays from it which I can send you via snail mail (uploading would be far too slow with my internet connection). Just let me know ... Yours Sincerely, Elmar Stellnberger From rodney_bates at lcwb.coop Mon Mar 2 00:26:19 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 01 Mar 2015 17:26:19 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> Message-ID: <54F3A01B.8000207@lcwb.coop> On 02/27/2015 03:46 PM, Olaf Wagner wrote: > On Fri, 27 Feb 2015 12:55:26 -0600 > "Rodney M. Bates" wrote: > >> I have quite a list of little (and some not so little) maintenance things to >> do to the Modula-3 distribution. I am learning to use git and github, but >> some involve other things than just the repository. > > I'm just in the last stages of migrating one of our customer projects > from Bazaar to Git, including a full-featured automated testing and > deployment system. We've already invested more than 400 hours for > that. Automation of building, testing and deploying software needs > a lot of maintenance and work, which nobody has done yet for the > M3 Github transition. I put a plan of things that would need to be > done into the old Trac Wiki, but I doubt that more than 10% of them > have been achieved. > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. I have asked michael? to give it to me. >> 1) Olaf, are you intending to continue hosting the web pages? > > Elego can do still do that if no one else volunteers. I cannot > guarantee more that a couple of minutes administration time per > week though. We're rather short of ressources currently. > >> 2) How can I update these? There appear to be copies of some/many of them >> in the cm3 repository, but these are not served from there as web pages, >> AFAIK, > > There used to be packages names www and doc, and some ship scripts, > which were used to transfer the contents to our web server. I assume > they're still in the Github repo in the same form, so you could use > them. You only need ssh access for birch.elegosoft.com, if Mike > hasn't moved that to another VM. I just tried and can login there. > There is an account > rodney:x:6011:6011:Rodney M. Bates:/home/rodney:/bin/bash > > You have quite a few keys installed there: > > % sudo ls /home/rodney/.ssh/ > allegheny.id_dsa.pub authorized_keys.1 authorized_keys.3 corliss.id_dsa.pub known_hosts selkirk.id_dsa.pub > authorized_keys authorized_keys2 authorized_keys.3~ garratt.id_dsa.pub rbates.id_dsa.pub yellowstone.id_dsa.pub > authorized_keys.0 authorized_keys.2 authorized_keys.4 id_dsa.pub runnymede.id_dsa.pub > > If you still have one of them, you have ssh access. If not, Mike > (manderson at elegosoft.com) will store another for you. I can log in to birch. Poking around, I find that birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net are the same machine. I find I also have a password on this machine as well. Is this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch to allow me to access the same machine would obviate needing this password. > > The shipping scripts can (could?) be found in the packages: > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > I also found copies of these is a number of places, most significanly, /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... The scripts are the same, but presumably other stuff is more up-to-date here. There are a _lot_ of differences in this directory that are not shipped to the web directory. I don't feel like shipping any of it without careful looking, to be sure it is what we want on the web. > The machine still is in use: > > elegos-MacBook-Pro:tools.git wagner$ host www.opencm3.net > www.opencm3.net is an alias for birch.elego.de. > birch.elego.de has address 46.4.219.187 > > (elego.de and elegosoft.com are mostly equivalent) > >> 3) I have never understood the automated build system on various platforms. >> What is its status? Can run these builds manually, or will they run >> automatically if I put source updates in the right place? What targets >> do we have? (I have LINUXLIBC6 and AMD64_LINUX myself.) > > I'm afraid that the Hudson installation on our server is out of order > for quiet a long time. I don't know of any adaptation even for the > source checkout from Github. > > I think some other public server should be used now for builds and > tests. The old scripts could be adapted, but most people prefer to > do these things in their own way. I used shell in 2010, but many > scripts have been rewritten in python after that. > Does anybody have a server to suggest here? I think a couple of my own machines would have enough storage, RAM, CPU, and possibly network capacity for the builds themselves if run only in overnight hours. But I definitely do not have the networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe LINUXLIBC6. >> 4) I have also never understood how to run all the many test cases that are >> in the repository. Has it been done at all recently? Are there scripts >> to do it? Can I add cases to them? Are there various targets I can get >> them to run on? > > I haven't done that recently, and I don't really know if anybody > else has. > I would really like to do it and set up a working CI system for > M3 again -- I still think this is the best programming language > for large systems I have encountered so far -- but nobody has ever > paid Elego one Euro for any M3 work, so it's just not business- > compatible ;-) I've got to care for my company, especially since > we've had some hard times during the last two years. > We all greatly appreciate all the support you have given to Modula3. I hope at least its benefits have helped you in other, revenue-producing projects. > I think the shell and python scripts could be adapted to set up > a working automated build and test system with some weeks of > effort. It's nothing that can be done by changing a few lines > I'm afraid. > > I just tried to access the Hudson CI system, and it's still > working, though nothing has been running there for some time. > You'll find it at http://hudson.modula3.com:8080. Some > build servers seem to be still online: master (Linux), > luthien (FreeBSD 8), and the Solaris buildfarm (opencsw). > The last successful builds seem to be 1 year and 4 months ago, > for example > http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-AMD64_LINUX/ > > You could transfer these jobs to another Hudson or Jenkins system > (the config.xml files should suffice for that), adapt the source > checkout to the Github repository and try to get some builds > running. If you really plan to do that, I offer to help with > all the advice and knowledge I have by answering your questions. > With some help, I could take this on. I have looked briefly at hudson/jenkins. It looks like there is enough info there to get things going. > I hope this hasn't been too discouraging, The most daunting part for me is mediocre network access. My latency has recently improved enough so that command-line character echo has gone from just about unusable to marginally usable. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From wagner at elegosoft.com Thu Mar 5 15:58:05 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 5 Mar 2015 15:58:05 +0100 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <54F3A01B.8000207@lcwb.coop> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> Message-ID: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> On Sun, 01 Mar 2015 17:26:19 -0600 "Rodney M. Bates" wrote: > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. > I have asked michael? to give it to me. OK. I spoke with Michael about you and M3 at Elego, but I haven't seen an answer from him, so I don't know if he got back to you. > >> 1) Olaf, are you intending to continue hosting the web pages? > > > > Elego can do still do that if no one else volunteers. I cannot > > guarantee more that a couple of minutes administration time per > > week though. We're rather short of ressources currently. > > > >> 2) How can I update these? There appear to be copies of some/many of them > >> in the cm3 repository, but these are not served from there as web pages, > >> AFAIK, [...] > I can log in to birch. Poking around, I find that > birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net > are the same machine. I find I also have a password on this machine as well. Is > this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It > is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch > to allow me to access the same machine would obviate needing this password. Yes, it's all the same VM currently, but it will probably be migrated to other servers and services separated. Yes, you can install any new key you like there. > > The shipping scripts can (could?) be found in the packages: > > > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > > I also found copies of these is a number of places, most significanly, > > /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... > > The scripts are the same, but presumably other stuff is more up-to-date here. > > There are a _lot_ of differences in this directory that are not shipped to the web > directory. I don't feel like shipping any of it without careful looking, to be > sure it is what we want on the web. Hm, I cannot really explain these offhand. Perhaps its best to ship to a sibling directory and then do a diff -r -b -B old new. > > I'm afraid that the Hudson installation on our server is out of order > > for quiet a long time. I don't know of any adaptation even for the > > source checkout from Github. > > > > I think some other public server should be used now for builds and > > tests. The old scripts could be adapted, but most people prefer to > > do these things in their own way. I used shell in 2010, but many > > scripts have been rewritten in python after that. > > Does anybody have a server to suggest here? I think a couple of my own machines > would have enough storage, RAM, CPU, and possibly network capacity for the builds > themselves if run only in overnight hours. But I definitely do not have the > networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe > LINUXLIBC6. I don't really know one offhand now. As far as I know Github does not have it's own build service. But Mike has suggested that if you don't find a suitable service, we can continue to host the CI server. But he wants to abandon the old and outdated Hudson and install a current Jenkins system. That would be much easier to maintain, too. > > You could transfer these jobs to another Hudson or Jenkins system > > (the config.xml files should suffice for that), adapt the source > > checkout to the Github repository and try to get some builds > > running. If you really plan to do that, I offer to help with > > all the advice and knowledge I have by answering your questions. > > With some help, I could take this on. I have looked briefly at hudson/jenkins. > It looks like there is enough info there to get things going. Look at Jenkins, this is the current free open source variant. > The most daunting part for me is mediocre network access. My latency has recently > improved enough so that command-line character echo has gone from just about > unusable to marginally usable. That's annoying of course. Excuse my curiosity: where are you located? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri Mar 6 22:38:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 06 Mar 2015 15:38:06 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> Message-ID: <54FA1E3E.8020804@lcwb.coop> On 03/05/2015 08:58 AM, Olaf Wagner wrote: > On Sun, 01 Mar 2015 17:26:19 -0600 > "Rodney M. Bates" wrote: > >> I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. >> I have asked michael? to give it to me. > > OK. I spoke with Michael about you and M3 at Elego, but I haven't > seen an answer from him, so I don't know if he got back to you. > Not yet. >>>> 1) Olaf, are you intending to continue hosting the web pages? >>> >>> Elego can do still do that if no one else volunteers. I cannot >>> guarantee more that a couple of minutes administration time per >>> week though. We're rather short of ressources currently. >>> >>>> 2) How can I update these? There appear to be copies of some/many of them >>>> in the cm3 repository, but these are not served from there as web pages, >>>> AFAIK, > [...] > >> I can log in to birch. Poking around, I find that >> birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net >> are the same machine. I find I also have a password on this machine as well. Is >> this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It >> is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch >> to allow me to access the same machine would obviate needing this password. > > Yes, it's all the same VM currently, but it will probably be migrated > to other servers and services separated. > > Yes, you can install any new key you like there. Done. I also changed the password. Do you want me to delete password login altogether? > >>> The shipping scripts can (could?) be found in the packages: >>> >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc >> >> I also found copies of these is a number of places, most significanly, >> >> /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... >> >> The scripts are the same, but presumably other stuff is more up-to-date here. >> >> There are a _lot_ of differences in this directory that are not shipped to the web >> directory. I don't feel like shipping any of it without careful looking, to be >> sure it is what we want on the web. > > Hm, I cannot really explain these offhand. Perhaps its best to ship > to a sibling directory and then do a diff -r -b -B old new. > There is not enough disk space to create a sibling directory, but I have been diffing directly between the likely place to ship from and the shipped web files. Looking at just the first file that differs (about-cm3.html), I find the web version is CVS version 1.14.2.2, in branch release_CM3_5_8_6, and dated 2010-04-29, whereas the head and just about everywhere I look is version 1.14, dated 2007-05-15. So the release branch versions are not merged back to the head. I will work on figuring out how many files this applies to and get them merged into the git head, as appropriate. >>> I'm afraid that the Hudson installation on our server is out of order >>> for quiet a long time. I don't know of any adaptation even for the >>> source checkout from Github. >>> >>> I think some other public server should be used now for builds and >>> tests. The old scripts could be adapted, but most people prefer to >>> do these things in their own way. I used shell in 2010, but many >>> scripts have been rewritten in python after that. >> >> Does anybody have a server to suggest here? I think a couple of my own machines >> would have enough storage, RAM, CPU, and possibly network capacity for the builds >> themselves if run only in overnight hours. But I definitely do not have the >> networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe >> LINUXLIBC6. > > I don't really know one offhand now. As far as I know Github does not > have it's own build service. But Mike has suggested that if you don't > find a suitable service, we can continue to host the CI server. But > he wants to abandon the old and outdated Hudson and install a current > Jenkins system. That would be much easier to maintain, too. > >>> You could transfer these jobs to another Hudson or Jenkins system >>> (the config.xml files should suffice for that), adapt the source >>> checkout to the Github repository and try to get some builds >>> running. If you really plan to do that, I offer to help with >>> all the advice and knowledge I have by answering your questions. >> >> With some help, I could take this on. I have looked briefly at hudson/jenkins. >> It looks like there is enough info there to get things going. > > Look at Jenkins, this is the current free open source variant. > I've started looking at it. I has some hudson->jenkins conversion instructions. >> The most daunting part for me is mediocre network access. My latency has recently >> improved enough so that command-line character echo has gone from just about >> unusable to marginally usable. > > That's annoying of course. Excuse my curiosity: where are you located? Tallgrass prairie, in the flint hills of eastern Kansas, where I have wanted to live for decades. But I am just barely out of reach of two land-line and one ground-level radio internet services, in one case, by just 200 feet, and by regulatory, not technical limitations. Fortunately, my satellite ISP has recently improved the round-trip latency from ~1.5 sec to ~.8 sec. which is just marginally usable for command-line access. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Mar 18 08:06:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 18 Mar 2015 07:06:49 +0000 Subject: [M3devel] access to github/modula3/cm3? Message-ID: Do I have write access to github modula3/cm3? I don't think so. Can I be added? i.e. we are allow direct writes, w/o going through pull requests? I have small changes to get started: 1) fix m3-demo/mentor/src/sorting/m3makefile for bootstrapping from last release 2) fix m3-sys/mklib/main.m3 similarly, by cloning everything it uses from m3core/winnt.i3 3) copy targets.txt to .gitignore everywhere Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Mar 21 01:10:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 21 Mar 2015 00:10:44 +0000 Subject: [M3devel] propose "output root" to replace shipping Message-ID: Building the current system fails in caltech-parser\parserlib\parserlib\test. It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. But even if shipped exists, it likely shouldn't be using it. This is a general problem we have been through somewhat before. There are multiple parts to our current partial solution: They can be seen here: C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl And those various "build tools" use build_standalone() This could imho be abstracted. However, I propose where today we have: source_root/m3-foo/bar/target/abc.obj source_root/m3-foo/bar/target/bar.exe source_root/m3-foo/bar/src/abc.m3 we instead have: output_root/obj/bar/target/abc.obj output_root/bin/target/bar.exe output_root/bin/bar.exe link or stub to target/abc.exe output_root/pkg/bar/... or: output_root_obj/bar/target/abc.obj output_root_install/bin/target/bar.exe output_root_install/bin/bar.exe link or stub to target/abc.exe output_root_install/pkg/bar/... and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it just "script", not anything in cm3, and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't use standalone for this, and on systems that don't support "origin", still use standalone and then shipping goes away, as a package operation. you can instead ship an output root. and then this override stuff goes away also; You bootstrap by copying in a few working files. Like make-dist.py does. The system is reduced in size, as build_standalone falls away -- you can still use it, but it won't be used usually. The larger source tree would become readonly, as it should be. Thoughts? I believe Modula-3 was originally ahead of its time in having the readonly src directories, but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which might as well not say "src", but preserve that instead of rearranging everything. I know, I've heard, having the separate buildlocal / buildglobal is useful. This doesn't elimine it that. It replaces it with "multiple buildglobal". Instead of having just the two options -- local and global -- you can create an arbitrary number of "global" scopes by creating a new output_root, copying it from a preexisting one. As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. This has several nice properties. readonly source tree less file copying possibly (depending on if final scripted ship is directory copy or move) cm3 can stop implementing ship no need to use build_standalone on most systems, including for cm3 itself (but it still works) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Mar 22 17:57:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 22 Mar 2015 11:57:35 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: Message-ID: <550EF47F.7080006@lcwb.coop> I haven't had time to digest your proposal yet, but I have often been uncomfortable about the current build system when building/bootstrapping itself. It is probably just lack of understanding of the right procedure, but I usually cringe before attempting to bootstrap. I have always been unclear about the sequence of when the newly built components overlay the preexisting ones, so I would welcome some attention to this. I would like to put in a vote for a way to build everything without overlaying any existing components, so a bootstrapping operation can be easily restarted back where it started. Right now, for the compiler itself, I am meticulously saving side copies of the compiler executables in /usr/local/cm3/bin every time. On 03/20/2015 07:10 PM, Jay K wrote: > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > But even if shipped exists, it likely shouldn't be using it. > > > > This is a general problem we have been through somewhat before. > > > > There are multiple parts to our current partial solution: > > > They can be seen here: > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > And those various "build tools" use build_standalone() > > > This could imho be abstracted. > > > However, I propose where today we have: > source_root/m3-foo/bar/target/abc.obj > source_root/m3-foo/bar/target/bar.exe > source_root/m3-foo/bar/src/abc.m3 > > > we instead have: > output_root/obj/bar/target/abc.obj > output_root/bin/target/bar.exe > output_root/bin/bar.exe link or stub to target/abc.exe > output_root/pkg/bar/... > > > or: > output_root_obj/bar/target/abc.obj > output_root_install/bin/target/bar.exe > output_root_install/bin/bar.exe link or stub to target/abc.exe > output_root_install/pkg/bar/... > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > just "script", not anything in cm3, > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > use standalone for this, and on systems that don't support "origin", still use standalone > > > and then shipping goes away, as a package operation. > you can instead ship an output root. > > > and then this override stuff goes away also; > You bootstrap by copying in a few working files. Like make-dist.py does. > The system is reduced in size, as build_standalone falls away -- you can still use it, > but it won't be used usually. > > > The larger source tree would become readonly, as it should be. > > > Thoughts? > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > might as well not say "src", but preserve that instead of rearranging everything. > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > This doesn't elimine it that. > It replaces it with "multiple buildglobal". > Instead of having just the two options -- local and global -- you can create an arbitrary number of > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > This has several nice properties. > readonly source tree > less file copying possibly (depending on if final scripted ship is directory copy or move) > cm3 can stop implementing ship > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Mar 24 09:19:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:19:06 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <550EF47F.7080006@lcwb.coop> References: , <550EF47F.7080006@lcwb.coop> Message-ID: The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Mar 24 09:29:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:29:19 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Mar 24 10:01:39 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 10:01:39 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: Has anybody ever considered to use Zero Install for shipping and distribution? From: Jay K Sent: Tuesday, March 24, 2015 9:29 AM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] propose "output root" to replace shipping There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay -------------------------------------------------------------------------------- From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 11:22:01 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 10:22:01 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> Message-ID: <20150324102200.GA2982@zoho.com> On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From dmuysers at hotmail.com Tue Mar 24 12:08:13 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 12:08:13 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <20150324102200.GA2982@zoho.com> References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: I doubt that you haven't given a look at 0install, because it cares well about security concerns. One of the problems with Windows 7 is that, if you install cm3 into the programs folder (where it should naturally belong to), you can't even create package directories or modify your code because of the very strict UAC that enforces immutability for everything belonging to a program. -----Original Message----- From: microcode at zoho.com Sent: Tuesday, March 24, 2015 11:22 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] propose "output root" to replace shipping On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and > distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From estellnb at elstel.org Tue Mar 24 18:25:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 24 Mar 2015 18:25:07 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: <55119DF3.8000803@elstel.org> Am 24.03.15 um 09:29 schrieb Jay K: > There is a problem here. > Something I also want to do is stop building things standalone, > on NT and systems that support "origin", which is pretty much all of them. > The problem .. I'm alluding to two solutions. > > > 1) "run from" and "install to" are the same > 2) or different > > > But we also want "atomic" update of cm3 and its dependencies > (m3core/libm3). > #1 only works if the old and new are compatible-enough. > #1 is more automatic > > > I think the answer is, if you are rebuilding m3core/libm3/cm3, and the > old and new are not compatible-enough, you must set the output to be a > new empty directory, and "run from" and "install to" must be > different. If you are building almost anything else, you can safely > update in place w/o breaking the compiler. > Dear Jay K. Yes, I had been thinking the same way as you those times when I hacked pm3 and cm3. A solution for it needed to be still around somewhere on my hard disk. However I remember it was not so easy to achieve this as there are mutual dependencies between the front end and the core library. If there are incompatibilities between both you may need an intermediate code version that ships b but still uses a, followed by a version that uses b and ships b. Nonetheless the approach works fine as long as no such dependencies are given. Unfortunately I currently do not have the time to dig it out for you; next month or more likely just in two month I can perhaps do that if you are interested. It would be great to have such a feature in the main tree (i.e ship to B and compile from A) though getting the changes there will be somewhat beyond my scope. Cheers, Elmar -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 19:37:23 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 18:37:23 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: <20150324183723.GA3046@zoho.com> I already said I don't care what happens with Windows. That platform is and has always been riddled with security holes like Swiss cheese and 99.9% of Windows victims need an installer anyway. It's not whether they need one it's just which one is least bad. Perhaps you are not aware that every application you add to your system is another vector for vulnerabilities and bugs. Perhaps you are also not aware people running UNIX or UNIX-like OS really are not interested and in most cases will simply not use anything that has any installation requirements beyond a normal build or untarring a tarball. I don't tolerate installers running as root downloading random crap from the web. I download what I want myself from the correct site and I check the signatures or checksums if provided. Then I build the application from source. Then I check it. If it looks ok then I install it. I don't think it's appropriate to burden non-Windows users with potential malware just for the sake of Windows users. We are not accustomed to keep clicking OK until the system reboots. On Tue, Mar 24, 2015 at 12:08:13PM +0100, dirk muysers wrote: > I doubt that you haven't given a look at 0install, because it cares well > about security concerns. One of the problems with Windows 7 is that, > if you install cm3 into the programs folder (where it should naturally > belong to), you can't even create package directories or modify your code > because of the very strict UAC that enforces immutability for everything > belonging to a program. > > -----Original Message----- From: microcode at zoho.com > Sent: Tuesday, March 24, 2015 11:22 AM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > >Has anybody ever considered to use Zero Install for shipping and > >distribution? > > Requiring infrastructure to "install" something is a good way to kill off > any remining interest. Please, keep the requirements to an absolute minimum > and don't expect people to add tools or security holes to their OS except > for Windows where that isn't a concern. Other platforms mostly don't expect > plug and play installations nor do they want them. > > -- From rodney_bates at lcwb.coop Sat Mar 28 16:12:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:12:34 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C4E2.9010701@lcwb.coop> This has suddenly reminded me of a problem and solution from the past. I once did a lot of work in and on a package containing several source code generators for components of compilers and such (Cocktail, if anybody cares). It was highly cyclic, several of the generators being used by several of themselves to generate parts of their own code. One day, after introducing a simple bug, I ended up with no working executable needed to regenerate itself after I fixed the bug. I don't remember how I got out of this, but it took a lot of work and it was tedious and very hacky. Afterwards, I spent some time setting up a new build environment. There could be any number of what I called "views" (named after a different development system from mars), each a complete, more-or-less copy of everything: hand edited source, generated source, compiled objects, executables. The build scripts used a symlink, found at a fixed place within a view, to access executables to be used in building. The view-cloning script set up this link in the newly cloned view to point to the executables in the old view. So each view was built using unmodified executables from the older view it was cloned from. The use process was to always keep a view named "stable", well tested to be able to rebuild itself. I would clone a new view named "working" and do development in that. Periodically, I would clone "phase1" from working, build in phase1 (which used executables in working), clone "phase2" from phase1, and build in phase2. If the sources in working could regenerate and rebuild themselves twice in this way, I considered it good evidence to make phase2 become the new stable. I don't remember the details, but I also had some scripted diffs between equivalent generated source files in different views, to show the regeneration process had converged to a fixed point. As I recall, there were often legitimate differences that had to be manually reviewed, but that was just a good test of the most recent changes. After that things worked very smoothly. It did take up quite a bit of disk space, with multiple copies of everything, but it was still feasible, even in days when 6 GB was a big disk. We don't have as much circularity in Modula-3, but there is some, and even a tiny bit can cause a lot of grief. So I suggest something like this. Installing from a view to a system-wide installation location would be a separate step. It would be greatly simplified if, as Jay suggested, the as-built and as-installed subdirectory structures could be made the same. On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay >-- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:15:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:15:03 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C577.1000503@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. I really like this idea. Things might need to be slightly different when it comes to installed copies of library interfaces. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:59:02 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:59:02 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516CFC6.6040504@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? I think this is still the list to use. I found, with a bit of difficulty, how to get git to send emails about git events, and set it up to send to me. Perhaps we would want to set these to send to m3commit? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > >-- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Tue Mar 31 15:40:38 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 31 Mar 2015 15:40:38 +0200 Subject: [M3devel] existant backup data against losses from server crash Message-ID: <551AA3D6.1090805@elstel.org> Dear Modula-3 maintainers and developers; dear Elegosoft, Last year I have downloaded the complete modula3.elegosoft.com/cm3 page including several source and compiled versions of CM3. Today I have visited your page again and saw the message that many links were broken on your page due to data losses on a server crash. I believe that I still have the web image of your page somewhere around as downloaded by wget -R. It should allow the recovery of all lost release data which was available to the public. If you are interested I will burn some blue rays from it which I can send you via snail mail (uploading would be far too slow with my internet connection). Just let me know ... Yours Sincerely, Elmar Stellnberger From rodney_bates at lcwb.coop Mon Mar 2 00:26:19 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 01 Mar 2015 17:26:19 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> Message-ID: <54F3A01B.8000207@lcwb.coop> On 02/27/2015 03:46 PM, Olaf Wagner wrote: > On Fri, 27 Feb 2015 12:55:26 -0600 > "Rodney M. Bates" wrote: > >> I have quite a list of little (and some not so little) maintenance things to >> do to the Modula-3 distribution. I am learning to use git and github, but >> some involve other things than just the repository. > > I'm just in the last stages of migrating one of our customer projects > from Bazaar to Git, including a full-featured automated testing and > deployment system. We've already invested more than 400 hours for > that. Automation of building, testing and deploying software needs > a lot of maintenance and work, which nobody has done yet for the > M3 Github transition. I put a plan of things that would need to be > done into the old Trac Wiki, but I doubt that more than 10% of them > have been achieved. > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. I have asked michael? to give it to me. >> 1) Olaf, are you intending to continue hosting the web pages? > > Elego can do still do that if no one else volunteers. I cannot > guarantee more that a couple of minutes administration time per > week though. We're rather short of ressources currently. > >> 2) How can I update these? There appear to be copies of some/many of them >> in the cm3 repository, but these are not served from there as web pages, >> AFAIK, > > There used to be packages names www and doc, and some ship scripts, > which were used to transfer the contents to our web server. I assume > they're still in the Github repo in the same form, so you could use > them. You only need ssh access for birch.elegosoft.com, if Mike > hasn't moved that to another VM. I just tried and can login there. > There is an account > rodney:x:6011:6011:Rodney M. Bates:/home/rodney:/bin/bash > > You have quite a few keys installed there: > > % sudo ls /home/rodney/.ssh/ > allegheny.id_dsa.pub authorized_keys.1 authorized_keys.3 corliss.id_dsa.pub known_hosts selkirk.id_dsa.pub > authorized_keys authorized_keys2 authorized_keys.3~ garratt.id_dsa.pub rbates.id_dsa.pub yellowstone.id_dsa.pub > authorized_keys.0 authorized_keys.2 authorized_keys.4 id_dsa.pub runnymede.id_dsa.pub > > If you still have one of them, you have ssh access. If not, Mike > (manderson at elegosoft.com) will store another for you. I can log in to birch. Poking around, I find that birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net are the same machine. I find I also have a password on this machine as well. Is this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch to allow me to access the same machine would obviate needing this password. > > The shipping scripts can (could?) be found in the packages: > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > I also found copies of these is a number of places, most significanly, /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... The scripts are the same, but presumably other stuff is more up-to-date here. There are a _lot_ of differences in this directory that are not shipped to the web directory. I don't feel like shipping any of it without careful looking, to be sure it is what we want on the web. > The machine still is in use: > > elegos-MacBook-Pro:tools.git wagner$ host www.opencm3.net > www.opencm3.net is an alias for birch.elego.de. > birch.elego.de has address 46.4.219.187 > > (elego.de and elegosoft.com are mostly equivalent) > >> 3) I have never understood the automated build system on various platforms. >> What is its status? Can run these builds manually, or will they run >> automatically if I put source updates in the right place? What targets >> do we have? (I have LINUXLIBC6 and AMD64_LINUX myself.) > > I'm afraid that the Hudson installation on our server is out of order > for quiet a long time. I don't know of any adaptation even for the > source checkout from Github. > > I think some other public server should be used now for builds and > tests. The old scripts could be adapted, but most people prefer to > do these things in their own way. I used shell in 2010, but many > scripts have been rewritten in python after that. > Does anybody have a server to suggest here? I think a couple of my own machines would have enough storage, RAM, CPU, and possibly network capacity for the builds themselves if run only in overnight hours. But I definitely do not have the networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe LINUXLIBC6. >> 4) I have also never understood how to run all the many test cases that are >> in the repository. Has it been done at all recently? Are there scripts >> to do it? Can I add cases to them? Are there various targets I can get >> them to run on? > > I haven't done that recently, and I don't really know if anybody > else has. > I would really like to do it and set up a working CI system for > M3 again -- I still think this is the best programming language > for large systems I have encountered so far -- but nobody has ever > paid Elego one Euro for any M3 work, so it's just not business- > compatible ;-) I've got to care for my company, especially since > we've had some hard times during the last two years. > We all greatly appreciate all the support you have given to Modula3. I hope at least its benefits have helped you in other, revenue-producing projects. > I think the shell and python scripts could be adapted to set up > a working automated build and test system with some weeks of > effort. It's nothing that can be done by changing a few lines > I'm afraid. > > I just tried to access the Hudson CI system, and it's still > working, though nothing has been running there for some time. > You'll find it at http://hudson.modula3.com:8080. Some > build servers seem to be still online: master (Linux), > luthien (FreeBSD 8), and the Solaris buildfarm (opencsw). > The last successful builds seem to be 1 year and 4 months ago, > for example > http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-AMD64_LINUX/ > > You could transfer these jobs to another Hudson or Jenkins system > (the config.xml files should suffice for that), adapt the source > checkout to the Github repository and try to get some builds > running. If you really plan to do that, I offer to help with > all the advice and knowledge I have by answering your questions. > With some help, I could take this on. I have looked briefly at hudson/jenkins. It looks like there is enough info there to get things going. > I hope this hasn't been too discouraging, The most daunting part for me is mediocre network access. My latency has recently improved enough so that command-line character echo has gone from just about unusable to marginally usable. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From wagner at elegosoft.com Thu Mar 5 15:58:05 2015 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 5 Mar 2015 15:58:05 +0100 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <54F3A01B.8000207@lcwb.coop> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> Message-ID: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> On Sun, 01 Mar 2015 17:26:19 -0600 "Rodney M. Bates" wrote: > I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. > I have asked michael? to give it to me. OK. I spoke with Michael about you and M3 at Elego, but I haven't seen an answer from him, so I don't know if he got back to you. > >> 1) Olaf, are you intending to continue hosting the web pages? > > > > Elego can do still do that if no one else volunteers. I cannot > > guarantee more that a couple of minutes administration time per > > week though. We're rather short of ressources currently. > > > >> 2) How can I update these? There appear to be copies of some/many of them > >> in the cm3 repository, but these are not served from there as web pages, > >> AFAIK, [...] > I can log in to birch. Poking around, I find that > birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net > are the same machine. I find I also have a password on this machine as well. Is > this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It > is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch > to allow me to access the same machine would obviate needing this password. Yes, it's all the same VM currently, but it will probably be migrated to other servers and services separated. Yes, you can install any new key you like there. > > The shipping scripts can (could?) be found in the packages: > > > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www > > ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc > > I also found copies of these is a number of places, most significanly, > > /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... > > The scripts are the same, but presumably other stuff is more up-to-date here. > > There are a _lot_ of differences in this directory that are not shipped to the web > directory. I don't feel like shipping any of it without careful looking, to be > sure it is what we want on the web. Hm, I cannot really explain these offhand. Perhaps its best to ship to a sibling directory and then do a diff -r -b -B old new. > > I'm afraid that the Hudson installation on our server is out of order > > for quiet a long time. I don't know of any adaptation even for the > > source checkout from Github. > > > > I think some other public server should be used now for builds and > > tests. The old scripts could be adapted, but most people prefer to > > do these things in their own way. I used shell in 2010, but many > > scripts have been rewritten in python after that. > > Does anybody have a server to suggest here? I think a couple of my own machines > would have enough storage, RAM, CPU, and possibly network capacity for the builds > themselves if run only in overnight hours. But I definitely do not have the > networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe > LINUXLIBC6. I don't really know one offhand now. As far as I know Github does not have it's own build service. But Mike has suggested that if you don't find a suitable service, we can continue to host the CI server. But he wants to abandon the old and outdated Hudson and install a current Jenkins system. That would be much easier to maintain, too. > > You could transfer these jobs to another Hudson or Jenkins system > > (the config.xml files should suffice for that), adapt the source > > checkout to the Github repository and try to get some builds > > running. If you really plan to do that, I offer to help with > > all the advice and knowledge I have by answering your questions. > > With some help, I could take this on. I have looked briefly at hudson/jenkins. > It looks like there is enough info there to get things going. Look at Jenkins, this is the current free open source variant. > The most daunting part for me is mediocre network access. My latency has recently > improved enough so that command-line character echo has gone from just about > unusable to marginally usable. That's annoying of course. Excuse my curiosity: where are you located? Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri Mar 6 22:38:06 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 06 Mar 2015 15:38:06 -0600 Subject: [M3devel] Some M3 maintenance questions In-Reply-To: <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> References: <54F0BD9E.2070901@lcwb.coop> <20150227224609.294ba8eb28f437704eddca44@elegosoft.com> <54F3A01B.8000207@lcwb.coop> <20150305155805.6a9381e7fc566c266029eb6d@elegosoft.com> Message-ID: <54FA1E3E.8020804@lcwb.coop> On 03/05/2015 08:58 AM, Olaf Wagner wrote: > On Sun, 01 Mar 2015 17:26:19 -0600 > "Rodney M. Bates" wrote: > >> I don't seem to have access to cm3-bugs.elegosoft.com, to update trac. >> I have asked michael? to give it to me. > > OK. I spoke with Michael about you and M3 at Elego, but I haven't > seen an answer from him, so I don't know if he got back to you. > Not yet. >>>> 1) Olaf, are you intending to continue hosting the web pages? >>> >>> Elego can do still do that if no one else volunteers. I cannot >>> guarantee more that a couple of minutes administration time per >>> week though. We're rather short of ressources currently. >>> >>>> 2) How can I update these? There appear to be copies of some/many of them >>>> in the cm3 repository, but these are not served from there as web pages, >>>> AFAIK, > [...] > >> I can log in to birch. Poking around, I find that >> birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net >> are the same machine. I find I also have a password on this machine as well. Is >> this the way you want it? I had to supply it to do a dry run of ship-cm3-www. It >> is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch >> to allow me to access the same machine would obviate needing this password. > > Yes, it's all the same VM currently, but it will probably be migrated > to other servers and services separated. > > Yes, you can install any new key you like there. Done. I also changed the password. Do you want me to delete password login altogether? > >>> The shipping scripts can (could?) be found in the packages: >>> >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www >>> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc >> >> I also found copies of these is a number of places, most significanly, >> >> /home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/... >> >> The scripts are the same, but presumably other stuff is more up-to-date here. >> >> There are a _lot_ of differences in this directory that are not shipped to the web >> directory. I don't feel like shipping any of it without careful looking, to be >> sure it is what we want on the web. > > Hm, I cannot really explain these offhand. Perhaps its best to ship > to a sibling directory and then do a diff -r -b -B old new. > There is not enough disk space to create a sibling directory, but I have been diffing directly between the likely place to ship from and the shipped web files. Looking at just the first file that differs (about-cm3.html), I find the web version is CVS version 1.14.2.2, in branch release_CM3_5_8_6, and dated 2010-04-29, whereas the head and just about everywhere I look is version 1.14, dated 2007-05-15. So the release branch versions are not merged back to the head. I will work on figuring out how many files this applies to and get them merged into the git head, as appropriate. >>> I'm afraid that the Hudson installation on our server is out of order >>> for quiet a long time. I don't know of any adaptation even for the >>> source checkout from Github. >>> >>> I think some other public server should be used now for builds and >>> tests. The old scripts could be adapted, but most people prefer to >>> do these things in their own way. I used shell in 2010, but many >>> scripts have been rewritten in python after that. >> >> Does anybody have a server to suggest here? I think a couple of my own machines >> would have enough storage, RAM, CPU, and possibly network capacity for the builds >> themselves if run only in overnight hours. But I definitely do not have the >> networking capacity for the web pages. This would be for on AMD64_LINUX, and maybe >> LINUXLIBC6. > > I don't really know one offhand now. As far as I know Github does not > have it's own build service. But Mike has suggested that if you don't > find a suitable service, we can continue to host the CI server. But > he wants to abandon the old and outdated Hudson and install a current > Jenkins system. That would be much easier to maintain, too. > >>> You could transfer these jobs to another Hudson or Jenkins system >>> (the config.xml files should suffice for that), adapt the source >>> checkout to the Github repository and try to get some builds >>> running. If you really plan to do that, I offer to help with >>> all the advice and knowledge I have by answering your questions. >> >> With some help, I could take this on. I have looked briefly at hudson/jenkins. >> It looks like there is enough info there to get things going. > > Look at Jenkins, this is the current free open source variant. > I've started looking at it. I has some hudson->jenkins conversion instructions. >> The most daunting part for me is mediocre network access. My latency has recently >> improved enough so that command-line character echo has gone from just about >> unusable to marginally usable. > > That's annoying of course. Excuse my curiosity: where are you located? Tallgrass prairie, in the flint hills of eastern Kansas, where I have wanted to live for decades. But I am just barely out of reach of two land-line and one ground-level radio internet services, in one case, by just 200 feet, and by regulatory, not technical limitations. Fortunately, my satellite ISP has recently improved the round-trip latency from ~1.5 sec to ~.8 sec. which is just marginally usable for command-line access. > > Olaf > -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Mar 18 08:06:49 2015 From: jay.krell at cornell.edu (Jay K) Date: Wed, 18 Mar 2015 07:06:49 +0000 Subject: [M3devel] access to github/modula3/cm3? Message-ID: Do I have write access to github modula3/cm3? I don't think so. Can I be added? i.e. we are allow direct writes, w/o going through pull requests? I have small changes to get started: 1) fix m3-demo/mentor/src/sorting/m3makefile for bootstrapping from last release 2) fix m3-sys/mklib/main.m3 similarly, by cloning everything it uses from m3core/winnt.i3 3) copy targets.txt to .gitignore everywhere Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Mar 21 01:10:44 2015 From: jay.krell at cornell.edu (Jay K) Date: Sat, 21 Mar 2015 00:10:44 +0000 Subject: [M3devel] propose "output root" to replace shipping Message-ID: Building the current system fails in caltech-parser\parserlib\parserlib\test. It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. But even if shipped exists, it likely shouldn't be using it. This is a general problem we have been through somewhat before. There are multiple parts to our current partial solution: They can be seen here: C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl And those various "build tools" use build_standalone() This could imho be abstracted. However, I propose where today we have: source_root/m3-foo/bar/target/abc.obj source_root/m3-foo/bar/target/bar.exe source_root/m3-foo/bar/src/abc.m3 we instead have: output_root/obj/bar/target/abc.obj output_root/bin/target/bar.exe output_root/bin/bar.exe link or stub to target/abc.exe output_root/pkg/bar/... or: output_root_obj/bar/target/abc.obj output_root_install/bin/target/bar.exe output_root_install/bin/bar.exe link or stub to target/abc.exe output_root_install/pkg/bar/... and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it just "script", not anything in cm3, and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't use standalone for this, and on systems that don't support "origin", still use standalone and then shipping goes away, as a package operation. you can instead ship an output root. and then this override stuff goes away also; You bootstrap by copying in a few working files. Like make-dist.py does. The system is reduced in size, as build_standalone falls away -- you can still use it, but it won't be used usually. The larger source tree would become readonly, as it should be. Thoughts? I believe Modula-3 was originally ahead of its time in having the readonly src directories, but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which might as well not say "src", but preserve that instead of rearranging everything. I know, I've heard, having the separate buildlocal / buildglobal is useful. This doesn't elimine it that. It replaces it with "multiple buildglobal". Instead of having just the two options -- local and global -- you can create an arbitrary number of "global" scopes by creating a new output_root, copying it from a preexisting one. As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. This has several nice properties. readonly source tree less file copying possibly (depending on if final scripted ship is directory copy or move) cm3 can stop implementing ship no need to use build_standalone on most systems, including for cm3 itself (but it still works) - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sun Mar 22 17:57:35 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 22 Mar 2015 11:57:35 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: Message-ID: <550EF47F.7080006@lcwb.coop> I haven't had time to digest your proposal yet, but I have often been uncomfortable about the current build system when building/bootstrapping itself. It is probably just lack of understanding of the right procedure, but I usually cringe before attempting to bootstrap. I have always been unclear about the sequence of when the newly built components overlay the preexisting ones, so I would welcome some attention to this. I would like to put in a vote for a way to build everything without overlaying any existing components, so a bootstrapping operation can be easily restarted back where it started. Right now, for the compiler itself, I am meticulously saving side copies of the compiler executables in /usr/local/cm3/bin every time. On 03/20/2015 07:10 PM, Jay K wrote: > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > But even if shipped exists, it likely shouldn't be using it. > > > > This is a general problem we have been through somewhat before. > > > > There are multiple parts to our current partial solution: > > > They can be seen here: > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > And those various "build tools" use build_standalone() > > > This could imho be abstracted. > > > However, I propose where today we have: > source_root/m3-foo/bar/target/abc.obj > source_root/m3-foo/bar/target/bar.exe > source_root/m3-foo/bar/src/abc.m3 > > > we instead have: > output_root/obj/bar/target/abc.obj > output_root/bin/target/bar.exe > output_root/bin/bar.exe link or stub to target/abc.exe > output_root/pkg/bar/... > > > or: > output_root_obj/bar/target/abc.obj > output_root_install/bin/target/bar.exe > output_root_install/bin/bar.exe link or stub to target/abc.exe > output_root_install/pkg/bar/... > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > just "script", not anything in cm3, > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > use standalone for this, and on systems that don't support "origin", still use standalone > > > and then shipping goes away, as a package operation. > you can instead ship an output root. > > > and then this override stuff goes away also; > You bootstrap by copying in a few working files. Like make-dist.py does. > The system is reduced in size, as build_standalone falls away -- you can still use it, > but it won't be used usually. > > > The larger source tree would become readonly, as it should be. > > > Thoughts? > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > might as well not say "src", but preserve that instead of rearranging everything. > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > This doesn't elimine it that. > It replaces it with "multiple buildglobal". > Instead of having just the two options -- local and global -- you can create an arbitrary number of > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > This has several nice properties. > readonly source tree > less file copying possibly (depending on if final scripted ship is directory copy or move) > cm3 can stop implementing ship > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > - Jay -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Tue Mar 24 09:19:06 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:19:06 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <550EF47F.7080006@lcwb.coop> References: , <550EF47F.7080006@lcwb.coop> Message-ID: The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Mar 24 09:29:19 2015 From: jay.krell at cornell.edu (Jay K) Date: Tue, 24 Mar 2015 08:29:19 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmuysers at hotmail.com Tue Mar 24 10:01:39 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 10:01:39 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: Has anybody ever considered to use Zero Install for shipping and distribution? From: Jay K Sent: Tuesday, March 24, 2015 9:29 AM To: rodney.m.bates at acm.org ; m3devel Subject: Re: [M3devel] propose "output root" to replace shipping There is a problem here. Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them. The problem .. I'm alluding to two solutions. 1) "run from" and "install to" are the same 2) or different But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). #1 only works if the old and new are compatible-enough. #1 is more automatic I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler. We might be able to achieve all of this by setting BUILD_DIR to an absolute path. But I think I really want it driven by environment variables and not config files. Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree. I did a lot of it already, years ago now, but some remains. And then replace that system, is my proposal, with "override" not doing/meaning anything ever. One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship". Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"? You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root. - Jay -------------------------------------------------------------------------------- From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: [M3devel] propose "output root" to replace shipping Date: Tue, 24 Mar 2015 08:19:06 +0000 The point is, kind of to make it better, by making it worse. Or worse by making it better. In particular, "ship" would go away. Files would always be overwritten right away. If you don't want this -- and indeed sometimes you don't, you would change the output root to a new empty/nonexistent directory, possibly copying all of the previous into the new. cm3 and its dependencies would require special attention. In particular, for sharing violations on .dlls, we'd rename away and copy back. Does that then work on all systems? It works on NT. I'm betting most Posix systems "just work" and AIX requires either the same as NT or can't be made to work so easily. If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and basically just them). For anything not used by cm3, just edit the source and recompile again. You could view this as taking away an escape hatch. But a simpler easier to understand escape hatch remains. And the escape hatch is really only critical for cm3 and its dependents. Or "build tools" maybe more generally, like klex, kyacc, m3bundle. Is this still the mailing list to use, or something new with git? Now -- another use of "ship" is "install" not on the same machine as compile. For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. A background point here is -- have fewer different directory layouts. Instead of ship rearranging stuff, just arrange stuff when you link in the first place. This would also argue that the source tree and install tree should be more-similar/identical except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" leaves by one, in order to keep the tree diffable against old trees. - Jay > Date: Sun, 22 Mar 2015 11:57:35 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > I haven't had time to digest your proposal yet, but I have often been uncomfortable > about the current build system when building/bootstrapping itself. It is probably > just lack of understanding of the right procedure, but I usually cringe before attempting > to bootstrap. I have always been unclear about the sequence of when the newly built > components overlay the preexisting ones, so I would welcome some attention to this. > > I would like to put in a vote for a way to build everything without overlaying > any existing components, so a bootstrapping operation can be easily restarted back > where it started. Right now, for the compiler itself, I am meticulously saving side > copies of the compiler executables in /usr/local/cm3/bin every time. > > On 03/20/2015 07:10 PM, Jay K wrote: > > Building the current system fails in caltech-parser\parserlib\parserlib\test. > > > > > > > > It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things. > > But even if shipped exists, it likely shouldn't be using it. > > > > > > > > This is a general problem we have been through somewhat before. > > > > > > > > There are multiple parts to our current partial solution: > > > > > > They can be seen here: > > C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl > > C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl > > C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl > > C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl > > > > > > And those various "build tools" use build_standalone() > > > > > > This could imho be abstracted. > > > > > > However, I propose where today we have: > > source_root/m3-foo/bar/target/abc.obj > > source_root/m3-foo/bar/target/bar.exe > > source_root/m3-foo/bar/src/abc.m3 > > > > > > we instead have: > > output_root/obj/bar/target/abc.obj > > output_root/bin/target/bar.exe > > output_root/bin/bar.exe link or stub to target/abc.exe > > output_root/pkg/bar/... > > > > > > or: > > output_root_obj/bar/target/abc.obj > > output_root_install/bin/target/bar.exe > > output_root_install/bin/bar.exe link or stub to target/abc.exe > > output_root_install/pkg/bar/... > > > > > > and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it > > just "script", not anything in cm3, > > > > > > and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't > > use standalone for this, and on systems that don't support "origin", still use standalone > > > > > > and then shipping goes away, as a package operation. > > you can instead ship an output root. > > > > > > and then this override stuff goes away also; > > You bootstrap by copying in a few working files. Like make-dist.py does. > > The system is reduced in size, as build_standalone falls away -- you can still use it, > > but it won't be used usually. > > > > > > The larger source tree would become readonly, as it should be. > > > > > > Thoughts? > > > > > > I believe Modula-3 was originally ahead of its time in having the readonly src directories, > > but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which > > might as well not say "src", but preserve that instead of rearranging everything. > > > > > > I know, I've heard, having the separate buildlocal / buildglobal is useful. > > This doesn't elimine it that. > > It replaces it with "multiple buildglobal". > > Instead of having just the two options -- local and global -- you can create an arbitrary number of > > "global" scopes by creating a new output_root, copying it from a preexisting one. > > > > > > As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR. > > But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET. > > > > > > This has several nice properties. > > readonly source tree > > less file copying possibly (depending on if final scripted ship is directory copy or move) > > cm3 can stop implementing ship > > no need to use build_standalone on most systems, including for cm3 itself (but it still works) > > > > > > > > - Jay > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 11:22:01 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 10:22:01 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> Message-ID: <20150324102200.GA2982@zoho.com> On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From dmuysers at hotmail.com Tue Mar 24 12:08:13 2015 From: dmuysers at hotmail.com (dirk muysers) Date: Tue, 24 Mar 2015 12:08:13 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: <20150324102200.GA2982@zoho.com> References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: I doubt that you haven't given a look at 0install, because it cares well about security concerns. One of the problems with Windows 7 is that, if you install cm3 into the programs folder (where it should naturally belong to), you can't even create package directories or modify your code because of the very strict UAC that enforces immutability for everything belonging to a program. -----Original Message----- From: microcode at zoho.com Sent: Tuesday, March 24, 2015 11:22 AM To: m3devel at elegosoft.com Subject: Re: [M3devel] propose "output root" to replace shipping On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > Has anybody ever considered to use Zero Install for shipping and > distribution? Requiring infrastructure to "install" something is a good way to kill off any remining interest. Please, keep the requirements to an absolute minimum and don't expect people to add tools or security holes to their OS except for Windows where that isn't a concern. Other platforms mostly don't expect plug and play installations nor do they want them. From estellnb at elstel.org Tue Mar 24 18:25:07 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 24 Mar 2015 18:25:07 +0100 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop>, Message-ID: <55119DF3.8000803@elstel.org> Am 24.03.15 um 09:29 schrieb Jay K: > There is a problem here. > Something I also want to do is stop building things standalone, > on NT and systems that support "origin", which is pretty much all of them. > The problem .. I'm alluding to two solutions. > > > 1) "run from" and "install to" are the same > 2) or different > > > But we also want "atomic" update of cm3 and its dependencies > (m3core/libm3). > #1 only works if the old and new are compatible-enough. > #1 is more automatic > > > I think the answer is, if you are rebuilding m3core/libm3/cm3, and the > old and new are not compatible-enough, you must set the output to be a > new empty directory, and "run from" and "install to" must be > different. If you are building almost anything else, you can safely > update in place w/o breaking the compiler. > Dear Jay K. Yes, I had been thinking the same way as you those times when I hacked pm3 and cm3. A solution for it needed to be still around somewhere on my hard disk. However I remember it was not so easy to achieve this as there are mutual dependencies between the front end and the core library. If there are incompatibilities between both you may need an intermediate code version that ships b but still uses a, followed by a version that uses b and ships b. Nonetheless the approach works fine as long as no such dependencies are given. Unfortunately I currently do not have the time to dig it out for you; next month or more likely just in two month I can perhaps do that if you are interested. It would be great to have such a feature in the main tree (i.e ship to B and compile from A) though getting the changes there will be somewhat beyond my scope. Cheers, Elmar -------------- next part -------------- An HTML attachment was scrubbed... URL: From microcode at zoho.com Tue Mar 24 19:37:23 2015 From: microcode at zoho.com (microcode at zoho.com) Date: Tue, 24 Mar 2015 18:37:23 +0000 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: <550EF47F.7080006@lcwb.coop> <20150324102200.GA2982@zoho.com> Message-ID: <20150324183723.GA3046@zoho.com> I already said I don't care what happens with Windows. That platform is and has always been riddled with security holes like Swiss cheese and 99.9% of Windows victims need an installer anyway. It's not whether they need one it's just which one is least bad. Perhaps you are not aware that every application you add to your system is another vector for vulnerabilities and bugs. Perhaps you are also not aware people running UNIX or UNIX-like OS really are not interested and in most cases will simply not use anything that has any installation requirements beyond a normal build or untarring a tarball. I don't tolerate installers running as root downloading random crap from the web. I download what I want myself from the correct site and I check the signatures or checksums if provided. Then I build the application from source. Then I check it. If it looks ok then I install it. I don't think it's appropriate to burden non-Windows users with potential malware just for the sake of Windows users. We are not accustomed to keep clicking OK until the system reboots. On Tue, Mar 24, 2015 at 12:08:13PM +0100, dirk muysers wrote: > I doubt that you haven't given a look at 0install, because it cares well > about security concerns. One of the problems with Windows 7 is that, > if you install cm3 into the programs folder (where it should naturally > belong to), you can't even create package directories or modify your code > because of the very strict UAC that enforces immutability for everything > belonging to a program. > > -----Original Message----- From: microcode at zoho.com > Sent: Tuesday, March 24, 2015 11:22 AM > To: m3devel at elegosoft.com > Subject: Re: [M3devel] propose "output root" to replace shipping > > On Tue, Mar 24, 2015 at 10:01:39AM +0100, dirk muysers wrote: > >Has anybody ever considered to use Zero Install for shipping and > >distribution? > > Requiring infrastructure to "install" something is a good way to kill off > any remining interest. Please, keep the requirements to an absolute minimum > and don't expect people to add tools or security holes to their OS except > for Windows where that isn't a concern. Other platforms mostly don't expect > plug and play installations nor do they want them. > > -- From rodney_bates at lcwb.coop Sat Mar 28 16:12:34 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:12:34 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C4E2.9010701@lcwb.coop> This has suddenly reminded me of a problem and solution from the past. I once did a lot of work in and on a package containing several source code generators for components of compilers and such (Cocktail, if anybody cares). It was highly cyclic, several of the generators being used by several of themselves to generate parts of their own code. One day, after introducing a simple bug, I ended up with no working executable needed to regenerate itself after I fixed the bug. I don't remember how I got out of this, but it took a lot of work and it was tedious and very hacky. Afterwards, I spent some time setting up a new build environment. There could be any number of what I called "views" (named after a different development system from mars), each a complete, more-or-less copy of everything: hand edited source, generated source, compiled objects, executables. The build scripts used a symlink, found at a fixed place within a view, to access executables to be used in building. The view-cloning script set up this link in the newly cloned view to point to the executables in the old view. So each view was built using unmodified executables from the older view it was cloned from. The use process was to always keep a view named "stable", well tested to be able to rebuild itself. I would clone a new view named "working" and do development in that. Periodically, I would clone "phase1" from working, build in phase1 (which used executables in working), clone "phase2" from phase1, and build in phase2. If the sources in working could regenerate and rebuild themselves twice in this way, I considered it good evidence to make phase2 become the new stable. I don't remember the details, but I also had some scripted diffs between equivalent generated source files in different views, to show the regeneration process had converged to a fixed point. As I recall, there were often legitimate differences that had to be manually reviewed, but that was just a good test of the most recent changes. After that things worked very smoothly. It did take up quite a bit of disk space, with multiple copies of everything, but it was still feasible, even in days when 6 GB was a big disk. We don't have as much circularity in Modula-3, but there is some, and even a tiny bit can cause a lot of grief. So I suggest something like this. Installing from a view to a system-wide installation location would be a separate step. It would be greatly simplified if, as Jay suggested, the as-built and as-installed subdirectory structures could be made the same. On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay >-- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:15:03 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:15:03 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516C577.1000503@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. I really like this idea. Things might need to be slightly different when it comes to installed copies of library interfaces. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Sat Mar 28 16:59:02 2015 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 28 Mar 2015 10:59:02 -0500 Subject: [M3devel] propose "output root" to replace shipping In-Reply-To: References: , <550EF47F.7080006@lcwb.coop> Message-ID: <5516CFC6.6040504@lcwb.coop> On 03/24/2015 03:19 AM, Jay K wrote: > The point is, kind of to make it better, by making it worse. > Or worse by making it better. > > > In particular, "ship" would go away. > Files would /always be overwritten right away/. > > > If you don't want this -- and indeed /sometimes /you don't, you > would change the output root to a new empty/nonexistent directory, > possibly copying all of the previous into the new. > > > > cm3 and its dependencies would require special attention. > In particular, for sharing violations on .dlls, we'd rename away and copy back. > Does that then work on all systems? It works on NT. > I'm betting most Posix systems "just work" and AIX requires either the same as NT > or can't be made to work so easily. > > > If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and > basically just them). For anything not used by cm3, just edit the source and recompile again. > > > You could view this as taking away an escape hatch. > But a simpler easier to understand escape hatch remains. > > > And the escape hatch is really only critical for cm3 and its dependents. > Or "build tools" maybe more generally, like klex, kyacc, m3bundle. > > > Is this still the mailing list to use, or something new with git? I think this is still the list to use. I found, with a bit of difficulty, how to get git to send emails about git events, and set it up to send to me. Perhaps we would want to set these to send to m3commit? > > > Now -- another use of "ship" is "install" not on the same machine as compile. > For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived. > > > A background point here is -- have fewer different directory layouts. > Instead of ship rearranging stuff, just arrange stuff when you link in the first place. > > > This would also argue that the source tree and install tree should be more-similar/identical > except for the root, but I'm reluctant to rearrange the source..even to lift up the "src" > leaves by one, in order to keep the tree diffable against old trees. > > > - Jay > > >-- Rodney Bates rodney.m.bates at acm.org From estellnb at elstel.org Tue Mar 31 15:40:38 2015 From: estellnb at elstel.org (Elmar Stellnberger) Date: Tue, 31 Mar 2015 15:40:38 +0200 Subject: [M3devel] existant backup data against losses from server crash Message-ID: <551AA3D6.1090805@elstel.org> Dear Modula-3 maintainers and developers; dear Elegosoft, Last year I have downloaded the complete modula3.elegosoft.com/cm3 page including several source and compiled versions of CM3. Today I have visited your page again and saw the message that many links were broken on your page due to data losses on a server crash. I believe that I still have the web image of your page somewhere around as downloaded by wget -R. It should allow the recovery of all lost release data which was available to the public. If you are interested I will burn some blue rays from it which I can send you via snail mail (uploading would be far too slow with my internet connection). Just let me know ... Yours Sincerely, Elmar Stellnberger