<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Here is the model to consider.<BR><br> <BR>Think of make install.<br>In gcc and glibc, is it special cased? Probably not.<br>Do they have an extra environment variable to turn it on/off? Probably not.<br>Is it fraught with problems like cm3? Probably.<br>Though DESTDIR is the safe way.<BR> <BR><br>The environment variable is a workaround for OS limitations, that may<br>or may not exist. If it might exist anywhere, then all OS'es are treated consistently.<br>If we can determine that it exists nowhere, then we can remove.<BR> <BR><br>The environment variable does not provide for shipping in the correct order.<br>The user is obligated to do that.<BR><br> <BR>Removing the environment variable from cm3 and cm3cg is meant to work.<br>You do this by splitting up build from ship.<br>Build everything, then ship everything.<BR>Since shipping doesn't use cm3cg.<BR> <BR><br>If there are problems, they aren't found till the results are partly commited<br>and must be backed out -- if you are shipping in-place. That is just the nature of<br>it. That is why make install for glibc is particularly dangerous.<BR> <BR><br>My upgrade.py's use of buildship is probably not the right approach.<br>That was probably a mistake and it is easy for me to fix.<br>So it won't end up mattering -- nobody will ever ship cm3 or cm3cg.<br>But when no incompatible change is being made, it is meant to just work.<br>When an incompatible change is made, more care is meant to be taken.<BR> <BR><br> - Jay<br><br><br><br> <BR><div><hr id="stopSpelling">From: jay.krell@cornell.edu<br>To: rodney.m.bates@acm.org<br>Date: Sat, 30 May 2015 16:24:17 +0000<br>CC: m3devel@elegosoft.com<br>Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110<br><br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr"> The variable is not for atomic update of multiple files.<br><br> The variable exists because updating in-use files reportedly<br> fails on some operating systems. Though I question the truth of even that.<br> If we change ship to rename away upon copy failure, and find a reliable<br> simple way to ensure eventual deletion of the renamed file, then we don't need this.<br> I'll have to experiment on NT. Presumably we've written off Win9x,<br> and I'm not sure about AIX.<br> <br>So consistency with cm3 isn't necessarily desirable/required.<br> <br><br> do-cm3-front.sh isn't necessarily meant to work when there is an IR change.<br> perhaps m3cc wasn't supposed to be in front. I'll have to double check that later.<br> upgrade.sh is meant to.<br> It doesn't ship anything and manually copies both cm3 and cm3cg.<br> So that still works.<br> <br><br> And upgrade.py handles it, but other way I think, so it doesn't work any longer.<br> Perhaps the upgrade.sh approach is better and perhaps I missed this when I rewrote it.<br> I can make it copy cm3cg manually.<br> <br><br>I haven't gotten a chance to fix anything yet.<br> - Jay<br><br><br> <br><div>> Date: Sat, 30 May 2015 09:58:31 -0500<br>> From: rodney_bates@lcwb.coop<br>> To: jay.krell@cornell.edu<br>> CC: m3devel@elegosoft.com<br>> Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110<br>> <br>> <br>> <br>> On 05/29/2015 05:31 PM, Jay K wrote:<br>> > cm3/cm3cg are more special than the others, because cm3 is statically linked.<br>> > There is no m3core.dll, that cm3 uses, or m3quake.dll, etc.<br>> > There is m3core.dll, but cm3 doesn't uses it.<br>> > Shipping it doesn't affect the in-use cm3.<br>> ><br>> ><br>> > If/when we fix this, in fact, we can stop doing that static linking.<br>> > It makes sense for m3quake, etc. as there are few/no other users.<br>> > But for libm3/m3core, it'll provide a nice size optimization.<br>> ><br>> ><br>> > The mutual incompatibilty between old/new cm3/cm3cg is perfectly ok.<br>> > It is meant to be dealt with, not avoided.<br>> ><br>> <br>> I agree it's OK. Disallowing it would overly hamstring future development.<br>> But dealing with it means we need to be able to prevent either package<br>> immediately after it is built.<br>> <br>> ><br>> > I believe upgrade.py did deal with this correctly, but this environment<br>> > variable required to quash shipping of cm3cg broke it.<br>> ><br>> ><br>> > The environment variable change really should imho be removed from cm3cg.<br>> > Please.<br>> ><br>> <br>> Jay, I am not sure what you are proposing. When I first looked at the commit for<br>> this change, I was quite confused by my own change. While the comments say it<br>> is to _not_ prematurely ship cm3cg, the code appears to add code to conditionally<br>> ship if INSTALL_CM3_IN_BIN=yes. After the n-th look, I realized that before<br>> the change, it _always_ shipped, regardless of the variable. So it really does<br>> provide for less premature shipping.<br>> <br>> If I just revert it, do-cm3-front.sh will be unconditionally and unusably broken.<br>> As is, it is at least usable if you don't set the variable. Even if there is<br>> some use case where one would want it to ship, (none occurs to me), as is, you<br>> can get this behavior by just setting the variable.<br>> <br>> Moreover, the current behavior for cm3cg is consistent with the current and<br>> long-standing behavior for cm3, i.e., ship iff the variable is set.<br>> <br>> It would make more sense to me to remove the variable check by unconditionally<br>> _not_ shipping. At least do-cm3-front.sh would be usable. But then, what<br>> would be the purpose of the inconsistent use of the variable for cm3?<br>> <br>> <br>> <br>> ><br>> > And then we can switch to "destdir" and remove all the in place updating, perhaps.<br>> ><br>> ><br>> > Or I need to update upgrade.py to ship cm3cg itself.<br>> > I guess I'll do that.<br>> ><br>> ><br>> > upgrade.py may not have worked for you, but I have used it many many times.<br>> > It worked. But currently it is probably broken by the m3cc/src/m3makefile change.<br>> > I'll update it to accommodate that change..<br>> ><br>> ><br>> > And upgrade.sh I believe was in heavy use by Hudson.<br>> ><br>> ><br>> > - Jay<br>> ><br>> ><br>> ><br>> > > Date: Fri, 29 May 2015 16:10:41 -0500<br>> > > From: rodney_bates@lcwb.coop<br>> > > CC: m3devel@elegosoft.com<br>> > > To:<br>> > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110<br>> > ><br>> > > Yes, we could have a better organization. I agree, the principal way<br>> > > should be to build a complete set of everything that matters to the<br>> > > implementation of the language, them replace them all at once. A<br>> > > separate DESTDIR or such is needed.<br>> > ><br>> > > The set, as Jay points out, is larger than just cm3 and m3cc. It includes<br>> > > at least m3core, probably libm3, and cm3, the executable depends on several<br>> > > separately built packages--cm3, m3middle, m3front, m3quake, m3makefile, etc.<br>> > ><br>> > > In general, we can't really assume anything about the kinds of dependencies<br>> > > of one package on old or new versions of another. Right now, cm3 and m3cc<br>> > > are known mutually incompatible between 5.8 and the head, because the numbering<br>> > > in M3CG_Binary.i3 has changed. We could probably establish some coding rules and<br>> > > order-of-install rules that would allow a newer m3cc to work with an older cm3,<br>> > > but that's only one pattern.<br>> > ><br>> > > I once thought do-cm3-*.sh build would do what we are talking about, but it<br>> > > if I remember right, puts every package into immediate use as it is built<br>> > > and also uses overrides, which keeps things inside the source tree, but makes<br>> > > it unshippable. So I always use buildship. This has never, to my knowledge<br>> > > run afoul of any compatibility problem other than between cm3 and m3cc, but it<br>> > > does ship every package besides these two as it is built, so the potential<br>> > > is there.<br>> > ><br>> > > I haven't had the energy to even try to understand just what the upgrade<br>> > > scripts do, but they have not treated me well. If I had done different<br>> > > patterns of incompatible changes, they would probably have been better<br>> > > than the procedure I use.<br>> > ><br>> > > On 05/29/2015 01:06 PM, Jay K wrote:<br>> > > > > I don't see an easy way to make the variable work, as there is no<br>> > > > > constraint on the order that cm3 and cm3cg are built. do-cm3-front.sh<br>> > > > > happens to do cm3cg first, but that is not necessarily required.<br>> > > ><br>> > > > [please pardon my editing...]<br>> > > ><br>> > > ><br>> > > > I see what you mean that cm3cg is an independent node in the graph and can be built in any order,<br>> > > > but, I guess, the story is/was, you have to know the safe order to ship things.<br>> > > ><br>> > > ><br>> > > > It is encoded in pkginfo.txt and the Python scripts honor it..though they don't always use pkginfo.txt for their ordering..<br>> > > > And encoding the dependency graph there isn't ideal. It is really encoded in the import declarations<br>> > > > in the m3makefiles, usually, but not in this case...<br>> > > ><br>> > > ><br>> > > > The way that works is to build but don't ship cm3cg.<br>> > > > Then ship cm3cg.<br>> > > > Then copy cm3.<br>> > > > Then use the new cm3.<br>> > > > But perhaps it isn't strictly enforced enough.<br>> > > ><br>> > > ><br>> > > ><br>> > > > I used do this fairly often.<br>> > > > Though breaking changes in cm3cg were rare, granted.<br>> > > > But I'd also occasionally bootstrap from an older release.<br>> > > > I'll have to try to do that again soon -- actually make the current<br>> > > > tree easily buildable from the last release.<br>> > > ><br>> > > ><br>> > > > update.py handled it.<br>> > > ><br>> > > ><br>> > > > I'm on the fence.<br>> > > > "buildship" of multiple packages is in general problematic.<br>> > > > I kinda think the whole mechanism needs to be revampled.<br>> > > ><br>> > > ><br>> > > > Let me draw a close analogy.<br>> > > ><br>> > > ><br>> > > > In the slightly larger software world we have:<br>> > > ><br>> > > > make<br>> > > > make install<br>> > > > make install DESTDIR=somewhere_new<br>> > > ><br>> > > ><br>> > > > buildship is equivalent make && make install, or a make install that includes the make part,<br>> > > > as they sometimes do.<br>> > > > Read up on building glibc and they point out how dangerous this is -- how dangerous make install is w/o DESTDIR.<br>> > > > We have the same danger on a smaller scale.<br>> > > > You aren't going to break the "OS", but you can break "cm3".<br>> > > ><br>> > > ><br>> > > > The safe recommendation is make install DESTDIR.<br>> > > > Basically "install" "everything" "on the side", get it all installed, multiple<br>> > > > "packages", then copy or tar/untar it all. Or chroot into it and test it out, and<br>> > > > then do an in-place update. Or swap into the chroot and never return? (tail call install)<br>> > > ><br>> > > ><br>> > > > Don't do a "rolling upgrade". Don't install each package as it is built. No package, no cm3,<br>> > > > not cm3cg, nothing.<br>> > > ><br>> > > ><br>> > > > I'm basically suggesting that.<br>> > > ><br>> > > ><br>> > > > Now, if you say "don't ever ship, but somehow otherwise manually copy", well<br>> > > > that does kind of resolve to the same thing.<br>> > > ><br>> > > ><br>> > > > If you say, "almost don't ever ship, but provide a way to", that is almost the same.<br>> > > ><br>> > > ><br>> > > > make-dist.py follows the make install DESTDIR way.<br>> > > ><br>> > > > It is "better" I guess because I wrote it later, and because I was following<br>> > > > the example of the other .sh files that make distributions. So blame/credit Olaf? :)<br>> > > ><br>> > > ><br>> > > > It starts with a working compiler, copies parts into /temp, builds a new compiler into /tmp, and<br>> > > > uses what it builds to build everything.<br>> > > ><br>> > > ><br>> > > > Again I suggest it be looked into, and maybe be promoted as basically<br>> > > > the replacement for upgrade.py and upgrade.sh.<br>> > > ><br>> > > ><br>> > > > upgrade.py could wrap it and do the final steps of copy/untar in-place.<br>> > > > Or make-dist.py could just print the last optional command.<br>> > > ><br>> > > ><br>> > > > Anyway, I have no time lately, just email. :(<br>> > > ><br>> > > ><br>> > > > - Jay<br>> > > ><br>> > > ><br>> > > ><br>> > > > > Date: Fri, 29 May 2015 11:35:41 -0500<br>> > > > > From: rodney_bates@lcwb.coop<br>> > > > > To: adacore@marino.st; wagner@elego.de<br>> > > > > CC: m3devel@elegosoft.com<br>> > > > > Subject: Re: [M3devel] m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110<br>> > > > ><br>> > > > > Setting INSTALL_CM3_IN_BIN=yes is what forces the premature install of cm3cg.<br>> > > > > I see I had encountered this problem two years ago, but then it was happening<br>> > > > > regardless of the value of this variable. My fix only addresses the case when<br>> > > > > the variable is not set. I naively went to extra trouble to preserve the<br>> > > > > existing behavior when it is set--obviously not right.<br>> > > > ><br>> > > > > I don't see an easy way to make the variable work, as there is no<br>> > > > > constraint on the order that cm3 and cm3cg are built. do-cm3-front.sh<br>> > > > > happens to do cm3cg first, but that is not necessarily required.<br>> > > > ><br>> > > > > So INSTALL_CM3_IN_BIN is broken and hard to fix, if the two need<br>> > > > > compatible upgrades. For now, it looks like install-cm3-compiler.sh<br>> > > > > is the only working way.<br>> > > > ><br>> > > > > But maybe John and Olaf have found a different way around this.<br>> > > > ><br>> > > > > On 05/29/2015 10:55 AM, Rodney M. Bates wrote:<br>> > > > > > I don't know yet why, or what needs to change, but here is definitely the problem, starting<br>> > > > > > in line 4575 of your m3c log:<br>> > > > > ><br>> > > > > ><br>> > > > > > g++48 -O2 -pipe -fstack-protector -Wl,-rpath=/usr/local/lib/gcc48 -fno-strict-aliasing -Wl,-rpath=/usr/local/lib/gcc48 -DIN_GCC -Wno-missing-field-initializers -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -DHAVE_CONFIG_H -o m3cgc1 m3cg/parse.o attribs.o main.o tree-browser.o libbackend.a libcommon-target.a libcommon.a ../libcpp/libcpp.a libcommon.a ../libcpp/libcpp.a ../libiberty/libiberty.a<br>> > > > > > gmake[1]: Leaving directory `/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a/m3-sys/m3cc/AMD64_FREEBSD/gcc'<br>> > > > > ><br>> > > > > ><br>> > > > > > This is copying the newly-built cm3cg (the compiler back end) to your<br>> > > > > > bin directory:<br>> > > > > ><br>> > > > > > Forced ship of cm3cg.<br>> > > > > > --- shipping from AMD64_FREEBSD ---<br>> > > > > ><br>> > > > > > . => /mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin<br>> > > > > > cm3cg<br>> > > > > > ==> m3-sys/m3cc done<br>> > > > > ><br>> > > > > > === package m3-libs/m3core ===<br>> > > > > ><br>> > > > > > And this is using the cm3 (compiler front end) that was already in the bin<br>> > > > > > directory, and is the release version.<br>> > > > > ><br>> > > > > > +++ /mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 -build -DROOT='/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a' $RARGS && /mech/construction/mech/ptrees/default/lang/modula3/work/bootstrap/bin/cm3 -ship $RARGS -DROOT='/mech/construction/mech/ptrees/default/lang/modula3/work/cm3-8c1b86a' +++<br>> > > > > > --- building in AMD64_FREEBSD ---<br>> > > > > ><br>> > > > > > ignoring ../src/m3overrides<br>> > > > > ><br>> > > > > > new source -> compiling RTHooks.i3<br>> > > > > ><br>> > > > > > This is cm3cg (aka m3cgc1) detecting the incompatibility:<br>> > > > > ><br>> > > > > > m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110<br>> > > > > > compilation terminated.<br>> > > > > ><br>> > > > > > Here is the counterpart section of my log:<br>> > > > > ><br>> > > > > > g++ -g -O2 -DIN_GCC -Wno-missing-field-initializers -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -DHAVE_CONFIG_H -o m3cgc1 m3cg/parse.o attribs.o main.o tree-browser.o libbackend.a libcommon-target.a libcommon.a ../libcpp/libcpp.a libcommon.a ../libcpp/libcpp.a ../libiberty/libiberty.a<br>> > > > > ><br>> > > > > > On my build, it's not shipping:<br>> > > > > ><br>> > > > > > Not shipping cm3cg.<br>> > > > > > --- shipping from AMD64_LINUX ---<br>> > > > > ><br>> > > > > > missing ".M3SHIP" file, build the package first.<br>> > > > > > ==> /home/rodney/proj/m3/git/cm3/m3-sys/m3cc done<br>> > > > > ><br>> > > > > > === package /home/rodney/proj/m3/git/cm3/m3-libs/m3core ===<br>> > > > > > +++ cm3 -build -DROOT='/home/rodney/proj/m3/git/cm3' $RARGS && cm3 -ship $RARGS -DROOT='/home/rodney/proj/m3/git/cm3' +++<br>> > > > > > --- building in AMD64_LINUX ---<br>> > > > > ><br>> > > > > > ignoring ../src/m3overrides<br>> > > > > ><br>> > > > > > new source -> compiling RTHooks.i3<br>> > > > > > new source -> compiling RT0.i3<br>> > > > > ><br>> > > > > > When it works, the copying of both cm3 and cm3cg to the bin directory is delayed until after both have been<br>> > > > > > built, in the 'install-cm3-compiler.sh upgrade' step.<br>> > > > > ><br>> > > > > > I see your build is using a full path to the bin directory and mine is letting<br>> > > > > > it come from the usual $PATH search, but that should make no difference, as long<br>> > > > > > as the bin directory is found in a consistent way, which seems to be the case<br>> > > > > > in both builds.<br>> > > > > ><br>> > > > > ><br>> > > > > ><br>> > > > > > On 05/29/2015 04:51 AM, John Marino wrote:<br>> > > > > >> On 5/29/2015 11:39, Olaf Wagner wrote:<br>> > > > > >>> On Fri, 29 May 2015 11:10:50 +0200<br>> > > > > >>> John Marino <adacore@marino.st> wrote:<br>> > > > > >>><br>> > > > > >>>> On 5/29/2015 10:54, Olaf Wagner wrote:<br>> > > > > >>>>> Please try what I suggested in the attached mail: define<br>> > > > > >>>>> INSTALL_CM3_IN_BIN=yes<br>> > > > > >>>>> in the environment and see if that helps.<br>> > > > > >>>>><br>> > > > > >>>><br>> > > > > >>>> Unfortunately it seems the result is exactly the same, even when<br>> > > > > >>>> INSTALL_CM3_IN_BIN is defined in the environment.<br>> > > > > >>><br>> > > > > >>> Can I see the new logs? Has the new backend (cm3cg) been installed now?<br>> > > > > >>><br>> > > > > >><br>> > > > > >> It's here:<br>> > > > > >> http://leaf.dragonflybsd.org/~marino/m3c.log<br>> > > > > >><br>> > > > > >> It never completes "do-cm3-front.sh buildship "<br>> > > > > >><br>> > > > > >> Note that I'm using a bootstrap install and set INSTALL_ROOT to another<br>> > > > > >> location, but it never gets to the point of installing anything.<br>> > > > > >><br>> > > > > >> John<br>> > > > > >><br>> > > > > ><br>> > > > ><br>> > > > > --<br>> > > > > Rodney Bates<br>> > > > > rodney.m.bates@acm.org<br>> > ><br>> > > --<br>> > > Rodney Bates<br>> > > rodney.m.bates@acm.org<br>> <br>> -- <br>> Rodney Bates<br>> rodney.m.bates@acm.org<br></div> </div></div> </div></body>
</html>