[M3devel] propose "output root" to replace shipping

Tue Mar 24 09:29:19 CET 2015

There is a problem here.
Something I also want to do is stop building things standalone, on NT and systems that support "origin", which is pretty much all of them.
The problem .. I'm alluding to two solutions.

 1) "run from" and "install to" are the same 
 2) or different 

 But we also want "atomic" update of cm3 and its dependencies (m3core/libm3). 
 #1 only works if the old and new are compatible-enough.
 #1 is more automatic 

I think the answer is, if you are rebuilding m3core/libm3/cm3, and the old and new are not compatible-enough, you must set the output to be a new empty directory, and "run from" and "install to" must be different. If you are building almost anything else, you can safely update in place w/o breaking the compiler.

We might be able to achieve all of this by setting BUILD_DIR to an absolute path.
But I think I really want it driven by environment variables and not config files.

Before I do anything here, it looks like we could cleanup the m3overrides files in the current tree.
I did a lot of it already, years ago now, but some remains.
And then replace that system, is my proposal, with "override" not doing/meaning anything ever.

One more problem..do people here do, like "do-cm3-all build && sudo do-cm3-all ship".
Is it ok if this becomes "do-cm3-all build && sudo cp -r something1 something2"?
You know in GNU build system parlance, if build implies make install DESTDIR=writable-by-non-root.

 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: [M3devel] propose "output root" to replace shipping
Date: Tue, 24 Mar 2015 08:19:06 +0000

The point is, kind of to make it better, by making it worse.
Or worse by making it better.

In particular, "ship" would go away.
Files would always be overwritten right away.

If you don't want this -- and indeed sometimes you don't, you
would change the output root to a new empty/nonexistent directory,
possibly copying all of the previous into the new.

cm3 and its dependencies would require special attention.
In particular, for sharing violations on .dlls, we'd rename away and copy back.
Does that then work on all systems? It works on NT.
I'm betting most Posix systems "just work" and AIX requires either the same as NT
or can't be made to work so easily.

If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and
basically just them). For anything not used by cm3, just edit the source and recompile again.

You could view this as taking away an escape hatch.
But a simpler easier to understand escape hatch remains.

And the escape hatch is really only critical for cm3 and its dependents.
Or "build tools" maybe more generally, like klex, kyacc, m3bundle.

Is this still the mailing list to use, or something new with git?

Now -- another use of "ship" is "install" not on the same machine as compile.
For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived.

A background point here is -- have fewer different directory layouts.
Instead of ship rearranging stuff, just arrange stuff when you link in the first place.

This would also argue that the source tree and install tree should be more-similar/identical
except for the root, but I'm reluctant to rearrange the source..even to lift up the "src"
leaves by one, in order to keep the tree diffable against old trees.

 - Jay

> Date: Sun, 22 Mar 2015 11:57:35 -0500
> From: rodney_bates at lcwb.coop
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] propose "output root" to replace shipping
> 
> I haven't had time to digest your proposal yet, but I have often been uncomfortable
> about the current build system when building/bootstrapping itself.  It is probably
> just lack of understanding of the right procedure, but I usually cringe before attempting
> to bootstrap.  I have always been unclear about the sequence of when the newly built
> components overlay the preexisting ones, so I would welcome some attention to this.
> 
> I would like to put in a vote for a way to build everything without overlaying
> any existing components, so a bootstrapping operation can be easily restarted back
> where it started.  Right now, for the compiler itself, I am meticulously saving side
> copies of the compiler executables in /usr/local/cm3/bin every time.
> 
> On 03/20/2015 07:10 PM, Jay K wrote:
> >   Building the current system fails in caltech-parser\parserlib\parserlib\test.
> >
> >
> >
> >   It tries to run an shipped klex but only the non-shipped one exist -- depending on the state of things.
> >   But even if shipped exists, it likely shouldn't be using it.
> >
> >
> >
> >   This is a general problem we have been through somewhat before.
> >
> >
> >
> >   There are multiple parts to our current partial solution:
> >
> >
> >   They can be seen here:
> >    C:\dev2\cm3\m3-comm\netobj\src\netobj-ov.tmpl
> >    C:\dev2\cm3\m3-comm\sharedobj\src\sharedobj-ov.tmpl
> >    C:\dev2\cm3\m3-libs\libm3\src\bundleintf\bundle-ov.tmpl
> >    C:\dev2\cm3\m3-ui\zeus\src\m3zume-ov.tmpl
> >
> >
> >   And those various "build tools" use build_standalone()
> >
> >
> >   This could imho be abstracted.
> >
> >
> >   However, I propose where today we have:
> >      source_root/m3-foo/bar/target/abc.obj
> >      source_root/m3-foo/bar/target/bar.exe
> >      source_root/m3-foo/bar/src/abc.m3
> >
> >
> >    we instead have:
> >       output_root/obj/bar/target/abc.obj
> >       output_root/bin/target/bar.exe
> >       output_root/bin/bar.exe link or stub to target/abc.exe
> >       output_root/pkg/bar/...
> >
> >
> >   or:
> >       output_root_obj/bar/target/abc.obj
> >       output_root_install/bin/target/bar.exe
> >       output_root_install/bin/bar.exe link or stub to target/abc.exe
> >       output_root_install/pkg/bar/...
> >
> >
> >   and that "install" become just directory operations, like deleting old and move or copy output_root{_install} to it
> >   just "script", not anything in cm3,
> >
> >
> >   and then, on systems that support "origin" -- NT, Linux, Solaris, FreeBSD, NetBSD >= 5.0, we don't
> >   use standalone for this, and on systems that don't support "origin", still use standalone
> >
> >
> >   and then shipping goes away, as a package operation.
> >   you can instead ship an output root.
> >
> >
> >   and then this override stuff goes away also;
> >   You bootstrap by copying in a few working files. Like make-dist.py does.
> >   The system is reduced in size, as build_standalone falls away -- you can still use it,
> >   but it won't be used usually.
> >
> >
> >   The larger source tree would become readonly, as it should be.
> >
> >
> >   Thoughts?
> >
> >
> >   I believe Modula-3 was originally ahead of its time in having the readonly src directories,
> >   but I believe time as passed it by, and what one really wants is a larger readonly tree of src directories/packages -- which
> >   might as well not say "src", but preserve that instead of rearranging everything.
> >
> >
> >   I know, I've heard, having the separate buildlocal / buildglobal is useful.
> >   This doesn't elimine it that.
> >   It replaces it with "multiple buildglobal".
> >   Instead of having just the two options -- local and global -- you can create an arbitrary number of
> >   "global" scopes by creating a new output_root, copying it from a preexisting one.
> >
> >
> >   As well, it is obscure, but you can also have "multiple local" by changing BUILD_DIR.
> >   But I don't think anyone does that. I think everyone leaves BUILD_DIR == TARGET.
> >
> >
> >   This has several nice properties.
> >   readonly source tree
> >   less file copying possibly (depending on if final scripted ship is directory copy or move)
> >   cm3 can stop implementing ship
> >   no need to use build_standalone on most systems, including for cm3 itself (but it still works)
> >
> >
> >
> >   - Jay
> 
> -- 
> Rodney Bates
> rodney.m.bates at acm.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20150324/87c337ce/attachment-0002.html>