[M3devel] propose "output root" to replace shipping

[Rodney M. Bates]

This has suddenly reminded me of a problem and solution from the past.
I once did a lot of work in and on a package containing several source
code generators for components of compilers and such (Cocktail, if
anybody cares).  It was highly cyclic, several of the generators being
used by several of themselves to generate parts of their own code.

One day, after introducing a simple bug, I ended up with no working
executable needed to regenerate itself after I fixed the bug.  I don't
remember how I got out of this, but it took a lot of work and it was
tedious and very hacky.

Afterwards, I spent some time setting up a new build environment.
There could be any number of what I called "views" (named after a
different development system from mars), each a complete, more-or-less
copy of everything: hand edited source, generated source, compiled
objects, executables.

The build scripts used a symlink, found at a fixed place within a
view, to access executables to be used in building.  The view-cloning
script set up this link in the newly cloned view to point to the
executables in the old view.  So each view was built using unmodified
executables from the older view it was cloned from.

The use process was to always keep a view named "stable", well tested
to be able to rebuild itself.  I would clone a new view named
"working" and do development in that.  Periodically, I would clone
"phase1" from working, build in phase1 (which used executables in
working), clone "phase2" from phase1, and build in phase2.  If the
sources in working could regenerate and rebuild themselves twice in
this way, I considered it good evidence to make phase2 become the new
stable.

I don't remember the details, but I also had some scripted diffs
between equivalent generated source files in different views, to show
the regeneration process had converged to a fixed point.  As I recall,
there were often legitimate differences that had to be manually
reviewed, but that was just a good test of the most recent changes.

After that things worked very smoothly.  It did take up quite a bit of
disk space, with multiple copies of everything, but it was still
feasible, even in days when 6 GB was a big disk.

We don't have as much circularity in Modula-3, but there is some, and
even a tiny bit can cause a lot of grief.

So I suggest something like this.  Installing from a view to a
system-wide installation location would be a separate step.  It would
be greatly simplified if, as Jay suggested, the as-built and
as-installed subdirectory structures could be made the same.


On 03/24/2015 03:19 AM, Jay K wrote:
> The point is, kind of to make it better, by making it worse.
> Or worse by making it better.
>
>
> In particular, "ship" would go away.
> Files would /always be overwritten right away/.
>
>
> If you don't want this -- and indeed /sometimes /you don't, you
> would change the output root to a new empty/nonexistent directory,
> possibly copying all of the previous into the new.
>
>
>
> cm3 and its dependencies would require special attention.
> In particular, for sharing violations on .dlls, we'd rename away and copy back.
> Does that then work on all systems? It works on NT.
> I'm betting most Posix systems "just work" and AIX requires either the same as NT
> or can't be made to work so easily.
>
>
> If the outputs are bad, you would have to restore from backup, for things like cm3/m3core/libm3 (and
> basically just them). For anything not used by cm3, just edit the source and recompile again.
>
>
> You could view this as taking away an escape hatch.
> But a simpler easier to understand escape hatch remains.
>
>
> And the escape hatch is really only critical for cm3 and its dependents.
> Or "build tools" maybe more generally, like klex, kyacc, m3bundle.
>
>
> Is this still the mailing list to use, or something new with git?
>
>
> Now -- another use of "ship" is "install" not on the same machine as compile.
> For this, I propose, just un-tar-gz or unzip instead, or recursive copy if not archived.
>
>
> A background point here is -- have fewer different directory layouts.
> Instead of ship rearranging stuff, just arrange stuff when you link in the first place.
>
>
> This would also argue that the source tree and install tree should be more-similar/identical
> except for the root, but I'm reluctant to rearrange the source..even to lift up the "src"
> leaves by one, in order to keep the tree diffable against old trees.
>
>
>   - Jay
>-- 
Rodney Bates
rodney.m.bates at acm.org