[M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110

[Olaf Wagner]

On Tue, 02 Jun 2015 11:18:03 -0500
"Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:

> 
> 
> On 06/02/2015 04:36 AM, Olaf Wagner wrote:
> > On Mon, 01 Jun 2015 19:54:45 -0500
> > "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:
> >
> >> Here is a short-term proposal (i.e., without major reorganization)
> >> for the do-cm3*.sh scripts:
> >>
> >> 1) 'build' only builds, as we seem to agree it should.
> >> 2) a new option 'override' (and only 'override') causes an override build
> >> 3) a new option 'partialship' ships, as each package is done, things that
> >>      will be needed to compile another package that does a quake import on the
> >>      just-built package (I think this means static library, if any, and .M3WEB),
> >>      but does not ship things that will be used to execute the just-built package
> >>      (I think this means executable or dynamic library).  I'm not sure right off
> >>      hand which ship group things like interface source files, etc. belong in.
> >
> > I don't know how you will implement this and how it should fit into the
> > M3 package concept, unless you ship to a completely different location,
> > i.e. another package pool (but cm3 currently just supports one).
> >
> > In my opinion the package system relies on the invariant that a shipped
> > (installed) package is completely consistent, i.e. the source code
> > interfaces correspond to the intermediate code interfaces and to the
> > binary equivalent of them (static and dynamic libraries). If I understand
> > you correctly, you want to give up this consistency in favour of being
> > able to do an easier bootstrap of the CM3 system.
> 
> Yes, but we already must have a temporarily inconsistent state of installation
> today.  This is a somewhat different kind of inconsistency, but inconsistent,
> nonetheless.
> 
> I guess -partialship could do an additional final pass over the package list,
> doing a full ship.  The inconsistent state would only last during a single
> user-requested step, which is about as short as it can get.  My intent was
> that one would immediately do a full ship after a partialship, although as
> a separate command, thus restoring consistency.

OK, I think I have a better idea of what you want to do now.

> > But bootstrapping a complex system is never easy and usually requires some
> > special or tricky steps. Ordinary use of the system, i.e. all other applications,
> > that just build on the standard tools, don't need this kind of steps.
> >
> 
> And my proposal is a simple, short-term way to provide somewhat more flexible
> options for just such special or tricky steps.  It will work the same way as now,
> if you don't use the new options, and would allow Jay to get rid of the hated
> INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps.
> 
> > If I could design (and implement) a better cm3 builder, I'd have one with
> > multiple package pools (shipping destinations, locations of installed
> > packages) and an integrated builder that knows all about the package
> > dependencies (so that we haven't to do that in scripts). The two points
> > are the only shortcomings in the cm3 build system in my opinion.
> > The prjm tools of the ComPact sources was a step in this direction:
> > it can handle multiple pools of packages and their dependencies, but
> > is language independent, therefore too complex, and never got integrated
> > into the compiler builder.
> >
> 
> Yes, absolutely.  We need a greatly reworked build system for multiple packages.
> For building within a package, cm3's existing build is really quite sophisticated.
> I don't know of any other that detects the need for recompilation
> on declaration-granularity.  But it does very little with inter-package
> problems -- just detection of link-time inconsistencies, with hopelessly
> uninformative error messages.  (I once set out to improve them, but it
> required more info in the .mx and .m3x files, with more incompatible
> changes, and I only got one message improved a little bit, before I gave
> up for the time.)
> 
> I agree, reworking it all is very desirable.  It will take a lot of rework.
> I think we all agree that some kind of multi-layer, multi-destination
> scheme is needed.  This will probably entail disentangling the source
> and build directories from being interspersed in each package, or something
> equally perturbing to the existing system.  It isn't going to be a little
> patch.  I think my proposal will be small to implement and have no impact
> on anything existing, if you don't use the new options.

I'm not against your proposal as a first or intermediate step. We have
to remain pragmatic and improve things incrementally. If people really
use the old shell scripts, I'm not against adapting them.  Jay may
do all the same things in Python even faster, but I always found that
was an additional unnecessary dependency. On the other hand, Python
can probably be installed in a couple of minutes on any system these
days.

I also agree that we would all like a multi-level fully integrated
build system for sets of M3 packages, but that will be a lot of work
and I won't be able to contribute much to it. Those who do the
work will decide about the design and its capabilities.

> > As for the shell scripts, I'm surprised that they have survived so long;
> > they just got created on the fly to support the building and publication
> > of the CM3 system as open source when we got the sources from Critical
> > Mass. They were never intended as a general user interface, but only
> > as tools for the CM3 maintainers, and later got used in the Hudson CI.
> >
> > BTW, if I find some time and access to suitable machines again, I'd
> > really like to set up a new CI system on Jenkins interfacing with the
> > Github infrastruture. But I won't make any promises here.
> 
> Yes, this would be very good.  Priorities, priorities! :-(.

It's not as improbable as me writing a new integrated build system
though, as I've set up several complex CI systems for other projects.
We'll se if I find the fime...

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com 
               Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
Geschäftsführer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194