[M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110

[Rodney M. Bates]

On 06/02/2015 04:36 AM, Olaf Wagner wrote:
> On Mon, 01 Jun 2015 19:54:45 -0500
> "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:
>
>> Here is a short-term proposal (i.e., without major reorganization)
>> for the do-cm3*.sh scripts:
>>
>> 1) 'build' only builds, as we seem to agree it should.
>> 2) a new option 'override' (and only 'override') causes an override build
>> 3) a new option 'partialship' ships, as each package is done, things that
>>      will be needed to compile another package that does a quake import on the
>>      just-built package (I think this means static library, if any, and .M3WEB),
>>      but does not ship things that will be used to execute the just-built package
>>      (I think this means executable or dynamic library).  I'm not sure right off
>>      hand which ship group things like interface source files, etc. belong in.
>
> I don't know how you will implement this and how it should fit into the
> M3 package concept, unless you ship to a completely different location,
> i.e. another package pool (but cm3 currently just supports one).
>
> In my opinion the package system relies on the invariant that a shipped
> (installed) package is completely consistent, i.e. the source code
> interfaces correspond to the intermediate code interfaces and to the
> binary equivalent of them (static and dynamic libraries). If I understand
> you correctly, you want to give up this consistency in favour of being
> able to do an easier bootstrap of the CM3 system.

Yes, but we already must have a temporarily inconsistent state of installation
today.  This is a somewhat different kind of inconsistency, but inconsistent,
nonetheless.

I guess -partialship could do an additional final pass over the package list,
doing a full ship.  The inconsistent state would only last during a single
user-requested step, which is about as short as it can get.  My intent was
that one would immediately do a full ship after a partialship, although as
a separate command, thus restoring consistency.


>
> But bootstrapping a complex system is never easy and usually requires some
> special or tricky steps. Ordinary use of the system, i.e. all other applications,
> that just build on the standard tools, don't need this kind of steps.
>

And my proposal is a simple, short-term way to provide somewhat more flexible
options for just such special or tricky steps.  It will work the same way as now,
if you don't use the new options, and would allow Jay to get rid of the hated
INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special steps.

> If I could design (and implement) a better cm3 builder, I'd have one with
> multiple package pools (shipping destinations, locations of installed
> packages) and an integrated builder that knows all about the package
> dependencies (so that we haven't to do that in scripts). The two points
> are the only shortcomings in the cm3 build system in my opinion.
> The prjm tools of the ComPact sources was a step in this direction:
> it can handle multiple pools of packages and their dependencies, but
> is language independent, therefore too complex, and never got integrated
> into the compiler builder.
>

Yes, absolutely.  We need a greatly reworked build system for multiple packages.
For building within a package, cm3's existing build is really quite sophisticated.
I don't know of any other that detects the need for recompilation
on declaration-granularity.  But it does very little with inter-package
problems -- just detection of link-time inconsistencies, with hopelessly
uninformative error messages.  (I once set out to improve them, but it
required more info in the .mx and .m3x files, with more incompatible
changes, and I only got one message improved a little bit, before I gave
up for the time.)

I agree, reworking it all is very desirable.  It will take a lot of rework.
I think we all agree that some kind of multi-layer, multi-destination
scheme is needed.  This will probably entail disentangling the source
and build directories from being interspersed in each package, or something
equally perturbing to the existing system.  It isn't going to be a little
patch.  I think my proposal will be small to implement and have no impact
on anything existing, if you don't use the new options.


> As for the shell scripts, I'm surprised that they have survived so long;
> they just got created on the fly to support the building and publication
> of the CM3 system as open source when we got the sources from Critical
> Mass. They were never intended as a general user interface, but only
> as tools for the CM3 maintainers, and later got used in the Hudson CI.
>
> BTW, if I find some time and access to suitable machines again, I'd
> really like to set up a new CI system on Jenkins interfacing with the
> Github infrastruture. But I won't make any promises here.
>

Yes, this would be very good.  Priorities, priorities! :-(.

> Olaf
>

-- 
Rodney Bates
rodney.m.bates at acm.org