[M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110

[Hendrik Boom]

On Tue, Jun 02, 2015 at 05:48:13PM +0000, Jay K wrote:
>  > Yes, absolutely. We need a greatly reworked build system for multiple packages.
>  > For building within a package, cm3's existing build is really quite sophisticated.
> 
> Exactly my thoughts for years. quake is nice and declarative for each package, and currently
> does nothing for multiple packages. 
> 
> 
> How far would we get with the following:
> 
> 
> include_dir to stitch together packages, the same as it stitches
> directories within packages.
> 
> "Flush" the state at each Library() or Program() invocation?
> 
> Library and Program have to be last currently? So that would work?
> Or they can be anywhere?
> 
> 
> Furthermore, target directory either be:
>   1) ./target -- might be what it'd do now, but is wrong choice 

Probably the wrong place.  ./cm3target might be slightly better.
Or make it not hidden.
Or have a quake conmend to set it.

>   2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. 
>      cd anywhere, that is still it
>   3) under some new new root, like go up from cwd until you find 
>      special marker file; this would be building outside source tree entirely 

Outside the source tree entirely would be good.  I'd really like to be 
able to compile a source tree that's on a read-only file system.  Or on 
a shadow file system.

>   4) same as currently, wherever is Library()/Program(), go ../<target>
> 
> If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"?
> 
> Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir.
> 
> it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib.
> 
> It looks like:dirs file: DIRS=a \  b \ c
> sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake
> I use this system a lot and quite like it.Though it is obscure.
> The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad.
> 
> I think scons works this way too, the language being Python.

Writing the cm3 stanzas to tell scons how to use cm3 sould be useful, 
too.

> 
> Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program.
> 
> Make sense?
> Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.)
> 
>  - Jay
> 
> 
> 
> 
> From: dmuysers at hotmail.com
> To: rodney.m.bates at acm.org; wagner at elegosoft.com
> Date: Tue, 2 Jun 2015 18:39:35 +0200
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] A proposal,	was m3cgc1: fatal error: *** bad M3CG version stamp (0x100),	expected 0x110
> 
> 
> 
> 
> 
> golang is an example of good 
> package management.
> It`s interaction of the builder with github repositories is an interesting 
> idea. 
>  
> -----Original Message----- 
> From: Rodney M. Bates 
> Sent: Tuesday, June 02, 2015 6:18 PM 
> To: Olaf Wagner 
> Cc: m3devel 
> Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG 
> version stamp (0x100), expected 0x110 
>  
>  
>  
> On 06/02/2015 04:36 AM, Olaf Wagner wrote:
> > On Mon, 01 Jun 2015 19:54:45 -0500
> > "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:
> >
> >> Here is a short-term proposal (i.e., without major 
> reorganization)
> >> for the do-cm3*.sh scripts:
> >>
> >> 1) 'build' only builds, as we seem to agree it should.
> >> 2) a new option 'override' (and only 'override') causes an 
> override build
> >> 3) a new option 'partialship' ships, as each package is done, 
> things that
> >>      will be needed to compile another 
> package that does a quake import on the
> >>      just-built package (I think this 
> means static library, if any, and .M3WEB),
> >>      but does not ship things that will 
> be used to execute the just-built package
> >>      (I think this means executable or 
> dynamic library).  I'm not sure right off
> >>      hand which ship group things like 
> interface source files, etc. belong in.
> >
> > I don't know how you will implement this and how it should fit into 
> the
> > M3 package concept, unless you ship to a completely different 
> location,
> > i.e. another package pool (but cm3 currently just supports one).
> >
> > In my opinion the package system relies on the invariant that a 
> shipped
> > (installed) package is completely consistent, i.e. the source 
> code
> > interfaces correspond to the intermediate code interfaces and to 
> the
> > binary equivalent of them (static and dynamic libraries). If I 
> understand
> > you correctly, you want to give up this consistency in favour of 
> being
> > able to do an easier bootstrap of the CM3 system.
>  
> Yes, but we already must have a temporarily inconsistent state of 
> installation
> today.  This is a somewhat different kind of inconsistency, but 
> inconsistent,
> nonetheless.
>  
> I guess -partialship could do an additional final pass over the package 
> list,
> doing a full ship.  The inconsistent state would only last during a 
> single
> user-requested step, which is about as short as it can get.  My intent 
> was
> that one would immediately do a full ship after a partialship, although 
> as
> a separate command, thus restoring consistency.
>  
>  
> >
> > But bootstrapping a complex system is never easy and usually requires 
> some
> > special or tricky steps. Ordinary use of the system, i.e. all other 
> applications,
> > that just build on the standard tools, don't need this kind of 
> steps.
> >
>  
> And my proposal is a simple, short-term way to provide somewhat more 
> flexible
> options for just such special or tricky steps.  It will work the same 
> way as now,
> if you don't use the new options, and would allow Jay to get rid of the 
> hated
> INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special 
> steps.
>  
> > If I could design (and implement) a better cm3 builder, I'd have one 
> with
> > multiple package pools (shipping destinations, locations of 
> installed
> > packages) and an integrated builder that knows all about the 
> package
> > dependencies (so that we haven't to do that in scripts). The two 
> points
> > are the only shortcomings in the cm3 build system in my opinion.
> > The prjm tools of the ComPact sources was a step in this 
> direction:
> > it can handle multiple pools of packages and their dependencies, 
> but
> > is language independent, therefore too complex, and never got 
> integrated
> > into the compiler builder.
> >
>  
> Yes, absolutely.  We need a greatly reworked build system for multiple 
> packages.
> For building within a package, cm3's existing build is really quite 
> sophisticated.
> I don't know of any other that detects the need for recompilation
> on declaration-granularity.  But it does very little with 
> inter-package
> problems -- just detection of link-time inconsistencies, with 
> hopelessly
> uninformative error messages.  (I once set out to improve them, but 
> it
> required more info in the .mx and .m3x files, with more incompatible
> changes, and I only got one message improved a little bit, before I 
> gave
> up for the time.)
>  
> I agree, reworking it all is very desirable.  It will take a lot of 
> rework.
> I think we all agree that some kind of multi-layer, multi-destination
> scheme is needed.  This will probably entail disentangling the 
> source
> and build directories from being interspersed in each package, or 
> something
> equally perturbing to the existing system.  It isn't going to be a 
> little
> patch.  I think my proposal will be small to implement and have no 
> impact
> on anything existing, if you don't use the new options.
>  
>  
> > As for the shell scripts, I'm surprised that they have survived so 
> long;
> > they just got created on the fly to support the building and 
> publication
> > of the CM3 system as open source when we got the sources from 
> Critical
> > Mass. They were never intended as a general user interface, but 
> only
> > as tools for the CM3 maintainers, and later got used in the Hudson 
> CI.
> >
> > BTW, if I find some time and access to suitable machines again, 
> I'd
> > really like to set up a new CI system on Jenkins interfacing with 
> the
> > Github infrastruture. But I won't make any promises here.
> >
>  
> Yes, this would be very good.  Priorities, priorities! :-(.
>  
> > Olaf
> >
>  
> -- 
> Rodney Bates
> rodney.m.bates at acm.org