[M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG version stamp (0x100), expected 0x110

[Jay K]

 > Yes, absolutely. We need a greatly reworked build system for multiple packages.
 > For building within a package, cm3's existing build is really quite sophisticated.

Exactly my thoughts for years. quake is nice and declarative for each package, and currently
does nothing for multiple packages. 


How far would we get with the following:


include_dir to stitch together packages, the same as it stitches
directories within packages.

"Flush" the state at each Library() or Program() invocation?

Library and Program have to be last currently? So that would work?
Or they can be anywhere?


Furthermore, target directory either be:
  1) ./target -- might be what it'd do now, but is wrong choice 
  2) under some new root, I'd suggest e.g. $CM3_OBJECT_ROOT. 
     cd anywhere, that is still it
  3) under some new new root, like go up from cwd until you find 
     special marker file; this would be building outside source tree entirely 
  4) same as currently, wherever is Library()/Program(), go ../<target>

If I'm wrong and library/program aren't last, well, then, just use a new name for include_dir?That implicity "flushes"?

Anyone out there familiar with NT build.exe?It very much resembles quake, but it does solve this last problem.But it doesn't have intra-package include_dir.

it has dirs files for package stitching and sources files at the leaves, and only the leaves.Something like m3core that has multiple directories would actually either be one flat directory,or be composed of multiple static libraries, or more efficiently but esoteric, composedof multiple TARGETTYPE=NOTARGET, which is like a library but leaves just lose object filesand doesn't make the .lib.

It looks like:dirs file: DIRS=a \  b \ c
sources file:declarative like quake, but actually an nmake snippet instead of a quake snippetescape hatch is more nmake, rather than more general purpose quake
I use this system a lot and quite like it.Though it is obscure.
The point is -- declarative system with escape hatches (quake and modula-3 are the escape hatches) are a good design.There are similar systems out there.Ours isn't bad.

I think scons works this way too, the language being Python.

Now, I misstated something. Other than just walking dirs and flushing, except for m3cc, we can walk in the correct orderbased on imports.m3cc we can probably fix the order by importing from cm3, if we allow importing from a program.

Make sense?
Or replace everything with cmake or scons or even automake or such?(Modula-3 was ahead of its time 20 years ago, but I think isn't any longer. It isn't really behind, but has near peers.)

 - Jay




From: dmuysers at hotmail.com
To: rodney.m.bates at acm.org; wagner at elegosoft.com
Date: Tue, 2 Jun 2015 18:39:35 +0200
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] A proposal,	was m3cgc1: fatal error: *** bad M3CG version stamp (0x100),	expected 0x110





golang is an example of good 
package management.
It`s interaction of the builder with github repositories is an interesting 
idea. 
 
-----Original Message----- 
From: Rodney M. Bates 
Sent: Tuesday, June 02, 2015 6:18 PM 
To: Olaf Wagner 
Cc: m3devel 
Subject: Re: [M3devel] A proposal, was m3cgc1: fatal error: *** bad M3CG 
version stamp (0x100), expected 0x110 
 
 
 
On 06/02/2015 04:36 AM, Olaf Wagner wrote:
> On Mon, 01 Jun 2015 19:54:45 -0500
> "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:
>
>> Here is a short-term proposal (i.e., without major 
reorganization)
>> for the do-cm3*.sh scripts:
>>
>> 1) 'build' only builds, as we seem to agree it should.
>> 2) a new option 'override' (and only 'override') causes an 
override build
>> 3) a new option 'partialship' ships, as each package is done, 
things that
>>      will be needed to compile another 
package that does a quake import on the
>>      just-built package (I think this 
means static library, if any, and .M3WEB),
>>      but does not ship things that will 
be used to execute the just-built package
>>      (I think this means executable or 
dynamic library).  I'm not sure right off
>>      hand which ship group things like 
interface source files, etc. belong in.
>
> I don't know how you will implement this and how it should fit into 
the
> M3 package concept, unless you ship to a completely different 
location,
> i.e. another package pool (but cm3 currently just supports one).
>
> In my opinion the package system relies on the invariant that a 
shipped
> (installed) package is completely consistent, i.e. the source 
code
> interfaces correspond to the intermediate code interfaces and to 
the
> binary equivalent of them (static and dynamic libraries). If I 
understand
> you correctly, you want to give up this consistency in favour of 
being
> able to do an easier bootstrap of the CM3 system.
 
Yes, but we already must have a temporarily inconsistent state of 
installation
today.  This is a somewhat different kind of inconsistency, but 
inconsistent,
nonetheless.
 
I guess -partialship could do an additional final pass over the package 
list,
doing a full ship.  The inconsistent state would only last during a 
single
user-requested step, which is about as short as it can get.  My intent 
was
that one would immediately do a full ship after a partialship, although 
as
a separate command, thus restoring consistency.
 
 
>
> But bootstrapping a complex system is never easy and usually requires 
some
> special or tricky steps. Ordinary use of the system, i.e. all other 
applications,
> that just build on the standard tools, don't need this kind of 
steps.
>
 
And my proposal is a simple, short-term way to provide somewhat more 
flexible
options for just such special or tricky steps.  It will work the same 
way as now,
if you don't use the new options, and would allow Jay to get rid of the 
hated
INSTALL_CM3_IN_BIN variable, with no loss of flexibility for special 
steps.
 
> If I could design (and implement) a better cm3 builder, I'd have one 
with
> multiple package pools (shipping destinations, locations of 
installed
> packages) and an integrated builder that knows all about the 
package
> dependencies (so that we haven't to do that in scripts). The two 
points
> are the only shortcomings in the cm3 build system in my opinion.
> The prjm tools of the ComPact sources was a step in this 
direction:
> it can handle multiple pools of packages and their dependencies, 
but
> is language independent, therefore too complex, and never got 
integrated
> into the compiler builder.
>
 
Yes, absolutely.  We need a greatly reworked build system for multiple 
packages.
For building within a package, cm3's existing build is really quite 
sophisticated.
I don't know of any other that detects the need for recompilation
on declaration-granularity.  But it does very little with 
inter-package
problems -- just detection of link-time inconsistencies, with 
hopelessly
uninformative error messages.  (I once set out to improve them, but 
it
required more info in the .mx and .m3x files, with more incompatible
changes, and I only got one message improved a little bit, before I 
gave
up for the time.)
 
I agree, reworking it all is very desirable.  It will take a lot of 
rework.
I think we all agree that some kind of multi-layer, multi-destination
scheme is needed.  This will probably entail disentangling the 
source
and build directories from being interspersed in each package, or 
something
equally perturbing to the existing system.  It isn't going to be a 
little
patch.  I think my proposal will be small to implement and have no 
impact
on anything existing, if you don't use the new options.
 
 
> As for the shell scripts, I'm surprised that they have survived so 
long;
> they just got created on the fly to support the building and 
publication
> of the CM3 system as open source when we got the sources from 
Critical
> Mass. They were never intended as a general user interface, but 
only
> as tools for the CM3 maintainers, and later got used in the Hudson 
CI.
>
> BTW, if I find some time and access to suitable machines again, 
I'd
> really like to set up a new CI system on Jenkins interfacing with 
the
> Github infrastruture. But I won't make any promises here.
>
 
Yes, this would be very good.  Priorities, priorities! :-(.
 
> Olaf
>
 
-- 
Rodney Bates
rodney.m.bates at acm.org 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20150602/ddb3b8cb/attachment-0002.html>