[M3devel] replace build system??

Fri Jun 3 09:22:45 CEST 2016

I understand the C model very well, but I'm not certain of all
the ramifications for Modula-3.

I understand C++ has limited checking via name mangling.

And that other systems carry names/signatures forward to runtime
and check them there. Even dynamically linked C++. And Modula-3.

However function names aren't everything -- i.e. they don't include
structural hashes of types. It is name-based, not structure-based.

The most common mistake in the loose C systems is not doing things
wrong per se, but accidentally mixing old and new versions,
and the result is undefined/crash.

We have the same problem with our "extern" mechanism -- how many times
I debugged stat corrupting stack until I put in the extra layer.

C++ systems are getting better and check for "ODR violations" at link time.
But checking at link time kinda contradicts my question.
Goal being to fit into "generic" build systems but C++ only starts
winning when the linker knows about it -- when the system is C++ specific.
If C++ requires C++ linker/driver, we shouldn't feel too bad
requiring a Modula-3 linker/driver.

 > Typical front end compilation actual work is similar to 
 > or less than executable program startup/shutdown overhead. 

 Ideally this is not the case. Ideally process startup is just referencing
 pages already in file system cache. Demand paging, unified file system cache
 and memory manager and all that (all systems NT, Linux, etc. are the same
 these days). But I realize in reality there is alway some amount of other CPU
 to burn at each process start. The ideal is not quite reached.

 > m3cc is a separate executable..has to start up for every .[im]3 file. B

 I wonder if we can fix this btw.
 If we can start it just once per library.
 Process create really is fast on typical Posix systems, so people maybe
 don't notice.

 There are a few methods:
   - alway keep the m3cg ir files
   - if any out of date, read them all into one large IR and write out one large assembly file
   - I want do that anyway, to achieve a rich/poor man's LTCG/LTO.
     poor: No need to worry about linker/driver support. 
     rich: We can afford the memory these days. Though at some point it wouldn't scale. 
  - I want do similar for C backend, as it seems noticable either the compiler startup
    or the few #includes I sometimes use (though C++ modules promise some relief..)

  Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them 
  separately as usual, but within one m3cg invocation. 

 > Does/can it retain its incrementality?

 I think the best but not great we could do is make would decide pessmisically
 to run the compiler, but the compiler could figure out that the old/existing
 output is still correct. The world to decide up-to-date is significant but
 perhaps somewhat less than the work to just do all the work.

 >  see no benefit and almost certainly losses, in putting these in different executables.

  Less code to maintain. Easier to redistribute.
  I like the model of distribution of C source.
     wget source && tar xf && configure && make && make install

  I want something like that.
  I am uncertain of "redistribution" and "active development" must/should be different.
  i.e. We do have something like this, and the two ways of working cooexist.

 - Jay

----------------------------------------
> Date: Wed, 1 Jun 2016 17:08:34 -0500
> From: rodney_bates at lcwb.coop
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] replace build system??
>
>
>
> On 06/01/2016 04:30 AM, Jay K wrote:
>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure.
>
> It doesn't, not very well. A deep principle, if you can call it that, of C is that
> each compilation has no knowledge of any other compilation. Likewise, when linking
> them together, it's all just binary stuff, addresses, constants, etc. To get a
> semblance of type-safe compilation, all coders have to correctly follow some variant
> of a header file/#include idiom and create a matching makefile that correctly reflects
> the actual interdependencies. There is no checking that these are done correctly, and
> from direct experience, I can say they are routinely done wrong.
>
> All the modular languages have direct linguistic support for type-safe separate
> compilation. Implementing this correctly while using the C compilation model lies
> somewhere between very kludgy and impossible.
>
>>
>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file?
>
> It couldn't possibly. Typical front end compilation actual work is similar to
> or less than executable program startup/shutdown overhead. This effect
> might be partially lost in the fact that m3cc is a separate executable and it
> has to start up for every .[im]3 file. But it would still about double the
> number of program executions and startup overheads.
>
>> Does/can it retain its incrementality?
>
> I don't think this is possible all. No matter what is in the makefile, Make
> only understands dependencies among whole source files and must rebuild if
> a depended-on file has been touched, even no changes to content. Our current
> build system works on declaration granularity. (I presume this is what you
> mean by incrementality." I am not aware of any language-independent build
> infrastructure that keeps track of dependencies on other that source file
> granularity.
>
>>
>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level?
>>
>
> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler
> frontend, and language-specific link-time functions in one executable. I see no
> benefit and almost certainly losses, in putting these in different executables.
>
>> - Jay
>>
>>
>>
>>
>> ----------------------------------------
>>> From: jay.krell at cornell.edu
>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es
>>> Date: Tue, 31 May 2016 17:18:46 +0000
>>> Subject: Re: [M3devel] replace build system??
>>>
>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic.
>>> Posix should standardize more of this. :(
>>>
>>> - Jay
>>>
>>>
>>>
>>>
>>>
>>>
>>> ----------------------------------------
>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com
>>>> To: dabenavidesd at yahoo.es
>>>> Date: Tue, 31 May 2016 10:16:05 -0700
>>>> From: mika at async.caltech.edu
>>>> Subject: Re: [M3devel] replace build system??
>>>>
>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly
>>>> parallel if you turn on the right options. Every back-end invocation can be done
>>>> in parallel.
>>>>
>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go...
>>>>
>>>> "Daniel Alejandro Benavides D." writes:
>>>> ...
>>>>> A make-based build solution would have other advantages as well like=20
>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20
>>>>> four times faster.
>>>> ...
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>
>>> _______________________________________________
>>> M3devel mailing list
>>> M3devel at elegosoft.com
>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>
>> _______________________________________________
>> M3devel mailing list
>> M3devel at elegosoft.com
>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>
>
> --
> Rodney Bates
> rodney.m.bates at acm.org
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel