[M3devel] replace build system??

Fri Jun 3 16:53:16 CEST 2016

On 06/03/2016 02:22 AM, Jay K wrote:
> I understand the C model very well, but I'm not certain of all
> the ramifications for Modula-3.
>
>
> I understand C++ has limited checking via name mangling.
>
>
> And that other systems carry names/signatures forward to runtime
> and check them there. Even dynamically linked C++. And Modula-3.
>
>
> However function names aren't everything -- i.e. they don't include
> structural hashes of types. It is name-based, not structure-based.
>
>
> The most common mistake in the loose C systems is not doing things
> wrong per se, but accidentally mixing old and new versions,
> and the result is undefined/crash.
>
>
> We have the same problem with our "extern" mechanism -- how many times
> I debugged stat corrupting stack until I put in the extra layer.
>
>
> C++ systems are getting better and check for "ODR violations" at link time.
> But checking at link time kinda contradicts my question.
> Goal being to fit into "generic" build systems but C++ only starts
> winning when the linker knows about it -- when the system is C++ specific.
> If C++ requires C++ linker/driver, we shouldn't feel too bad
> requiring a Modula-3 linker/driver.
>
>
>   > Typical front end compilation actual work is similar to
>   > or less than executable program startup/shutdown overhead.
>
>
>   Ideally this is not the case. Ideally process startup is just referencing
>   pages already in file system cache. Demand paging, unified file system cache
>   and memory manager and all that (all systems NT, Linux, etc. are the same
>   these days). But I realize in reality there is alway some amount of other CPU
>   to burn at each process start. The ideal is not quite reached.
>
>   > m3cc is a separate executable..has to start up for every .[im]3 file. B
>
>   I wonder if we can fix this btw.
>   If we can start it just once per library.
>   Process create really is fast on typical Posix systems, so people maybe
>   don't notice.
>
>   There are a few methods:
>     - alway keep the m3cg ir files
>     - if any out of date, read them all into one large IR and write out one large assembly file
>     - I want do that anyway, to achieve a rich/poor man's LTCG/LTO.
>       poor: No need to worry about linker/driver support.
>       rich: We can afford the memory these days. Though at some point it wouldn't scale.
>    - I want do similar for C backend, as it seems noticable either the compiler startup
>      or the few #includes I sometimes use (though C++ modules promise some relief..)
>
>
>    Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them
>    separately as usual, but within one m3cg invocation.
>

I like this idea, but we don't want to force serialization where we now do parallel
runs of m3cg.

OTOH, it has puzzled me why, when compiling m3cc or m3gdb, the C compile commands
seem to scroll by slower than the unit compiles when running cm3, despite the fact
that the latter include a main program invocation of m3cc.  Surely it's not merely
smaller units?  And we are also running separately too.

>
>   > Does/can it retain its incrementality?
>
>   I think the best but not great we could do is make would decide pessmisically
>   to run the compiler, but the compiler could figure out that the old/existing
>   output is still correct. The world to decide up-to-date is significant but
>   perhaps somewhat less than the work to just do all the work.
>
>   >  see no benefit and almost certainly losses, in putting these in different executables.
>
>    Less code to maintain. Easier to redistribute.
>    I like the model of distribution of C source.
>       wget source && tar xf && configure && make && make install
>

We could just wrap what we have in makefile that, almost trivially invokes cm3 with
a few odd options.  Or is that cheating?

>    I want something like that.
>    I am uncertain of "redistribution" and "active development" must/should be different.
>    i.e. We do have something like this, and the two ways of working cooexist.
>
>
>   - Jay
>
>
>
>
>
> ----------------------------------------
>> Date: Wed, 1 Jun 2016 17:08:34 -0500
>> From: rodney_bates at lcwb.coop
>> To: m3devel at elegosoft.com
>> Subject: Re: [M3devel] replace build system??
>>
>>
>>
>> On 06/01/2016 04:30 AM, Jay K wrote:
>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure.
>>
>> It doesn't, not very well. A deep principle, if you can call it that, of C is that
>> each compilation has no knowledge of any other compilation. Likewise, when linking
>> them together, it's all just binary stuff, addresses, constants, etc. To get a
>> semblance of type-safe compilation, all coders have to correctly follow some variant
>> of a header file/#include idiom and create a matching makefile that correctly reflects
>> the actual interdependencies. There is no checking that these are done correctly, and
>> from direct experience, I can say they are routinely done wrong.
>>
>> All the modular languages have direct linguistic support for type-safe separate
>> compilation. Implementing this correctly while using the C compilation model lies
>> somewhere between very kludgy and impossible.
>>
>>>
>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file?
>>
>> It couldn't possibly. Typical front end compilation actual work is similar to
>> or less than executable program startup/shutdown overhead. This effect
>> might be partially lost in the fact that m3cc is a separate executable and it
>> has to start up for every .[im]3 file. But it would still about double the
>> number of program executions and startup overheads.
>>
>>> Does/can it retain its incrementality?
>>
>> I don't think this is possible all. No matter what is in the makefile, Make
>> only understands dependencies among whole source files and must rebuild if
>> a depended-on file has been touched, even no changes to content. Our current
>> build system works on declaration granularity. (I presume this is what you
>> mean by incrementality." I am not aware of any language-independent build
>> infrastructure that keeps track of dependencies on other that source file
>> granularity.
>>
>>>
>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level?
>>>
>>
>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler
>> frontend, and language-specific link-time functions in one executable. I see no
>> benefit and almost certainly losses, in putting these in different executables.
>>
>>> - Jay
>>>
>>>
>>>
>>>
>>> ----------------------------------------
>>>> From: jay.krell at cornell.edu
>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es
>>>> Date: Tue, 31 May 2016 17:18:46 +0000
>>>> Subject: Re: [M3devel] replace build system??
>>>>
>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic.
>>>> Posix should standardize more of this. :(
>>>>
>>>> - Jay
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ----------------------------------------
>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com
>>>>> To: dabenavidesd at yahoo.es
>>>>> Date: Tue, 31 May 2016 10:16:05 -0700
>>>>> From: mika at async.caltech.edu
>>>>> Subject: Re: [M3devel] replace build system??
>>>>>
>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly
>>>>> parallel if you turn on the right options. Every back-end invocation can be done
>>>>> in parallel.
>>>>>
>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go...
>>>>>
>>>>> "Daniel Alejandro Benavides D." writes:
>>>>> ...
>>>>>> A make-based build solution would have other advantages as well like=20
>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20
>>>>>> four times faster.
>>>>> ...
>>>>> _______________________________________________
>>>>> M3devel mailing list
>>>>> M3devel at elegosoft.com
>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>
>>> _______________________________________________
>>> M3devel mailing list
>>> M3devel at elegosoft.com
>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>
>>
>> --
>> Rodney Bates
>> rodney.m.bates at acm.org
>> _______________________________________________
>> M3devel mailing list
>> M3devel at elegosoft.com
>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>   		 	   		
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>

-- 
Rodney Bates
rodney.m.bates at acm.org