[M3devel] replace build system??

Fri Jun 3 21:51:07 CEST 2016

Moving a module from one package to another is a problem I have hit.
I moved some path manipulation module a few years ago and incremental builds failed.

Not tracking optimization can be seen as a feature, or a bug.
  I might just want to not optimize a few files that I'm debugging.

WIDECHAR certainly.

Choice of C backend is another. :)
Nested functions have a different interface.
But appending "c" to BUILD_DIR is maybe a good workaround. Maybe.
I wish I could be link compatible but I don't see how.
gcc puts the static link often in a special register dedicated to the purpose -- ABIs actually designed and optimized with this in mind!
I put it as the last parameter.

 - Jay

----------------------------------------
> Date: Fri, 3 Jun 2016 09:44:12 -0500
> From: rodney_bates at lcwb.coop
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] replace build system??
>
> BTW, 'though the current build system is considerably sophisticated,
> it does have some bugs, which we should fix. I have the exact setup &
> symptom for one of them written down somewhere. I think it's when adding
> a procedure signature to an interface when a procedure body for it was
> already present in a compiled exporting module, things don't get recompiled
> right.
>
> Also, I occasionally get recompile failures that I don't think are valid,
> and when I clean the package and recompile that, all is well. I haven't
> ever tracked down how to reproduce this or noted its exact symptom.
>
> We also don't keep track of compile options that affect generated code,
> so as to recompile when only the options change. E.g., optimization level,
> debug info presence and format. I did do this for the size of WIDECHAR,
> since mismatches here would be disastrous.
>
> On 06/03/2016 02:22 AM, Jay K wrote:
>> I understand the C model very well, but I'm not certain of all
>> the ramifications for Modula-3.
>>
>>
>> I understand C++ has limited checking via name mangling.
>>
>>
>> And that other systems carry names/signatures forward to runtime
>> and check them there. Even dynamically linked C++. And Modula-3.
>>
>>
>> However function names aren't everything -- i.e. they don't include
>> structural hashes of types. It is name-based, not structure-based.
>>
>>
>> The most common mistake in the loose C systems is not doing things
>> wrong per se, but accidentally mixing old and new versions,
>> and the result is undefined/crash.
>>
>>
>> We have the same problem with our "extern" mechanism -- how many times
>> I debugged stat corrupting stack until I put in the extra layer.
>>
>>
>> C++ systems are getting better and check for "ODR violations" at link time.
>> But checking at link time kinda contradicts my question.
>> Goal being to fit into "generic" build systems but C++ only starts
>> winning when the linker knows about it -- when the system is C++ specific.
>> If C++ requires C++ linker/driver, we shouldn't feel too bad
>> requiring a Modula-3 linker/driver.
>>
>>
>>> Typical front end compilation actual work is similar to
>>> or less than executable program startup/shutdown overhead.
>>
>>
>> Ideally this is not the case. Ideally process startup is just referencing
>> pages already in file system cache. Demand paging, unified file system cache
>> and memory manager and all that (all systems NT, Linux, etc. are the same
>> these days). But I realize in reality there is alway some amount of other CPU
>> to burn at each process start. The ideal is not quite reached.
>>
>>> m3cc is a separate executable..has to start up for every .[im]3 file. B
>>
>> I wonder if we can fix this btw.
>> If we can start it just once per library.
>> Process create really is fast on typical Posix systems, so people maybe
>> don't notice.
>>
>> There are a few methods:
>> - alway keep the m3cg ir files
>> - if any out of date, read them all into one large IR and write out one large assembly file
>> - I want do that anyway, to achieve a rich/poor man's LTCG/LTO.
>> poor: No need to worry about linker/driver support.
>> rich: We can afford the memory these days. Though at some point it wouldn't scale.
>> - I want do similar for C backend, as it seems noticable either the compiler startup
>> or the few #includes I sometimes use (though C++ modules promise some relief..)
>>
>>
>> Or, really, more simply, just accept a bunch of .mc/ms file pairs and process them
>> separately as usual, but within one m3cg invocation.
>>
>>
>>> Does/can it retain its incrementality?
>>
>> I think the best but not great we could do is make would decide pessmisically
>> to run the compiler, but the compiler could figure out that the old/existing
>> output is still correct. The world to decide up-to-date is significant but
>> perhaps somewhat less than the work to just do all the work.
>>
>>> see no benefit and almost certainly losses, in putting these in different executables.
>>
>> Less code to maintain. Easier to redistribute.
>> I like the model of distribution of C source.
>> wget source && tar xf && configure && make && make install
>>
>> I want something like that.
>> I am uncertain of "redistribution" and "active development" must/should be different.
>> i.e. We do have something like this, and the two ways of working cooexist.
>>
>>
>> - Jay
>>
>>
>>
>>
>>
>> ----------------------------------------
>>> Date: Wed, 1 Jun 2016 17:08:34 -0500
>>> From: rodney_bates at lcwb.coop
>>> To: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] replace build system??
>>>
>>>
>>>
>>> On 06/01/2016 04:30 AM, Jay K wrote:
>>>> We need to understand if and how well Modula-3 fits in the traditional and widespread C build infrastructure.
>>>
>>> It doesn't, not very well. A deep principle, if you can call it that, of C is that
>>> each compilation has no knowledge of any other compilation. Likewise, when linking
>>> them together, it's all just binary stuff, addresses, constants, etc. To get a
>>> semblance of type-safe compilation, all coders have to correctly follow some variant
>>> of a header file/#include idiom and create a matching makefile that correctly reflects
>>> the actual interdependencies. There is no checking that these are done correctly, and
>>> from direct experience, I can say they are routinely done wrong.
>>>
>>> All the modular languages have direct linguistic support for type-safe separate
>>> compilation. Implementing this correctly while using the C compilation model lies
>>> somewhere between very kludgy and impossible.
>>>
>>>>
>>>> Does/can it retain its build speed if you invoke cm3 per .i3 and per .m3 file?
>>>
>>> It couldn't possibly. Typical front end compilation actual work is similar to
>>> or less than executable program startup/shutdown overhead. This effect
>>> might be partially lost in the fact that m3cc is a separate executable and it
>>> has to start up for every .[im]3 file. But it would still about double the
>>> number of program executions and startup overheads.
>>>
>>>> Does/can it retain its incrementality?
>>>
>>> I don't think this is possible all. No matter what is in the makefile, Make
>>> only understands dependencies among whole source files and must rebuild if
>>> a depended-on file has been touched, even no changes to content. Our current
>>> build system works on declaration granularity. (I presume this is what you
>>> mean by incrementality." I am not aware of any language-independent build
>>> infrastructure that keeps track of dependencies on other that source file
>>> granularity.
>>>
>>>>
>>>> Or do we really need to be more of the "driver" and do a lot of stuff at the lib/link level?
>>>>
>>>
>>> In a sense, we already have this. Cm3 contains a sophisticated driver, a compiler
>>> frontend, and language-specific link-time functions in one executable. I see no
>>> benefit and almost certainly losses, in putting these in different executables.
>>>
>>>> - Jay
>>>>
>>>>
>>>>
>>>>
>>>> ----------------------------------------
>>>>> From: jay.krell at cornell.edu
>>>>> To: mika at async.caltech.edu; estellnb at elstel.org; m3devel at elegosoft.com; dabenavidesd at yahoo.es
>>>>> Date: Tue, 31 May 2016 17:18:46 +0000
>>>>> Subject: Re: [M3devel] replace build system??
>>>>>
>>>>> We should probably learn how to get the number of processors and optionally system load and make this more automatic.
>>>>> Posix should standardize more of this. :(
>>>>>
>>>>> - Jay
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ----------------------------------------
>>>>>> To: estellnb at elstel.org; jay.krell at cornell.edu; m3devel at elegosoft.com
>>>>>> To: dabenavidesd at yahoo.es
>>>>>> Date: Tue, 31 May 2016 10:16:05 -0700
>>>>>> From: mika at async.caltech.edu
>>>>>> Subject: Re: [M3devel] replace build system??
>>>>>>
>>>>>> I mentioned this before a few times on this mailing list.... CM3 is already fairly
>>>>>> parallel if you turn on the right options. Every back-end invocation can be done
>>>>>> in parallel.
>>>>>>
>>>>>> Set M3_PARALLEL_BACK in the config to 10 or 20 and watch it go...
>>>>>>
>>>>>> "Daniel Alejandro Benavides D." writes:
>>>>>> ...
>>>>>>> A make-based build solution would have other advantages as well like=20
>>>>>>> f.i. parallel build by make --jobs=3D4. That way CM3 could build up to=20
>>>>>>> four times faster.
>>>>>> ...
>>>>>> _______________________________________________
>>>>>> M3devel mailing list
>>>>>> M3devel at elegosoft.com
>>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>>
>>>>> _______________________________________________
>>>>> M3devel mailing list
>>>>> M3devel at elegosoft.com
>>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>> _______________________________________________
>>>> M3devel mailing list
>>>> M3devel at elegosoft.com
>>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>>>
>>>
>>> --
>>> Rodney Bates
>>> rodney.m.bates at acm.org
>>> _______________________________________________
>>> M3devel mailing list
>>> M3devel at elegosoft.com
>>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>
>> _______________________________________________
>> M3devel mailing list
>> M3devel at elegosoft.com
>> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>>
>
> --
> Rodney Bates
> rodney.m.bates at acm.org
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel