[M3devel] multi-threaded m3front?

[Darko Volaric]

The most speedup you'll get is where there's least dependencies (imports or
exports), with zero being the optimal number (a "root" file). If there are
more than zero dependencies for a given file then the total time it will
take to finish the file will be its own compile time plus the total time of
its slowest dependency, that is, the sum of the compile times of the
slowest compiling files between it and some root file. The slowest of those
is the "critical path."

Of course that assumes that you have an infinite number of processors.
Because you don't you then have to ensure that the critical path gets
priority in processing, since delaying it for any other work would be
counterproductive. Realistically the critical path and that scheduling can
only be guessed before the first compile, unless file size is an accurate
predictor of compile time, or some sort of prior analysis is done.

The critical path represents the serial portion of the compilation so if
you figure out that as the percentage of the total compilation you can get
an idea of the maximum possible speedup using Amdahl's law. Here is a
rather depressing graph that illustrates it:

https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/AmdahlsLaw.svg/648px-AmdahlsLaw.svg.png

Also while compiling the entire repository in parallel may yield a
significant speedup, most people compile incrementally, involving
relatively few files.




On Sat, Aug 8, 2015 at 8:05 PM, Jay K <jay.krell at cornell.edu> wrote:

> Is anyone interested in updating m3front to be multi-threaded?
>
> I haven't seen a single core system in a while.
>
> Surely each module can be compiled separately, possibly with some
> serialization around compiling interfaces?
>
> Thanks,
>  - Jay
>
>
> _______________________________________________
> M3devel mailing list
> M3devel at elegosoft.com
> https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20150810/df3f89b3/attachment-0002.html>