[M3devel] Build Server - Plan

[Jay K]

ARM_DARWIN I never quite finished.It was almost working on a jail-broken iPhone 3gs.Setting up and testing just a C development environmentis often the difficult part.
If you do have a C cross compiler yes that can shift slightly,easily where some things happen.
Let me explain.The compilation process has a few different steps it can go.Including that we have some C code.So that goes .c => .o, or internally .c => .s => .o.
Ignoring LLVM...We have the NT386 backend.Code goes right from .m3 => .obj.cm3 does "everything".

We have the gcc backend.Code goes.m3 => .mc => .ms => .o

.ms files are just assembly source.mc files are an internal format that cm3 writes out.

We have the C backend.Code goes.m3 => .c => .s => .ocm3 writes the .c files.
And then there is linking.

This is all *hightly* automated.From the users point of view, cm3 seems to go from .m3 files to a .a/.dylib/executable.

But cm3 only has within it some of the stages.It doesn't have a C compiler, an assembler, or a linker.We depend on a  "C" toolchain for this (I'm calling assembler/linker part of the C toolchain).
If you don't have the C toolchain, then you are stuck part way through those pipelines.Either with a bunch of .o files you can't link.Or a bunch of .ms files you can't assemble and link.Or a bunch of .c files you can't compile/assemble/link.

Native toolchains are historically more common than cross tool chains.So we have a "boot" workflow -- that needs work -- where on the "first" machine,you stop where I say you are "stuck", you archive up the files, copy themto the target machine, which has a native toolchain, and then resume the pipeline --running C compiler/assembler/linker.
Now, if you have a cross toolchain there is not quite reason for that.
But, it comes down mainly to the config files and the builder, and another detail I'll get to.
The "boot" workflow was well automated 20 years ago.Back when "quake" was written in C. The 3.6 release.I don't remember what the 4.0/4.1 release looked like.
I have good automated flow for producing a cm3 for cross scenarios,and it is useful, but it requires a matching source tree on the other side.
This is boot1.py and boot2.sh.
If there is an incompatibility, like I recently added to m3core, then it doesn't work.So it dawns on me now, that boot1.py should produce m3core/libm3, and maybe cm3cg.Though cm3cg would require a cross compiler, so sometimes a pain.
"another detail I'll get to".We have a small number of tools that when we build the system, we buildthem and run time. Other than cm3. There is mklib, which I already handle.But there is also m3bundle and PklFonts and some RPC stub generators.

Given a cross C toolset, we don't have conventions/automation around,hey, I'm targeting x, building on y, my y-hosted tools might need an update.I need to build all of y, then all of x, and when I'm building x, what is the cm3I use? What is the C compiler I use?

Again it is mostly just a thing with config files.The config files tend to run "gcc" and find it in $PATH.They assume it is for the current host/target.But there needs to be arguably some convention, I'm hosting on I386_LINUX,targeting AMD64_DARWIN -- where are the native tools and where are the cross tools?In gcc-land, this is basically you run ad64-apple-darwin-gcc for the AMD64 target toolset,instead of just "gcc"
You might also run e.g. /usrl/ocal/cm3/bin/<target>/cm3cg.
So, very long story short -- we largely need a file system layout.
The "GNU platform" mapping I386_DARWIN => i386-apple-darwin8 for example,is somewhat encoded in our tree, but for this to work it needs more formalization.

With LLVM, you get all the backends together in on tree and you givethe executable a command line switch and it finds the right .so, I imagine,so this sort of gets easier. We still have to know the switches and work that into config and such.We should just do that for native targets anyway and then cross will reuse that.
Again, sorry, long story short -- paths and switches to compiler/assembler/linker is largely the thing.
Apple has a good system here btw.You say gcc -arch ppc.And that combines with -x assembler.So you can cross compile with gcc -arch ppc.You can cross assemble with gcc -arch -x assembler.They put the work in the gcc driver. It then farms out to the correct cc1/cc1plus/as.

If only this was the norm..

Another thing -- a cross C compiler that can handle something with no #includes andno linking, is really easy to build. It is the headers and libraries that make it more difficult.The assembler/linker are also sometimes a sticking point, because sometimes, if youread the gcc config/install manual, they prefer the native closed source assembler/linker.

I have a working VMS C cross compiler from Mac, but I had to copy various filesfrom a running VMS machine to do it -- the headers and libraries.

In as much as the assembler/linker is GNU, it is easier.And Apple's stuff is open source and people have ported it.But imagine targeting HP-UX or AIX, or NT using the Microsoft toolset.There you can't just conjure cross tools with arbitrary host, only what the vendor ships.


 - Jay


Date: Fri, 14 Aug 2015 09:30:16 -0700
From: lists at darko.org
To: m3devel at elegosoft.com
Subject: Re: [M3devel] Build Server - Plan

What hardware setup did you use to ARM_DARWIN? I want to support ARM too. I'm not sure if a separate ARM server is in order, or a QEMU setup. I can just plug some Apple hardware into a USB port if that works. Maybe a cross compiler is the best solution.
On Thu, Aug 13, 2015 at 11:36 PM,  <microcode at zoho.com> wrote:
On Thu, Aug 13, 2015 at 01:27:54PM -0700, Darko Volaric wrote:

> I'm setting up a server for building CM3 that takes a "minimalist" approach.

>

> It's a machine running several virtual machines, one for each platform

> supported by CM3. Each VM will contain clean install of the OS plus any

> external tool dependencies. It will have a minimal compiler install,

> basically enough to compile itself for the host target.



Doesn't CM3 support Solaris SPARC? As far as I know there is no cheap way to

emulate this from Intel.



> The publicly available build products will be:

>

> - minimal executables for bootstrap, eg the frontend and a backend

> - model compiler config files

> - compilation logs for bootstrap executables

> - compilation logs for most modules in the github repository

> - logs for certain tests



This sounds like a setup for experts. Why not make a turn-key tarball

available like was available (I think) before?



> Packages, libraries, scripts and non-essential tools or executables will

> not be built or used, the idea being that people take the minimal

> bootstraps and build from there.



That's fine for the 3 or 4 core guys doing all the work! For everybody else

this is a big inhibitor to making CM3 generally useful to the rest of the

world- unless the instructions to get a complete install are very clear and

easy to follow. But it seems from watching the discussion here it is

non-trivial to get CM3 installed.





_______________________________________________

M3devel mailing list

M3devel at elegosoft.com

https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel




_______________________________________________
M3devel mailing list
M3devel at elegosoft.com
https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20150814/50915846/attachment-0002.html>