[M3devel] moving to new target names, in Hudson?

[Jay K]

Clearly this stuff is hard to describe. I think both Olaf and I sound like we're rambling. Esp. me. :)
Olaf clearly gets my point of view though. :)


> (b) is often partially provided by commercial systems like Windows, though
> in practice this doesn't always work as expected, too.


Yes it does! That's tangential though.



 > What if different loadable
> formats are supported, as under Darwin (PPC, I386, combined)

Let's table the "universal binaries" subject for now.
I think it is slightly interesting, but nobody has asked for it, and it is fairly unique to Darwin.
 It can be done either by building everything twice, or by combing files after the fact.
  Nice thing about combining after the fact is basically nothing changes.


> or Windows (com, exe (coff))?

No, really, no.

But I'll explain since you brought it up.


There are two or so formats of MS-DOS file.
A .com file is just under 64K of bytes loaded anywhere, all addresses are relative and <= 64K.
That is, there is no file format. It just must be small.
This is for 16bit code only. We don't support 16bit MS-DOS.


There is MS-DOS .exe files which can be large and are relocatable.
Again, this is a 16bit thing.


In more general reality, people write their own loaders for MS-DOS.
There's a DOS extender that loads Win32 files for example.


Now, in Windows there is just one file format, PE, COFF, PE/COFF, whatever. All the same thing.
The file *name* of such a file is arbitrary and varies.
"more.com" is of the same file format as "findstr.exe".
It just has that wierdo extension for compatibility. It is not a .com file like in MS-DOS.
P in PE is "portable", as in all versions of NT: MIPS, PowerPC, 32bit Alpha, x86, use the same format.
Later there was some revision, PE was renamed PE32, PE32+ was introduced. It is almost the same thing,
but a few fields were widened to 64bits. Still, 64bit Alpha, IA64, AMD64 all use the same format.
So there are only two formats: 32bit and 64bit. That's still nicely pretty "portable".


So, short story, drop executable file format as a variable, at least on Windows.


I understand some Unixes switched from a.out to ELF.
That is/was relevant. LINUX => LINUXELF.
But 32bit Windows never switched formats for files containing 32bit code.


True, Win3.1 and OS/2 had various NE (new executable?), LE (linear executable?) but I don't think we care.
Maybe at some point we'll have I386_OS2_NE, I386_OS2_LE but I don't know enough to say.
More likely we'd have I386_WIN31, with one file format, I386_OS2 with one file format.



> I would tend to start with something like the
> output of uname -rsm here:
>
> o FreeBSD 8.0-STABLE amd64
> o Linux 2.6.26-2-amd64 x86_64
> o FreeBSD 6.4-RELEASE-p5 i386
> o Darwin 9.8.0 i386
> o Darwin 8.11.0 Power Macintosh


I mostly agree, but I think it is maybe a little two much information in some cases.
Should just be e.g. FreeBSD 8.0 amd64.
FreeBSD 6.4 i386 -- does RELEASE p5 really help?


I think you should look at the output (not the code) of the GNU config.guess file.
I think it is mostly the way to go.
It has those dumb "unknown" and "pc" things, and the "vendor" (apple, sun, dec) is generally meaningless.
But the first and third/fourth parts are good.


It is similar to your uname -srm.


There are things beyond uname, like "sizer" on Alpha/OSF.
config.guess finds the "g" in 4.0g, but uname doesn't.


I'll admit another evidence of the problem.
PPC64_DARWIN
 I think the last release added a bunch.
 So PPC64_DARWIN < 10.5 wouldn't have any Trestle/X stuff.
 But PPC64_DARWIN10.5 would.


>From an implementation/compiler/library point of view, this can be implemented by having
the config/m3makefiles probe at config/install/runtime. I've been putting probes in at runtime.
Stuff that possibly belongs in config or install.


Now, there is this this thorny issue you mention.
We build binaries for redistribution.
This does in a sense resemble a cross build.
Building on machine X to run on machine Y, which may be more or less the same.


If you consider something like Debian where they produce tons of binary packages,
or even FreeBSD/NetBSD/OpenBSD, they have a luxury. They aren't really
doing cross builds. They know the packages will be installed on the same OS release.
They rebuild all the packages for each release.
And Ubuntu vs. Debian are different builds.


What do we do?


Perhaps we sort of go back to the old model where customers do their own build?
Or partly their own build?
I think this is a definite possibility.
And we use autoconf a bit in there? I think definitely maybe.


The "boot" packages I build are a possible start.
Though they need to come with matching Modula-3 source tree, including m3cc.
  => Just like the 3.6 DEC SRC release.


Should we work towards that?
Does it stink too much to make people build it?
It has upside and downside.
Upside is we get largely out of this configuration problem.
Downside is they have to be much more patient before using the system.


Perhaps we provide similar to what we already have, but with "overspecified configuration" names.
If we have an exact match for you, great.
If not, you build it yourself?


I don't know about these Linux kernel version numbers.
I hope/think Linux might have good compatibility?
Our usage is fairly light.


This is partly why i was interested in Modula-3 threads over Linux, instead of over glibc.
So we could avoid the ARMEL_LINUX_glibc vs ARMEL_LINUX_uclibc vs. ARMEL_LINUX_bionic (Android) question.


I'm rambling terribly, I realize. Sorry.


In conclusion...I think we probably do something like uname -srm or config.guess, distribute
"boot" + m3src.tar.gz + m3cc.tar.gz like DEC SRC did, we provide some binaries.

User can rename build_dir to whatever he/she wants, but we'd start with something like config.guess?

??


Things like word size, endian, jmpbuf size, setjmp name, move to config file?
  Try to configure these things with autoconf? Make my "boot" packages use autoconf? e.g. to compile the C?
  To find libraries and build the config file and fill out SYSTEM_LIBS? i.e. to replace cminstall?
Remove all target knowledge from Target.m3?
????


It's a bunch of work, makes these confusing (as autoconf does), but it does solve some problems

I don't know.

 - Jay


----------------------------------------
> Date: Mon, 19 Jul 2010 14:38:44 +0200
> From: wagner at elegosoft.com
> To: mika at async.async.caltech.edu
> CC: jay.krell at cornell.edu; m3devel at elegosoft.com
> Subject: Re: [M3devel] moving to new target names, in Hudson?
>
> Quoting Mika Nystrom :
> [...]
> > The point of these names, I thought, was that they were supposed to map
> > to binary-incompatible systems. I remember having FreeBSD2 and FreeBSD3
> > at the same time. We had a cluster with machines with both OSes, which
> > were not binary compatible. The point is so that you can run cm3/m3build
> > for both in the same repository (even concurrently), and not have them
> > interfere with each other. It's not a pain to have different names
> > if they actually mean different things. If you just call everything
> > I386_FREEBSD all hell will break loose in that situation... (just as when
> > I by mistake run "FreeBSD4" (really FreeBSD6) CM3 in a directory where
> > I have already compiled things with "FreeBSD4" (really FreeBSD4) PM3...)
> >
> > Or am I misunderstanding something here? I would have thought you really
> > wanted to have a different name for each binary-incompatible system so
> > you can build them all in the same place. Consistency in naming would be
> > nice but I think it is secondary.
>
> In theory, yes, but it hasn't always worked out in practice.
> Binary compatibility works at least in two directions:
>
> (a) Allow applications compiled on an older system with older tools
> and libraries to be run on a newer system.
>
> (b) Allow applications compiled on a newer system to run on all or
> certain selected older system versions.
>
> Usually (a) is provided by Open Source systems like Linux, FreeBSD etc.,
> at least within a major release line, sometimes also crossing boundaries
> (remember the switch from aout to elf format in FreeBSD).
>
> (b) is often partially provided by commercial systems like Windows, though
> in practice this doesn't always work as expected, too.
>
> It would be nice if we really could follow all binary compatibility
> changes in all supported operating system targets with a new unique
> name, but I'm afraid we haven't been able to do that in the past nor
> will we succeed to do that in the future.
>
> You have experienced one failure as you tried the FreeBSD4 packages
> built on FreeBSD 6.4 on an actual FreeBSD 4.11 system.
>
> We may get better if we distinguish between different uses of the target
> platform name though:
>
> (1) TARGET as a choice for compilation variants that cm3 distinguishes
> internally
>
> (2) TARGET as a name for the directories containing derived files (which is
> already configurable in quake config files). For example, the profiling
> setups I made some years ago added a `p' to the TARGET name.
>
> (3) TARGET as an indicator for installation or other binary packages
> on which system these binaries may run
>
> (3) should include as many information as possible, though it is not
> clear to me where to stop here. Do we actually want the X version or
> the GUI support libraries to be included there? What if different loadable
> formats are supported, as under Darwin (PPC, I386, combined) or Windows
> (com, exe (coff))? I would tend to start with something like the
> output of uname -rsm here:
>
> o FreeBSD 8.0-STABLE amd64
> o Linux 2.6.26-2-amd64 x86_64
> o FreeBSD 6.4-RELEASE-p5 i386
> o Darwin 9.8.0 i386
> o Darwin 8.11.0 Power Macintosh
>
> We could also add a more or less unspecified variant part there, for
> example, if we need to distinguish between aout/coff/elf format.
>
> (1) should be the list of distinctions we need to make to support
> different target platforms. The current compromise seems to be that
> this is a combination of processor/machine architecture (like I386, AMD64)
> and operating system name (not version).
>
> Then (2) could be an installation-specific choice: either just use
> the more generic name from (1), or a combination of (1) and (3), or
> some arbitrary name. It's not completely thought through yet, so suggestions
> on what exactly would be required or great are welcome.
>
> I've already added the information of (3) to the CM3 5.8.6 download
> pages; the next release should include it in the actual package names.
>
> (1) should be consistently standardized if possible (i.e. FreeBSD1/2/3/4
> --> I386_FREEBSD etc.). Many actual configuration options may be
> determined by auto-configuration then.
>
> Olaf
> --
> Olaf Wagner -- elego Software Solutions GmbH
> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
>