<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
</head>
<body class='hmmessage'>
ps: another suggestion here:<BR>
Don't use overrides in Hudson or package building.<BR>
Instead either build from scratch or with a new install that is a copy of the current (ie. of m3core/libm3). I alread automated this in make-dist.py, which is a based on some other *.sh.<BR>
And then copy/swap at the end, or after some testing.<BR>
Except, er, um, either this is with $ORIGIN, or with relink upon install -- not just copy/swap.<BR>
<BR>
<BR>
The central sticking point in "all" this, in software configuration in general -- is "run path".<BR>
<BR>
<BR>
And then, delete all the m3overrides files in the tree. And then optionally remove the entire mechanism, or leave it alone, unused, to bitrot.<BR>
<BR>
<BR>
To some extent this is: don't bother implementing a mechanism in cm3, just do it in scripts.<BR>
<BR>
> M3 build system for large projects is based on the way the work<BR>> of the single developers can be split up and isolated. And in large<BR><BR>
<BR>
Imagine a system composed of five pieces: bottom, almost bottom, middle, almost top, top.<BR>
The developer of "top" has it easy, he uses the "existing" bottom, almost bottom, middle, almost top.<BR>
What are the developers of any other piece to do?<BR>
There are at least two factors to consider:<BR>
run path <BR>
public interfaces <BR>
<BR>
Even if public interfaces are unchanged, developer building a new "almost top", must either put it where "top" expects, or rebuild top to get it from the new location.<BR>
<BR>
<BR>
It has been said that LD_LIBRARY_PATH is for development scenarios. So maybe that is the answer.<BR>
<BR>
<BR>
To whatever extent systems are composed of just bottom and top and nobody changes bottom, which I think is actually pretty common, no problem, and people don't even understand the potential problem (I theorize). But that seems..wrong.<BR>
<BR>
<BR>
- Jay<BR> <BR>
<HR id=stopSpelling>
From: jay.krell@cornell.edu<BR>To: wagner@elegosoft.com; m3devel@elegosoft.com<BR>Date: Wed, 4 Aug 2010 10:56:13 +0000<BR>Subject: Re: [M3devel] m3override<BR><BR>
<META name=Generator content="Microsoft SafeHTML">
<STYLE>
.ExternalClass .ecxhmmessage P
{padding:0px;}
.ExternalClass body.ecxhmmessage
{font-size:10pt;font-family:Tahoma;}
</STYLE>
Specifically it is a warning about overriding the package you are building.<BR> <BR> <BR>The thing is, isn't the "bits and pieces overridden" model prone to just not work?<BR>Which pieces do you build vs. which do you leave alone? You need "everything" to be<BR>"fairly" consistent, for some definition of "everything" and "fairly".<BR> <BR> <BR>There is a larger unsolved problem here -- which parts of a system can you change<BR>independently of which parts?<BR> <BR> <BR>There are two extremes: one thing at a time, or everything at a time.<BR>The truth is generally in between: some things at a time. Which?<BR> <BR> <BR>"Everything" at a time is more difficult the larger you define "everything".<BR>"one" thing at a time, and again "one" possibly varies -- maybe you distribute .o files?<BR>"one" thing at at time leads to a combinatorial explosion of combination of "things"<BR>that are "supposed" to work. With some (enough?) runtime checking that a valid<BR>composition has been made. (Certainly Modula-3 tries much harder here than C.)<BR> <BR> <BR> <BR>I kind of espouse a model of "everything at time", but you can "pre-seed" the new package store/install<BR>with a copy/link of the original (as long you delete before writing over links).<BR> <BR> <BR>> work well in this setup (more or less). For one notable exception,<BR>> which others have noticed (who were involved with the original design):<BR>> linking has never been tried implement portably in a general way.<BR><BR> <BR>Do you mean like, regarding $ORIGIN? Or something else?<BR>I'm too blame for giving us $ORIGIN, and I'm now very ambivalent about it.<BR>libtool + relink upon install seems the better approach and is widely used in practise.<BR> e.g. OpenBSD.<BR> This has a side benefit that uninstalled/unshipped dynamic binaries do work.<BR> <BR> <BR>I do suspect $ORIGIN was an improvement, since we don't relink upon install.<BR> <BR> <BR>There is also a hybrid suggestion of what I have suggested and the current model.<BR>You can still "build" into a new empty directory, with a hierarchy matching the install,<BR>leaving open either later install in-place or "reroot".<BR> <BR> <BR>I think, really, I have to back down from $ORIGIN, which restores "pick and chose"<BR>to work when it can, we can still, as I said, build into an install-like structure, and install<BR>by "reroot" or "delete and copy". I think the key is doing relink upon install.<BR>Along with, no longer distributing fully built binaries.<BR>There is a big task here but I think we have seen enough real problems and discussed<BR>enough the theory, to end up at a conclusion -- no $ORIGIN, no binaries, distribute<BR>C source, assembly source (or maybe .o files), and do final link on the target.<BR> <BR> <BR>We can distribute some common binaries, like with hardcoded install location.<BR> The hudon builds might be tricky, since if we build binaries to install to /usr/local/cm3,<BR> we can't then install and test them. Unless maybe chroot is possible as non-root??<BR> <BR> <BR>Possibly Hudson builds would install to $HOME/... and just the final distribution wouldn't.<BR>Probably.<BR> <BR> <BR>I kind of..can't think of the word...really don't like this conclusion. Am nervous.<BR>How many people install binaries vs. compile/link?<BR>This would be signing up for much additional headache all around? Compile/link would<BR>fail for lots of people and we'd have to help them a lot more?<BR>Or it's not that bad and we just have to do it?<BR> <BR> <BR> - Jay<BR><BR> <BR>> Date: Wed, 4 Aug 2010 08:49:30 +0200<BR>> From: wagner@elegosoft.com<BR>> To: m3devel@elegosoft.com<BR>> Subject: Re: [M3devel] m3override<BR>> <BR>> I wouldn't worry much about the warning Jay removed.<BR>> It only said that an override wasn't used, which is the default<BR>> behaviour of cm3 (without -x or -override option). One could argue<BR>> that if one wanted this warning, one should use an option like<BR>> -warn=overrides or -warn=all (which don't actually exist (yet)).<BR>> <BR>> However, I'd still like others to think about the two perspectives<BR>> one can take here, and what should be the direction we are tweaking<BR>> our build system.<BR>> <BR>> Traditionally, the focus was on the package view, i.e. the view of<BR>> one developer, and not a whole system. The builder was designed to<BR>> work well in this setup (more or less). For one notable exception,<BR>> which others have noticed (who were involved with the original design):<BR>> linking has never been tried implement portably in a general way.<BR>> With this focus, overrides are the sole responsibility of the<BR>> developer, and one could argue that he should know exactly what<BR>> he does if he uses them.<BR>> <BR>> In the CM3 distribution, I used the override feature to allow<BR>> building the whole set of package in the workspace, which can be<BR>> seen equivalent to use a completely different package pool.<BR>> The scripts (which originally were only meant as a tool for the<BR>> distribution maintainer) distinguish between build(local) and<BR>> buildglobal (or buildship). In the first case, the global package<BR>> store isn't involved at all.<BR>> <BR>> Jay has suggested to optimize towards building the whole distribution,<BR>> but I'm not sure yet if that's the correct direction. Obviously,<BR>> we need to support both perspectives. Much of the charm of the<BR>> M3 build system for large projects is based on the way the work<BR>> of the single developers can be split up and isolated. And in large<BR>> projects there are of course more developers than build or release<BR>> engineers. But then, there seems to have been no real use of CM3 for this<BR>> kind of development for many years; most (all?) active users seem to<BR>> work on their projects alone. So perhaps Jay is right after all?<BR>> <BR>> Olaf<BR>> -- <BR>> Olaf Wagner -- elego Software Solutions GmbH<BR>> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany<BR>> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95<BR>> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin<BR>> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194<BR>> <BR> </body>
</html>