[M3devel] Output from "cron" command

Fri Jan 15 07:07:56 CET 2010

Not quite. shipping cm3 doesn't put it in the right place, so you have to "manually" copy it.

At some point I'd like to see about making that work better
but it isn't high priority.

In particular, on Windows you can rename away an in-use .exe/.dll and then copy over.

But I think I've seen the behavior on other systems that replacing in-use executables/.so files causes a "random" mix of in-memory code/data and crashes. (And I know Windows is usually criticized for not letting you replace "running code", but the reality is more lenient than people realize, the reality of allowing such things is far more complicated than people realize (how do I ensure the old version eventually is not used?) and I believe other systems have similar or more stringent restrictions, e.g. AIX).

We should probably make shipping in cminstall put the configuration files in place or somesuch, instead of leaving it to all the scripts.
There is a tension in that programming in Python being much easier and more pleasant than any of the alternatives such as Modula-3 or Quake..

Anyway, I just use the *.py files, on Linux, Darwin, NT, OpenBSD, FreeBSD, NetBSD, Solaris, etc. (MIPS64_OPENBSD so far I don't have Python for, it's like the only one).

 - Jay

----------------------------------------
> From: hosking at cs.purdue.edu
> Date: Thu, 14 Jan 2010 23:49:18 -0500
> To: rcolebur at SCIRES.COM
> CC: m3devel at elegosoft.com; jay.krell at cornell.edu
> Subject: Re: [M3devel] Output from "cron" command
>
> I think so. Jay can comment more accurately re Windows.
>
> On 14 Jan 2010, at 23:45, Coleburn, Randy wrote:
>
>> Jay / Tony:
>>
>> I have tried to capture this scenario in my latest scripts in "scripts/dev/windows".
>>
>> RCC_upgrade.cmd is my version of Jay's upgrade.py, but it uses Windows CMD files only; no python dependencies, AND it parses "pkgInfo.txt".
>>
>> Do-cm3.cmd is my version of the various "do.sh" scripts rolled up into one CMD file. It depends on "pkgInfo.txt".
>>
>> RCC_upgrade.cmd assumes you have a working cm3 installation, then does the following:
>> 1. Copies some of the other scripts and documentation I use to the "bin" folder.
>> 2. Copies all the cm3.cfg config stuff set up by Jay to the appropriate places.
>> 3. Stage-1: Builds and Ships all packages in the "front" group, except for "m3core", "libm3", and "mklib".
>> 4. Stage-2: Builds and ships all packages in the "front" group.
>> 5. Stage-3: Builds and ships a distribution. Default is the "min" group, but you can use the "-all" parameter to build everything, i.e., a complete distribution.
>>
>> Jay, do these 5 points reliably and succinctly express what you are saying; or am I missing something else?
>>
>> Regards,
>> Randy Coleburn
>>
>> -----Original Message-----
>> From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
>> Sent: Thursday, January 14, 2010 6:59 PM
>> To: Tony
>> Cc: m3devel
>> Subject: Re: [M3devel] Output from "cron" command
>>
>>
>> Tony you know all this.
>> This email is redundant for you.
>>
>>
>> A near current release_branch cm3 cannot build a current release_branch m3core, due to the VAL changes regarding LONGINT. A current release_branch cm3 can.
>>
>>
>> We have as I understand "pairs" of automated builds.
>> Though actually more than two could make sense.
>>
>>
>> There is the variety that starts with the "last release", builds "up to cm3" but skipping m3core/libm3, and then I guess rebuilds "up to cm3" with the new cm3 not skipping m3core/libm3, and then builds reverything. (possibly the second step is skipped).
>>
>>
>> If this works, its cm3 output can be "blessed" as "something".
>>
>>
>>
>> There is the variety or varieties that starts with the most recent "blessed "something" and either does the same process, or never skips libm3/m3core.
>>
>>
>> Again the output can be "blessed" as "something".
>>
>>
>> Whatever succeeds should be identical, so there isn't a need to "bless" all the outputs. However there can be failures.
>>
>>
>> There are so many of these "somethings"; I don't know what they are called. I also hadn't thought through one of the points until just now: that keeping all the outputs is redundant, but you aren't guaranteed which ones you'll get.
>>
>>
>> Hm. What should always work actually, is starting with the "latest blessed something", and *skip* m3core/libm3.
>>
>>
>> All this is to say, that, neither the "latest (or older) release" nor the "latest build" can guaranteeably build latest m3core/libm3.
>>
>>
>> This varies in time, but the typical problem with "releases" is lack of LONGINT. The current problem with recent builds is VAL.
>>
>>
>> To further try to summarize, there should probably be *three* recurring builds of each architecture. However, you don't really have to always do all three, you can "fallback" in case of failure. In particular, if you are starting with "latest build", you would first try building m3core and libm3. If that succeeds proceeds -- ship them. If that fails, well, you still proceed, but you remember to do another go around, which maybe you want to do anyway (builds are so fast on NT386 anyway, throw in a few extra :) )
>>
>>
>> You know all this already, I know, but maybe my attempt to summarize yet again will help spread the word. It is not the easiest thing to understand.
>>
>>
>> - Jay
>>
>>
>>
>> ________________________________
>>> From: hosking at cs.purdue.edu
>>> Date: Thu, 14 Jan 2010 18:24:06 -0500
>>> To: jay.krell at cornell.edu
>>> CC: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] Output from "cron" command
>>>
>>>
>>>
>>> I'm not sure what that implies...
>>>
>>>
>>> Antony Hosking | Associate Professor | Computer Science | Purdue University
>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 14 Jan 2010, at 17:21, Jay K wrote:
>>>
>>> "lastok" of the release branch, which this appears to be, was expected to fail.
>>> (and for head was expected a few days ago)
>>> Previous compiler can't build m3core.
>>> Must first build compiler, then m3core.
>>> The usual occasional problem.
>>>
>>> - Jay
>>>
>>> ________________________________
>>> From: hosking at cs.purdue.edu
>>> Date: Thu, 14 Jan 2010 14:57:15 -0500
>>> To: rcolebur at SCIRES.COM
>>> CC: m3devel at elegosoft.com
>>> Subject: Re: [M3devel] Output from "cron" command
>>>
>>> Hmm. The only other commits I've seen are Jay's. Jay?
>>>
>>> On 14 Jan 2010, at 14:34, Coleburn, Randy wrote:
>>>
>>> No, the only scripts I've changed are mine in scripts/dev/windows. These don't have anything to do with Tinderbox.
>>> Regards,
>>> Randy
>>>
>>> From: Tony Hosking [mailto:hosking at cs.purdue.edu]
>>> Sent: Thursday, January 14, 2010 12:41 PM
>>> To: Tony Hosking
>>> Cc: m3devel m3devel
>>> Subject: Re: [M3devel] Output from "cron" command
>>>
>>> Looks like the change in the last 24 hours to the build scripts results in improper bootstrapping of the compiler...
>>>
>>> http://tinderbox.elegosoft.com/tinderbox/cgi-bin/gunzip.cgi?tree=cm3&full-log=1263489074.21419
>>>
>>> Randy/Jay, did you change something related to this?
>>>
>>> Antony Hosking | Associate Professor | Computer Science | Purdue University
>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>> Office +1 765 494 6001 +1 765 494 6001 | Mobile +1 765 427 5484 +1 765 427 5484
>>>
>>>
>>>
>>>
>>>
>>> On 14 Jan 2010, at 12:14, Tony Hosking wrote:
>>>
>>>
>>> Your "cron" job on niagara.cs.purdue.edu
>>> $HOME/cm3/scripts/regression/cron.sh
>>>
>>> produced the following output:
>>>
>>> GMAKE=gmake
>>> export GMAKE
>>> TAR=gtar
>>> export TAR
>>> TESTHOSTNAME=niagara
>>> WS=/homes/hosking/work/cm3-ws/niagara-2010-01-14-11-30-07
>>> LASTREL=5.4.0
>>> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0
>>> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok
>>> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok
>>> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current
>>> CM3_OSTYPE=POSIX
>>> CM3_TARGET=SOLgnu
>>> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> CM3CVSSERVER=birch.elegosoft.com
>>> CM3CVSROOT=birch.elegosoft.com:/usr/cvs
>>> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> CM3CVSUSER=
>>> testing ssh birch.elegosoft.com..
>>> ssh birch.elegosoft.com ok
>>> Building cm3.
>>> Tinderbox Tree: "cm3"
>>> Buildname: "SOLgnu SunOS 5.10 niagara release-build"
>>>
>>> creating log file /tmp/build-cm3-20100114-063009-n6aGOt/log.txt
>>>
>>> ---
>>>
>>> checkout, compile and test of cm3 ...
>>> 2010.01.14 06:30:09 -- checkout in progress.
>>> [start checkout 2010.01.14 06:30:17]
>>> cd /tmp/build-cm3-20100114-063009-n6aGOt/build
>>> cvs return value: 0
>>> [end checkout 2010.01.14 06:45:08]
>>> CHECKOUT_RETURN = 0
>>> --
>>> 2010.01.14 06:45:13 -- compile in progress.
>>> [start compile 2010.01.14 06:45:13]
>>> compile return value: 0
>>> [end compile 2010.01.14 08:49:36]
>>> COMPILE_RETURN = 0
>>> 2010.01.14 08:49:56 -- tests in progress.
>>> [start run-tests 2010.01.14 08:49:56]
>>> cd /tmp/build-cm3-20100114-063009-n6aGOt/build
>>> [end run-tests 2010.01.14 11:02:11]
>>> TESTS_RETURN = 510
>>> *** TESTS FAILED
>>> removing build tree /tmp/build-cm3-20100114-063009-n6aGOt ...
>>> cleaning CM3 workspaces...
>>> /homes/hosking/work/cm3-ws/niagara-*
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n /homes/hosking/work/cm3-ws/niagara-2010-01-11-16-05-08
>>>
>>> cleaning regression test log files...
>>> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> cleaning m3test log files...
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n /homes/hosking/tmp/cm3/niagara/m3tests-2010-01-09-11-30-05.stderr
>>>
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n /homes/hosking/tmp/cm3/niagara/m3tests-2010-01-09-11-30-05.stderr.extract
>>>
>>> cleaning snapshot files...
>>> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> cleaning package reports...
>>> /tmp/cm3-pkg-report-SOLgnu-*.html
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> done with cleanup_all
>>> GMAKE=gmake
>>> export GMAKE
>>> TAR=gtar
>>> export TAR
>>> TESTHOSTNAME=niagara
>>> WS=/homes/hosking/work/cm3-ws/niagara-2010-01-14-16-05-02
>>> LASTREL=5.4.0
>>> INSTROOT_REL=/homes/hosking/work/cm3-inst/niagara/rel-5.4.0
>>> INSTROOT_POK=/homes/hosking/work/cm3-inst/niagara/prev-ok
>>> INSTROOT_LOK=/homes/hosking/work/cm3-inst/niagara/last-ok
>>> INSTROOT_CUR=/homes/hosking/work/cm3-inst/niagara/current
>>> CM3_OSTYPE=POSIX
>>> CM3_TARGET=SOLgnu
>>> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> CM3CVSSERVER=birch.elegosoft.com
>>> CM3CVSROOT=birch.elegosoft.com:/usr/cvs
>>> BINDISTMIN_NAME=cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> BINDISTMIN=/homes/hosking/work/cm3-min-POSIX-SOLgnu-5.4.0.tgz
>>> CM3CVSUSER=
>>> testing ssh birch.elegosoft.com..
>>> ssh birch.elegosoft.com ok
>>> Building cm3.
>>> Tinderbox Tree: "cm3"
>>> Buildname: "SOLgnu SunOS 5.10 niagara lastok-build"
>>>
>>> creating log file /tmp/build-cm3-20100114-110504-6ga4NN/log.txt
>>>
>>> ---
>>>
>>> checkout, compile and test of cm3 ...
>>> 2010.01.14 11:05:04 -- checkout in progress.
>>> [start checkout 2010.01.14 11:05:06]
>>> cd /tmp/build-cm3-20100114-110504-6ga4NN/build
>>> cvs return value: 0
>>> [end checkout 2010.01.14 11:20:02]
>>> CHECKOUT_RETURN = 0
>>> --
>>> 2010.01.14 11:20:08 -- compile in progress.
>>> [start compile 2010.01.14 11:20:08]
>>> compile return value: 1
>>> [end compile 2010.01.14 12:11:12]
>>> COMPILE_RETURN = 1
>>> *** COMPILE FAILED
>>> removing build tree /tmp/build-cm3-20100114-110504-6ga4NN ...
>>> cleaning CM3 workspaces...
>>> /homes/hosking/work/cm3-ws/niagara-*
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n /homes/hosking/work/cm3-ws/niagara-2010-01-12-11-30-05
>>>
>>> cleaning regression test log files...
>>> /homes/hosking/tmp/cm3/niagara/cm3-rlog-*
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> cleaning m3test log files...
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stdout
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> /homes/hosking/tmp/cm3/niagara/m3tests-*.stderr.extract
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> cleaning snapshot files...
>>> /homes/hosking/tmp/cm3/niagara/cm3-min-POSIX-SOLgnu-*-*.tgz
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> cleaning package reports...
>>> /tmp/cm3-pkg-report-SOLgnu-*.html
>>> cleanup_all_but_last_n
>>> cleanup_all_but_last_n
>>>
>>> done with cleanup_all
>>>
>>>
>>>
>>>
> 		 	   		  