[M3devel] Hudson updating config files or not
Olaf Wagner
wagner at elegosoft.com
Fri Jul 23 08:39:49 CEST 2010
The distinction between lastok-build and release-build jobs is
obsolete; they have been combined to the -build- jobs ,amy months
ago.
Just forget about all lastok-build and release-build jobs that may still
linger there; they should all be disabled.
The -build- jobs should all try a straight forward compile with the
cm3 built the last time; if that fails, they will clean everything
and start over again with upgrade.sh. That's the only one where
configuration files used to get changed.
Only the -build- and -test- jobs have been copied over to current.
Almost all sources for the hudson jobs should reside in the
scripts/regression directory and be under version control.
I don't think updates/merges are needed there.
Where I wasn't so sure are the scripts one level above.
Olaf
Quoting Jay K <jay.krell at cornell.edu>:
> I'm not sure, but here is some analysis.
>
> In the release hudson tasks, we have
> http://hudson.modula3.com:8080/view/SOLgnu/job/cm3-release-build-SOLgnu/110/consoleFull
> and
> http://hudson.modula3.com:8080/view/SOLgnu/job/cm3-build-SOLgnu/163/console
>
>
> "build" and "release-build".
> I think these are, like, "build from last release" and "build from
> latest successful build".
> I'm not sure what they do, but instead of fighting over installs and
> CVS trees, they should perhaps just each maintain their own.
> That way, e.g., they could both update config files.
>
> Well, except if we are low on diskspace, which we are. And it is a
> defficiency of the system to put outputs in the source tree.
> We should be able to ge by with one source tree per node, and all
> outputs should go elsewhere, including the PGS files and
> including those install.sh that get generated. The source tree
> should never be written to, as an option at least. Anyway.
>
>
> In regression/defs.sh I believe they go here:
>
>
> test_build_current
> ...
> if [ "$1" = "rel" ]; then # This is the important recurring
> thing that makes things different?
>
> "release" goes here, "build" does not
> This path does update config files.
>
>
> echo " === clean up before cm3 upgrade "
> if [ -z "$NOCLEAN" ]; then
> OMIT_GCC=yes ./scripts/do-cm3-core.sh realclean || exit 1
> ./scripts/do-pkg.sh realclean cminstall
> fi
> echo " === perform cm3 upgrade "
> UPGRADE_CM3_CFG=yes ./scripts/upgrade.sh || exit 1
> echo " >>> OK build_${1}_upgrade ${DS} ${WS}"
> fi
>
>
> and then:
>
>
> echo " === build ${BSET} system with current compiler"
> BUILDSCRIPT="./scripts/do-cm3-${BSET}.sh"
> if [ "$1" = "rel" ]; then # This is the important recurring
> thing that makes things different?
> if [ -z "$NOCLEAN" ]; then
> OMIT_GCC=yes ./scripts/do-cm3-core.sh realclean || exit 1
> fi
> else
> if [ -z "$NOCLEAN" ]; then
> $BUILDSCRIPT realclean || exit 1
> fi
> fi
> $BUILDSCRIPT buildship || exit 1
>
>
> or, oh no, now I'm confused, here?:
>
>
> test_build_system
>
> test_build_system and test_build_current are very similar! Confusing?!
>
>
> echo " === build core system with current compiler"
> BUILDSCRIPT="./scripts/do-cm3-core.sh"
> if [ -z "$NOCLEAN" ]; then
> $BUILDSCRIPT realclean || exit 1
> fi
> if $BUILDSCRIPT build; then
> echo " >>> OK build_system ${DS} ${WS}; now rebuild and ship"
> $BUILDSCRIPT buildship
> echo " === install new compiler"
>
> This is what we are running for head, I think.
> Historically it didn't touch config files.
> This is what I changed a day or so ago, to update them.
>
> if ./scripts/install-cm3-compiler.sh upgrade; then
> echo "compiler upgraded successfully"
> cm3 -version
> else
> echo "compiler upgrade failed" 1>&2
> exit 1
> fi
> else
>
> There is this which will update config files, but only if the
> previous fails.
>
> echo " === perform cm3 upgrade after cleaning everything"
> $BUILDSCRIPT realclean || exit 1
> UPGRADE_CM3_CFG=yes ./scripts/upgrade.sh || exit 1
> echo " >>> OK build_upgrade ${DS} ${WS}"
> fi
>
>
> Anyway, it is confusing, release has a pair of tasks and only some
> paths updated config files, even if other/more/different paths
> update the compiler.
>
>
> I think:
> - we want the pairs of tasks, at least for testing coverage
> - or, I guess, just build from previous release
> which can be constraining, it is a policy issue
> - whichever tasks we have, we should have config file updating
> - tangentially: we should get all outputs, *all* outouts out of
> the source tree
> The original design was nice, get outputs "per package" outside
> of source,
> but it failed to address a multi-package tree. This is a big
> change I'm proposing.
> I guess another mail on the topic.
>
> - Jay
>
>
> ----------------------------------------
>> From: jay.krell at cornell.edu
>> To: wagner at elegosoft.com
>> CC: m3commit at elegosoft.com
>> Subject: RE: [M3commit] CVS Update: cm3
>> Date: Thu, 22 Jul 2010 07:21:33 +0000
>>
>>
>> (ps: the change I made does do the backup foo-date thing, parallel
>> to cm3 and cm3cg
>> yes I'll compare the branches, can't promise a full merge though)
>>
>>
>> ----------------------------------------
>> > Date: Thu, 22 Jul 2010 09:19:12 +0200
>> > From: wagner at elegosoft.com
>> > To: jay.krell at cornell.edu
>> > CC: m3commit at elegosoft.com
>> > Subject: Re: [M3commit] CVS Update: cm3
>> >
>> > I can live with a cm3-local.cfg file which gets included when it exists,
>> > contains local overrides and is never touched by he installation.
>> >
>> > We have to be careful to backup existing configs and informing the
>> > users before the installation though.
>> >
>> > I think upgrade did backups of the config, too.
>> >
>> > Can you check the scripts in the release branch if anything needs
>> > to be merged back? I won't be able to do that today.
>> >
>> > Olaf
>> >
>> > Quoting Jay K :
>> >
>> > > I thought it was too, before I looked at it.
>> > > Could be that head and release are very divergent.
>> > > I didn't look at release, oops.
>> > > I probably will "soon".
>> > >
>> > >
>> > > The config files are "partly part of the compiler", and partly
>> > > system-specific.
>> > > So they might have to be updated with the compiler.
>> > >
>> > >
>> > > Perhaps we can have cm3.cfg.user or cm3.cfg.local, that we never edit,
>> > > and maybe we should constrain it, like very line
>> > > must have an equals sign, and if it has a percent, percent must
>> > > follow equals.
>> > >
>> > >
>> > > e.g.
>> > > m3back_flags = "-custom"
>> > > SYSTEM_CC = "custom" % comment blah blah
>> > > SYSTEM_LIBS{"LIBC"} = "custom"
>> > > SYSTEM_LIBORDER += [custom]
>> > >
>> > > But that might not be flexible enough.
>> > >
>> > > Another option is that cm3.cfg file do like:
>> > > if exists("cm3.cfg.user.first")
>> > > include("cm3.cfg.user.first")
>> > > end
>> > > ...
>> > > if exists("cm3.cfg.user.last")
>> > >
>> > > include("cm3.cfg.user.last
>> > >
>> > > end
>> > >
>> > >
>> > >
>> > > which is infinitely flexible and not well defined.
>> > > Even limiting to assignments is not well defined.
>> > > You know, there's an interface somewhere in there, but it isn't
>> > > well specified. My fault.
>> > > I let things evolve and get discovered gradually, sometimes hard
>> > > to know ahead of time what the result will be.
>> > >
>> > >
>> > > I've made changes since 5.8.6 released such that config files from
>> > > prior to 5.8.6 will not suffice.
>> > > I can undo that, but there is an important policy and
>> architecture question.
>> > >
>> > >
>> > > - Jay
>> > >
>> > > ----------------------------------------
>> > >> Date: Thu, 22 Jul 2010 08:53:23 +0200
>> > >> From: wagner at elegosoft.com
>> > >> To: m3commit at elegosoft.com
>> > >> Subject: Re: [M3commit] CVS Update: cm3
>> > >>
>> > >> Quoting Jay K :
>> > >>
>> > >> > I've manually updated the config files on SPARC32_LINUX Hudson node
>> > >> > to try to fix this.
>> > >>
>> > >> That should be done by the upgrade script (and it already was AFAIR).
>> > >> I wouldn't like the script shipping a newly built compiler to
>> > >> unconditionally overwrite all my local configuration.
>> > >>
>> > >> Olaf
>> > >>
>> > >> >
>> > >> > - Jay
>> > >> >
>> > >> > ----------------------------------------
>> > >> >> Date: Wed, 21 Jul 2010 17:53:14 +0000
>> > >> >> To: m3commit at elegosoft.com
>> > >> >> From: jkrell at elego.de
>> > >> >> Subject: [M3commit] CVS Update: cm3
>> > >> >>
>> > >> >> CVSROOT: /usr/cvs
>> > >> >> Changes by: jkrell at birch. 10/07/21 17:53:14
>> > >> >>
>> > >> >> Modified files:
>> > >> >> cm3/scripts/: install-cm3-compiler.sh
>> > >> >>
>> > >> >> Log message:
>> > >> >> upgrade config files when upgrading compiler
>> > >> >>
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Olaf Wagner -- elego Software Solutions GmbH
>> > >> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
>> > >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30
>> 23 45 86 95
>> > >> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
>> > >> Handelregister: Amtsgericht Charlottenburg HRB 77719 |
>> USt-IdNr: DE163214194
>> > >>
>> > >
>> >
>> >
>> >
>> > --
>> > Olaf Wagner -- elego Software Solutions GmbH
>> > Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
>> > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
>> > http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
>> > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr:
>> DE163214194
>> >
>>
>
--
Olaf Wagner -- elego Software Solutions GmbH
Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
More information about the M3devel
mailing list