[M3devel] Hudson updating config files or not

Jay K jay.krell at cornell.edu
Thu Jul 22 16:25:06 CEST 2010 

I'm not sure, but here is some analysis.



In the release hudson tasks, we have
http://hudson.modula3.com:8080/view/SOLgnu/job/cm3-release-build-SOLgnu/110/consoleFull
and
http://hudson.modula3.com:8080/view/SOLgnu/job/cm3-build-SOLgnu/163/console


"build" and "release-build".
I think these are, like, "build from last release" and "build from latest successful build".
I'm not sure what they do, but instead of fighting over installs and CVS trees, they should perhaps just each maintain their own.
That way, e.g., they could both update config files.

  Well, except if we are low on diskspace, which we are. And it is a defficiency of the system to put outputs in the source tree.
  We should be able to ge by with one source tree per node, and all outputs should go elsewhere, including the PGS files and
  including those install.sh that get generated. The source tree should never be written to, as an option at least. Anyway.


In regression/defs.sh I believe they go here:


test_build_current
...
  if [ "$1" = "rel" ]; then   # This is the important recurring thing that makes things different? 
  
  "release" goes here, "build" does not
  This path does update config files.
  
 
    echo " === clean up before cm3 upgrade "
    if [ -z "$NOCLEAN" ]; then
      OMIT_GCC=yes ./scripts/do-cm3-core.sh realclean || exit 1
      ./scripts/do-pkg.sh realclean cminstall
    fi
    echo " === perform cm3 upgrade "
    UPGRADE_CM3_CFG=yes ./scripts/upgrade.sh || exit 1
    echo " >>> OK build_${1}_upgrade ${DS} ${WS}"
  fi


and then:


  echo " === build ${BSET} system with current compiler"
  BUILDSCRIPT="./scripts/do-cm3-${BSET}.sh"
  if [ "$1" = "rel" ]; then   # This is the important recurring thing that makes things different? 
    if [ -z "$NOCLEAN" ]; then
      OMIT_GCC=yes ./scripts/do-cm3-core.sh realclean || exit 1
    fi
  else
    if [ -z "$NOCLEAN" ]; then
      $BUILDSCRIPT realclean || exit 1
    fi
  fi
  $BUILDSCRIPT buildship || exit 1


or, oh no, now I'm confused, here?:


test_build_system

  test_build_system and test_build_current are very similar! Confusing?!
  

  echo " === build core system with current compiler"
  BUILDSCRIPT="./scripts/do-cm3-core.sh"
  if [ -z "$NOCLEAN" ]; then
    $BUILDSCRIPT realclean || exit 1
  fi
  if $BUILDSCRIPT build; then
    echo " >>> OK build_system ${DS} ${WS}; now rebuild and ship"
    $BUILDSCRIPT buildship
    echo " === install new compiler"
    
      This is what we are running for head, I think.
      Historically it didn't touch config files.
      This is what I changed a day or so ago, to update them.

    if ./scripts/install-cm3-compiler.sh upgrade; then
      echo "compiler upgraded successfully"
      cm3 -version
    else
      echo "compiler upgrade failed" 1>&2
      exit 1
    fi
  else
  
    There is this which will update config files, but only if the previous fails.

    echo " === perform cm3 upgrade after cleaning everything"
    $BUILDSCRIPT realclean || exit 1
    UPGRADE_CM3_CFG=yes ./scripts/upgrade.sh || exit 1
    echo " >>> OK build_upgrade ${DS} ${WS}"
  fi


Anyway, it is confusing, release has a pair of tasks and only some
paths updated config files, even if other/more/different paths update the compiler.


I think:
  - we want the pairs of tasks, at least for testing coverage 
  - or, I guess, just build from previous release
    which can be constraining, it is a policy issue
  - whichever tasks we have, we should have config file updating
  - tangentially: we should get all outputs, *all* outouts out of the source tree
     The original design was nice, get outputs "per package" outside of source,
     but it failed to address a multi-package tree. This is a big change I'm proposing.
     I guess another mail on the topic.

 - Jay


----------------------------------------
> From: jay.krell at cornell.edu
> To: wagner at elegosoft.com
> CC: m3commit at elegosoft.com
> Subject: RE: [M3commit] CVS Update: cm3
> Date: Thu, 22 Jul 2010 07:21:33 +0000
>
>
> (ps: the change I made does do the backup foo-date thing, parallel to cm3 and cm3cg
> yes I'll compare the branches, can't promise a full merge though)
>
>
> ----------------------------------------
> > Date: Thu, 22 Jul 2010 09:19:12 +0200
> > From: wagner at elegosoft.com
> > To: jay.krell at cornell.edu
> > CC: m3commit at elegosoft.com
> > Subject: Re: [M3commit] CVS Update: cm3
> >
> > I can live with a cm3-local.cfg file which gets included when it exists,
> > contains local overrides and is never touched by he installation.
> >
> > We have to be careful to backup existing configs and informing the
> > users before the installation though.
> >
> > I think upgrade did backups of the config, too.
> >
> > Can you check the scripts in the release branch if anything needs
> > to be merged back? I won't be able to do that today.
> >
> > Olaf
> >
> > Quoting Jay K :
> >
> > > I thought it was too, before I looked at it.
> > > Could be that head and release are very divergent.
> > > I didn't look at release, oops.
> > > I probably will "soon".
> > >
> > >
> > > The config files are "partly part of the compiler", and partly
> > > system-specific.
> > > So they might have to be updated with the compiler.
> > >
> > >
> > > Perhaps we can have cm3.cfg.user or cm3.cfg.local, that we never edit,
> > > and maybe we should constrain it, like very line
> > > must have an equals sign, and if it has a percent, percent must
> > > follow equals.
> > >
> > >
> > > e.g.
> > > m3back_flags = "-custom"
> > > SYSTEM_CC = "custom" % comment blah blah
> > > SYSTEM_LIBS{"LIBC"} = "custom"
> > > SYSTEM_LIBORDER += [custom]
> > >
> > > But that might not be flexible enough.
> > >
> > > Another option is that cm3.cfg file do like:
> > > if exists("cm3.cfg.user.first")
> > > include("cm3.cfg.user.first")
> > > end
> > > ...
> > > if exists("cm3.cfg.user.last")
> > >
> > > include("cm3.cfg.user.last
> > >
> > > end
> > >
> > >
> > >
> > > which is infinitely flexible and not well defined.
> > > Even limiting to assignments is not well defined.
> > > You know, there's an interface somewhere in there, but it isn't
> > > well specified. My fault.
> > > I let things evolve and get discovered gradually, sometimes hard
> > > to know ahead of time what the result will be.
> > >
> > >
> > > I've made changes since 5.8.6 released such that config files from
> > > prior to 5.8.6 will not suffice.
> > > I can undo that, but there is an important policy and architecture question.
> > >
> > >
> > > - Jay
> > >
> > > ----------------------------------------
> > >> Date: Thu, 22 Jul 2010 08:53:23 +0200
> > >> From: wagner at elegosoft.com
> > >> To: m3commit at elegosoft.com
> > >> Subject: Re: [M3commit] CVS Update: cm3
> > >>
> > >> Quoting Jay K :
> > >>
> > >> > I've manually updated the config files on SPARC32_LINUX Hudson node
> > >> > to try to fix this.
> > >>
> > >> That should be done by the upgrade script (and it already was AFAIR).
> > >> I wouldn't like the script shipping a newly built compiler to
> > >> unconditionally overwrite all my local configuration.
> > >>
> > >> Olaf
> > >>
> > >> >
> > >> > - Jay
> > >> >
> > >> > ----------------------------------------
> > >> >> Date: Wed, 21 Jul 2010 17:53:14 +0000
> > >> >> To: m3commit at elegosoft.com
> > >> >> From: jkrell at elego.de
> > >> >> Subject: [M3commit] CVS Update: cm3
> > >> >>
> > >> >> CVSROOT: /usr/cvs
> > >> >> Changes by: jkrell at birch. 10/07/21 17:53:14
> > >> >>
> > >> >> Modified files:
> > >> >> cm3/scripts/: install-cm3-compiler.sh
> > >> >>
> > >> >> Log message:
> > >> >> upgrade config files when upgrading compiler
> > >> >>
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Olaf Wagner -- elego Software Solutions GmbH
> > >> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
> > >> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
> > >> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
> > >> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
> > >>
> > >
> >
> >
> >
> > --
> > Olaf Wagner -- elego Software Solutions GmbH
> > Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
> > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
> > http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
> > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
> >
>

More information about the M3devel mailing list