[M3devel] making cvs checkout/update more robust in tinderbox (and maybe Hudson)

[Olaf Wagner]

Quoting Jay K <jay.krell at cornell.edu>:

> CVS checkout and update of an entire tree seem to be extremely unreliable.
> It takes a while and often times out.

It should not take longer than about 5 minutes, and considering
the number of files and amount of data to be transferred, that's
quite acceptable.

> I don't have time to do this right now, I think the cvs   
> update/checkout should go like this:
>
> if directory doesn't exist:
>   checkout with -l
>
> if directory does exist (it always will now actually, this if can   
> go, except to error if it doesn't exist)
>   cd into it
>   upd -l various hardcoded directories, m3-sys scripts m3-ui m3-libs www doc
>     This list need not be complete and it may contain directories   
> that don't exist.
>   for each directory cvs -z3 upd -dAP that directory
>      notice that after the first one, the directory list will be correct
>   Possibly do it like m3-sys m3-sys/m3cc m3-sys/m3gdb m3-libs   
> m3-libs/m3core m3-libs/libm3
>     to breakdown some of the larger ones.
>   and then one cvs -z3 upd -dAP at the root
>   check return code of just that line one, if failed, wait five   
> minutes try again, if failed, fail
>
> I plan to implement this and I'd /prefer/ to use Python.
>
> If nobody else wants it, I'll turn it off by default.
>
> Even just leaving the workspace aorund doesn't seem to solve the problem.

I don't think it's a good idea.
If you really want to speed it up and make it more robust, replicate the
CM3 repo locally and do checkouts via the file system (much faster
without ssh). You may run cvsup in a loop to get all changes quickly.

If we really run out of resources on birch, we may need to set up
a public r/o copy of the repository for cvs access.

> I'd also be curious about -z1 vs. -z3.
> I usually use -z3. That's what some documentation recommended.

I think it may rather make performance worse as compression needs
much CPU on birch, which may not be available under high system loads.

Please remember that birch is used for other purposes by Elego than
only to serve the CM3 repository.

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194