[M3devel] changes in CM3 continuous integration setup

[Olaf Wagner]

Quoting Jay K <jay.krell at cornell.edu>:

> I don't really understand all of this.
> Surely CVS can tolerate more than 10 "active" users?
> Or is it really awful?
>   (or, most likely, both)
> What if they all poll much less than once an hour?

It's not just CVS. The machine is running our WWW services, FTP,
CVSup, backup for several hosts and secondary mail services.

> Let's say we have 12 machines.
> Have them each poll 4 times per day: 48 polls.
> So that's just one operation every 30 minutes.
>  One poll, possibly followed by one update.

That would be an individual setup for all jobs regarding polling.
It's of course possible, but more tedious work again.

> Or, didn't you set it up so only one machine polled and all the   
> other builds would follow it, serially?
> If not, that makes sense.
> Have one machine poll every 30 minutes.
> If it finds something, it builds, and the next one builds.
> If it finds nothing, then nothing occurs.
> Surely this would work ok?
> It merely serializes all of our Hudson jobs?
>   Which is, granted, rather unfortunate.
>   Adding machines should allow great parallelism.
>   And CVS is read-mostly, so should handle plenty of load.   
> "Read-mostly" being
>   a property that makes systems easier to scale.

The current setup is suboptimal.
I've changed things in a way that could be done quickly.
But it's not easy to foretell how any change will affect the load
and responsiveness, as it isn't even clear where the limits are; and
other services are continually competing for resources, too.

I've set up continuous logging of system load, top and iotop.
You can get a quick summary with this job:

http://hudson.modula3.com:8080/view/zzz/job/sysload

We'll monitor this for some days and see how our changes affect
the situation.

Complete logs are written to Hudson's ~/log/ directory.

> I assume if I bothered to learn to use cvsup, it would help much?
> We'd have a few mirrors and it'd "fan out well"?
> e.g. 5 cvsup mirrors could each serve 5 cvs clients, 25 for the price of 5?

I think we'd only need one CVS mirror that the continous integration
could use. Unfortunately, our system administrator is on vacance (his wife's
expecting their second baby) right now, and I haven't got any machine
for that readily available.

We'll sort that out, though; don't worry too much. It may need some
weeks, but we'll find an acceptable solution without much limitations.

Olaf
-- 
Olaf Wagner -- elego Software Solutions GmbH
                Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
phone: +49 30 23 45 86 96  mobile: +49 177 2345 869  fax: +49 30 23 45 86 95
    http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194