[M3devel] Some M3 maintenance questions

Mon Mar 2 00:26:19 CET 2015

On 02/27/2015 03:46 PM, Olaf Wagner wrote:
> On Fri, 27 Feb 2015 12:55:26 -0600
> "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:
>
>> I have quite a list of little (and some not so little) maintenance things to
>> do to the Modula-3 distribution.  I am learning to use git and github, but
>> some involve other things than just the repository.
>
> I'm just in the last stages of migrating one of our customer projects
> from Bazaar to Git, including a full-featured automated testing and
> deployment system. We've already invested more than 400 hours for
> that. Automation of building, testing and deploying software needs
> a lot of maintenance and work, which nobody has done yet for the
> M3 Github transition. I put a plan of things that would need to be
> done into the old Trac Wiki, but I doubt that more than 10% of them
> have been achieved.
>

I don't seem to have access to cm3-bugs.elegosoft.com, to update trac.
I have asked michael? to give it to me.

>> 1) Olaf, are you intending to continue hosting the web pages?
>
> Elego can do still do that if no one else volunteers. I cannot
> guarantee more that a couple of minutes administration time per
> week though. We're rather short of ressources currently.
>
>> 2) How can I update these?  There appear to be copies of some/many of them
>>      in the cm3 repository, but these are not served from there as web pages,
>>      AFAIK,
>
> There used to be packages names www and doc, and some ship scripts,
> which were used to transfer the contents to our web server. I assume
> they're still in the Github repo in the same form, so you could use
> them. You only need ssh access for birch.elegosoft.com, if Mike
> hasn't moved that to another VM. I just tried and can login there.
> There is an account
> rodney:x:6011:6011:Rodney M. Bates:/home/rodney:/bin/bash
>
> You have quite a few keys installed there:
>
> % sudo ls /home/rodney/.ssh/
> allegheny.id_dsa.pub  authorized_keys.1  authorized_keys.3   corliss.id_dsa.pub  known_hosts           selkirk.id_dsa.pub
> authorized_keys       authorized_keys2   authorized_keys.3~  garratt.id_dsa.pub  rbates.id_dsa.pub     yellowstone.id_dsa.pub
> authorized_keys.0     authorized_keys.2  authorized_keys.4   id_dsa.pub          runnymede.id_dsa.pub
>
> If you still have one of them, you have ssh access. If not, Mike
> (manderson at elegosoft.com) will store another for you.

I can log in to birch.  Poking around, I find that
birch.elego.de, modula3.elegosoft.com, hudson.modula3.com, and www.opencm3.net
are the same machine.  I find I also have a password on this machine as well.  Is
this the way you want it?  I had to supply it to do a dry run of ship-cm3-www.  It
is doing rsync birch-to-birch, so I suppose I can generate a new key pair on birch
to allow me to access the same machine would obviate needing this password.

>
> The shipping scripts can (could?) be found in the packages:
>
> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/www/ship-cm3-www
> ./work/cm3-ws/birch.elegosoft.com-2012-04-11-01-34-27/cm3/doc/ship-cm3-www-doc
>

I also found copies of these is a number of places, most significanly,

/home/m3/work/cm3-ws/birch.elegosoft.com-2012-08-27-02-27-31/cm3/...

The scripts are the same, but presumably other stuff is more up-to-date here.

There are a _lot_ of differences in this directory that are not shipped to the web
directory.  I don't feel like shipping any of it without careful looking, to be
sure it is what we want on the web.

> The machine still is in use:
>
> elegos-MacBook-Pro:tools.git wagner$ host www.opencm3.net
> www.opencm3.net is an alias for birch.elego.de.
> birch.elego.de has address 46.4.219.187
>
> (elego.de and elegosoft.com are mostly equivalent)
>
>> 3) I have never understood the automated build system on various platforms.
>>      What is its status?  Can run these builds manually, or will they run
>>      automatically if I put source updates in the right place?  What targets
>>      do we have?  (I have LINUXLIBC6 and AMD64_LINUX myself.)
>
> I'm afraid that the Hudson installation on our server is out of order
> for quiet a long time. I don't know of any adaptation even for the
> source checkout from Github.
>
> I think some other public server should be used now for builds and
> tests. The old scripts could be adapted, but most people prefer to
> do these things in their own way. I used shell in 2010, but many
> scripts have been rewritten in python after that.
>

Does anybody have a server to suggest here?  I think a couple of my own machines
would have enough storage, RAM, CPU, and possibly network capacity for the builds
themselves if run only in overnight hours.  But I definitely do not have the
networking capacity for the web pages.  This would be for on AMD64_LINUX, and maybe
LINUXLIBC6.

>> 4) I have also never understood how to run all the many test cases that are
>>      in the repository.  Has it been done at all recently?  Are there scripts
>>      to do it?  Can I add cases to them?  Are there various targets I can get
>>      them to run on?
>
> I haven't done that recently, and I don't really know if anybody
> else has.
> I would really like to do it and set up a working CI system for
> M3 again -- I still think this is the best programming language
> for large systems I have encountered so far -- but nobody has ever
> paid Elego one Euro for any M3 work, so it's just not business-
> compatible ;-) I've got to care for my company, especially since
> we've had some hard times during the last two years.
>

We all greatly appreciate all the support you have given to Modula3.
I hope at least its benefits have helped you in other, revenue-producing
projects.

> I think the shell and python scripts could be adapted to set up
> a working automated build and test system with some weeks of
> effort. It's nothing that can be done by changing a few lines
> I'm afraid.
>
> I just tried to access the Hudson CI system, and it's still
> working, though nothing has been running there for some time.
> You'll find it at http://hudson.modula3.com:8080. Some
> build servers seem to be still online: master (Linux),
> luthien (FreeBSD 8), and the Solaris buildfarm (opencsw).
> The last successful builds seem to be 1 year and 4 months ago,
> for example
> http://hudson.modula3.com:8080/job/cm3-current-test-all-pkgs-AMD64_LINUX/
>
> You could transfer these jobs to another Hudson or Jenkins system
> (the config.xml files should suffice for that), adapt the source
> checkout to the Github repository and try to get some builds
> running. If you really plan to do that, I offer to help with
> all the advice and knowledge I have by answering your questions.
>

With some help, I could take this on.  I have looked briefly at hudson/jenkins.
It looks like there is enough info there to get things going.

> I hope this hasn't been too discouraging,

The most daunting part for me is mediocre network access.  My latency has recently
improved enough so that command-line character echo has gone from just about
unusable to marginally usable.

>
> Olaf
>

-- 
Rodney Bates
rodney.m.bates at acm.org