[M3devel] cvsup

Tue Mar 16 17:38:00 CET 2010

User threads is apparently sufficient to let it work -- no change to sbrk vs. mmap.

You end up with a wierd mix of shared and not shared data, right?

I'll try to make it use pthreads and Modula-3 threads and not fork().

 - Jay

From: jay.krell at cornell.edu
To: hosking at cs.purdue.edu
Date: Tue, 16 Mar 2010 16:10:15 +0000
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] cvsup

I could have sworn I had not seen hanging with sbrk or map_shared, but I can't get that now.
I'll dig a bit more..maybe not today.
To be clear, this isn't even the usual fork+exec. This is fork + do work + exit.
We can probably fix it by having it create a Modula-3 thread.
   The first fork (daemonize) is probably fine. It is the per-client one that I think is a problem.
I'd like to find a theory as to why it ever worked, maybe see it work, and then fix it differently.
  Maybe sbrk + user threads?

(Can anyone claim to have seen cvsup server work with kernel threads?)

 - Jay

From: hosking at cs.purdue.edu
Date: Tue, 16 Mar 2010 11:43:52 -0400
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] cvsup

mmap is strongly preferred over sbrk. 

I'd need to understand the control flow here in and across the processes, but yes, generally, mixing pthreads with fork may require significant care.

On 16 Mar 2010, at 11:33, Jay K wrote:

Daniel thank you for the bug report.
Thank you for suggesting strace. I used strace
and compared working vs. non-working.
And started added RTIO everywhere.
You can also use cvsup -f to slightly simplify -- one fork instead of two.
gdb set follow mode didn't seem to help.

I almost have it nailed down.

in CVSup, FSServer.m3, this code:

FINALLY
    IF isChild THEN
        SigHandler.ShutDown(); <== 
    ELSE
        SigHandler.Unblock();
    END;
END;

which runs fairly early, never returns in the child.

It ends up here:

PROCEDURE ChangeState(d: Dispatcher; state: State) =
  (* Ask the dispatcher thread to change to a new state, and wait until
     it has made the change. *)
  BEGIN
    LOCK d.mu DO
      d.desiredState := state;
      IF d.state # d.desiredState THEN
        IF d.state = State.Running THEN
          (* Send dummy signal 0 to wake up the dispatcher. *)
          Catch(0);
        ELSE
          Thread.Signal(d.changeState);
        END;
        WHILE d.state # d.desiredState DO
          Thread.Wait(d.mu, d.stateChanged);   <== this never returns
        END;
      END;
    END;
  END ChangeState;

It's a bit wierd to be mixing fork() and Modula-3 Thread?
Or maybe it is ok?

See, they are asking another process, from the fork() point of view, to change the state.
  It does so, but the write is private.

Remember...Olaf changed from sbrk to mmap(anon|private) in RTOS.GetMemory()?
  sbrk is maybe shared? mmap(anon|private) is not.

Right now I have
#ifndef apple
sbrk
#else
mmap(anon|shared)
#endif

and it gets further.
Hit an assertion failure in pthread.
I'll try again.
Cleanup all my RTIO.

Maybe this notion of using fork() is just not supportable?

In either case...you could paint it as an m3core problem, but
?it won't affect much code?.

 - Jay

From: jay.krell at cornell.edu
To: m3devel at elegosoft.com
Subject: cvsup
Date: Tue, 16 Mar 2010 12:32:49 +0000

I can reproduce the cvsup problem.
An important point to consider is that cvsupd forks a server?
 Maybe that messes up initialization?
 I'll dig further.

I think these steps are complete.
I'll test them on a second clean system.
You don't need any real data.

jay at xlin2:~$ cat cvsupd.cfg
*default tag=.
*default host=xlin2.
*default prefix=/home/jay
*default base=/home/jay/var/db
*default release=cvs delete use-rel-suffix compress

src-all

mkdir -p $HOME/var/db
mkdir -p $HOME/data/cvsupd
pkill cvsupd # cleanup any previous

You don't really need any data.
cvsup/cvsupd will issue an error in the "working" case, and
hang in the non-working case.

run the server:

jay at xlin2:~$ /cm3/bin/cvsupd -b $HOME/data/cvsupd
2010.03.16 05:24:25 PDT [22555]: CVSup server started
2010.03.16 05:24:25 PDT [22555]: Software version: 2010-03-05 10:55:15
2010.03.16 05:24:25 PDT [22555]: Protocol version: 17.0
2010.03.16 05:24:25 PDT [22555]: Ready to service requests

run the client:

 /cm3/bin/cvsup -g -L 2 $HOME/cvsupd.cfg

Parsing supfile "/home/jay/cvsupd.cfg"
Connecting to xlin2.
Connected to xlin2.
Server software version: 2010-03-05
Negotiating file attribute support
Exchanging collection information
Server message: Unknown collection "src-all"
Establishing multiplexed-mode data connection
Running
Skipping collection src-all/cvs
Shutting down connection to server
Finished successfully

it is able to talk to the server, and then fails.
Ok -- at least it talked to the server.

The server then exits too.
I think that is correct (read the usage on the -C option).

Then try with -C.
The bug says -C 2. Let's use -C 1.
They behave the same for our purposes.

Just add -C 1 to the above server.
Run the same client.
The client hangs -- it never connects to the server.

I reproduced this on LINUXLIBC6 and PPC_LINUX.
Maybe more soon.
I'm just rebuilding first.

 - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100316/2e31dd90/attachment-0002.html>