[M3devel] cvsup

[Tony Hosking]

I thought I did a year or so back.  But not certain.

On 16 Mar 2010, at 12:10, Jay K wrote:

> I could have sworn I had not seen hanging with sbrk or map_shared, but I can't get that now.
> I'll dig a bit more..maybe not today.
> To be clear, this isn't even the usual fork+exec. This is fork + do work + exit.
> We can probably fix it by having it create a Modula-3 thread.
>    The first fork (daemonize) is probably fine. It is the per-client one that I think is a problem.
> I'd like to find a theory as to why it ever worked, maybe see it work, and then fix it differently.
>   Maybe sbrk + user threads?
>  
> (Can anyone claim to have seen cvsup server work with kernel threads?)
>  
>  - Jay
>  
> From: hosking at cs.purdue.edu
> Date: Tue, 16 Mar 2010 11:43:52 -0400
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] cvsup
> 
> mmap is strongly preferred over sbrk.
> 
> I'd need to understand the control flow here in and across the processes, but yes, generally, mixing pthreads with fork may require significant care.
> 
> On 16 Mar 2010, at 11:33, Jay K wrote:
> 
> Daniel thank you for the bug report.
> Thank you for suggesting strace. I used strace
> and compared working vs. non-working.
> And started added RTIO everywhere.
> You can also use cvsup -f to slightly simplify -- one fork instead of two.
> gdb set follow mode didn't seem to help.
>  
>  
> I almost have it nailed down.
>  
>  
> in CVSup, FSServer.m3, this code:
>  
> FINALLY
>     IF isChild THEN
>         SigHandler.ShutDown(); <== 
>     ELSE
>         SigHandler.Unblock();
>     END;
> END;
> 
>  
> which runs fairly early, never returns in the child.
>  
>  
> It ends up here:
>  
>  
> PROCEDURE ChangeState(d: Dispatcher; state: State) =
>   (* Ask the dispatcher thread to change to a new state, and wait until
>      it has made the change. *)
>   BEGIN
>     LOCK d.mu DO
>       d.desiredState := state;
>       IF d.state # d.desiredState THEN
>         IF d.state = State.Running THEN
>           (* Send dummy signal 0 to wake up the dispatcher. *)
>           Catch(0);
>         ELSE
>           Thread.Signal(d.changeState);
>         END;
>         WHILE d.state # d.desiredState DO
>           Thread.Wait(d.mu, d.stateChanged);   <== this never returns
>         END;
>       END;
>     END;
>   END ChangeState;
>  
>  
> It's a bit wierd to be mixing fork() and Modula-3 Thread?
> Or maybe it is ok?
>  
>  
> See, they are asking another process, from the fork() point of view, to change the state.
>   It does so, but the write is private.
>  
>  
> Remember...Olaf changed from sbrk to mmap(anon|private) in RTOS.GetMemory()?
>   sbrk is maybe shared? mmap(anon|private) is not.
>  
> Right now I have
> #ifndef apple
> sbrk
> #else
> mmap(anon|shared)
> #endif
>  
>  
> and it gets further.
> Hit an assertion failure in pthread.
> I'll try again.
> Cleanup all my RTIO.
>  
>  
> Maybe this notion of using fork() is just not supportable?
>  
>  
> In either case...you could paint it as an m3core problem, but
> ?it won't affect much code?.
>  
>  
>  - Jay
>  
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Subject: cvsup
> Date: Tue, 16 Mar 2010 12:32:49 +0000
> 
> I can reproduce the cvsup problem.
> An important point to consider is that cvsupd forks a server?
>  Maybe that messes up initialization?
>  I'll dig further.
>  
>  
> I think these steps are complete.
> I'll test them on a second clean system.
> You don't need any real data.
>  
>  
> jay at xlin2:~$ cat cvsupd.cfg
> *default tag=.
> *default host=xlin2.
> *default prefix=/home/jay
> *default base=/home/jay/var/db
> *default release=cvs delete use-rel-suffix compress
>  
> src-all
>  
>  
> mkdir -p $HOME/var/db
> mkdir -p $HOME/data/cvsupd
> pkill cvsupd # cleanup any previous
>  
>  
> You don't really need any data.
> cvsup/cvsupd will issue an error in the "working" case, and
> hang in the non-working case.
>  
> 
> 
> 
> run the server:
>  
>  
> jay at xlin2:~$ /cm3/bin/cvsupd -b $HOME/data/cvsupd
> 2010.03.16 05:24:25 PDT [22555]: CVSup server started
> 2010.03.16 05:24:25 PDT [22555]: Software version: 2010-03-05 10:55:15
> 2010.03.16 05:24:25 PDT [22555]: Protocol version: 17.0
> 2010.03.16 05:24:25 PDT [22555]: Ready to service requests
> 
> 
> run the client:
>  
>  /cm3/bin/cvsup -g -L 2 $HOME/cvsupd.cfg
>  
>  
> Parsing supfile "/home/jay/cvsupd.cfg"
> Connecting to xlin2.
> Connected to xlin2.
> Server software version: 2010-03-05
> Negotiating file attribute support
> Exchanging collection information
> Server message: Unknown collection "src-all"
> Establishing multiplexed-mode data connection
> Running
> Skipping collection src-all/cvs
> Shutting down connection to server
> Finished successfully
>  
> it is able to talk to the server, and then fails.
> Ok -- at least it talked to the server.
>  
> The server then exits too.
> I think that is correct (read the usage on the -C option).
>  
>  
> Then try with -C.
> The bug says -C 2. Let's use -C 1.
> They behave the same for our purposes.
>  
>  
> Just add -C 1 to the above server.
> Run the same client.
> The client hangs -- it never connects to the server.
>  
>  
> I reproduced this on LINUXLIBC6 and PPC_LINUX.
> Maybe more soon.
> I'm just rebuilding first.
>  
>  
>  - Jay
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100316/da4b759b/attachment-0002.html>