[M3devel] cvsup

Tue Mar 16 20:32:52 CET 2010

fork should still work with pthreads.  Why wouldn't it?
Unless there is something weird about the semaphore used to start and stop threads?
If the child ends up with the same semaphore then that would be a problem.
Perhaps we need to ensure it is replicated it in forked children using pthread_atfork?
You could try things on a target like OSX where threads are stopped/started natively (i.e., no semaphore being used).
Though, on inspection I see that sem_init is called with pshared == false.

Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484

On 16 Mar 2010, at 13:43, Jay K wrote:

> With fork, which data is copy-on-write vs. shared-write?
> mmap has a flag, so it is up to each caller. You end up with a mix.
>  
>  
> Whether or not you get one forked thread or all of them I believe varies.
> Solaris has "fork1()".
> There is no obvious answer.
> It just goes to show how wierd fork() is.
>  
>  
> I'm not sure any longer there is a connection to mmap/sbrk.
> It might be just user threads vs. not.
> Though there again there is a question as to which data is
> shared-write vs. copy-on-write.
>  
>  
> The server should not crash.
> It is written in an optionally save language, after all.
> More so, that is I believe a much more larger more difficult fix to apply.
> Using processes for such separation is deemed overkill, depending
> on whose story you believe.
> I'll have to think about it more.
>  
>  
>  - Jay
>  
> > Date: Tue, 16 Mar 2010 18:07:38 +0100
> > From: wagner at elegosoft.com
> > To: m3devel at elegosoft.com
> > Subject: Re: [M3devel] cvsup
> > 
> > Quoting Jay K <jay.krell at cornell.edu>:
> > 
> > >
> > > User threads is apparently sufficient to let it work -- no change to 
> > > sbrk vs. mmap.
> > >
> > > You end up with a wierd mix of shared and not shared data, right?
> > >
> > > I'll try to make it use pthreads and Modula-3 threads and not fork().
> > 
> > I cannot really follow you here. Why should fork not work for
> > threaded processes? The kernel should duplicate all threads and all
> > memory regions, right?
> > 
> > I don't seem to get the connection you make to sbrk vs. mmap.
> > Can you explain?
> > 
> > Using a thread for the process fork will probably not work contrary
> > to my former statement that it might, as each client process is
> > expected to have multiple threads.
> > 
> > And the address spaces of the server processes should be separate
> > for security and robustness reasons, as they are each connected to
> > different clients, and one client's server crash must not impact the
> > other sessions.
> > 
> > Or am I misunderstanding what you intend to do?
> > 
> > Olaf
> > 
> > > - Jay
> > >
> > >
> > >
> > >
> > > From: jay.krell at cornell.edu
> > > To: hosking at cs.purdue.edu
> > > Date: Tue, 16 Mar 2010 16:10:15 +0000
> > > CC: m3devel at elegosoft.com
> > > Subject: Re: [M3devel] cvsup
> > >
> > >
> > >
> > > I could have sworn I had not seen hanging with sbrk or map_shared, 
> > > but I can't get that now.
> > > I'll dig a bit more..maybe not today.
> > > To be clear, this isn't even the usual fork+exec. This is fork + do 
> > > work + exit.
> > > We can probably fix it by having it create a Modula-3 thread.
> > > The first fork (daemonize) is probably fine. It is the per-client 
> > > one that I think is a problem.
> > > I'd like to find a theory as to why it ever worked, maybe see it 
> > > work, and then fix it differently.
> > > Maybe sbrk + user threads?
> > >
> > > (Can anyone claim to have seen cvsup server work with kernel threads?)
> > >
> > > - Jay
> > >
> > >
> > >
> > > From: hosking at cs.purdue.edu
> > > Date: Tue, 16 Mar 2010 11:43:52 -0400
> > > To: jay.krell at cornell.edu
> > > CC: m3devel at elegosoft.com
> > > Subject: Re: [M3devel] cvsup
> > >
> > > mmap is strongly preferred over sbrk.
> > >
> > >
> > > I'd need to understand the control flow here in and across the 
> > > processes, but yes, generally, mixing pthreads with fork may require 
> > > significant care.
> > >
> > >
> > >
> > > On 16 Mar 2010, at 11:33, Jay K wrote:
> > >
> > > Daniel thank you for the bug report.
> > > Thank you for suggesting strace. I used strace
> > > and compared working vs. non-working.
> > > And started added RTIO everywhere.
> > > You can also use cvsup -f to slightly simplify -- one fork instead of two.
> > > gdb set follow mode didn't seem to help.
> > >
> > >
> > > I almost have it nailed down.
> > >
> > >
> > > in CVSup, FSServer.m3, this code:
> > >
> > > FINALLY
> > > IF isChild THEN
> > > SigHandler.ShutDown(); <==
> > > ELSE
> > > SigHandler.Unblock();
> > > END;
> > > END;
> > >
> > >
> > > which runs fairly early, never returns in the child.
> > >
> > >
> > > It ends up here:
> > >
> > >
> > > PROCEDURE ChangeState(d: Dispatcher; state: State) =
> > > (* Ask the dispatcher thread to change to a new state, and wait until
> > > it has made the change. *)
> > > BEGIN
> > > LOCK d.mu DO
> > > d.desiredState := state;
> > > IF d.state # d.desiredState THEN
> > > IF d.state = State.Running THEN
> > > (* Send dummy signal 0 to wake up the dispatcher. *)
> > > Catch(0);
> > > ELSE
> > > Thread.Signal(d.changeState);
> > > END;
> > > WHILE d.state # d.desiredState DO
> > > Thread.Wait(d.mu, d.stateChanged); <== this never returns
> > > END;
> > > END;
> > > END;
> > > END ChangeState;
> > >
> > >
> > > It's a bit wierd to be mixing fork() and Modula-3 Thread?
> > > Or maybe it is ok?
> > >
> > >
> > > See, they are asking another process, from the fork() point of view, 
> > > to change the state.
> > > It does so, but the write is private.
> > >
> > >
> > > Remember...Olaf changed from sbrk to mmap(anon|private) in RTOS.GetMemory()?
> > > sbrk is maybe shared? mmap(anon|private) is not.
> > >
> > > Right now I have
> > > #ifndef apple
> > > sbrk
> > > #else
> > > mmap(anon|shared)
> > > #endif
> > >
> > >
> > > and it gets further.
> > > Hit an assertion failure in pthread.
> > > I'll try again.
> > > Cleanup all my RTIO.
> > >
> > >
> > > Maybe this notion of using fork() is just not supportable?
> > >
> > >
> > > In either case...you could paint it as an m3core problem, but
> > > ?it won't affect much code?.
> > >
> > >
> > > - Jay
> > >
> > >
> > >
> > > From: jay.krell at cornell.edu
> > > To: m3devel at elegosoft.com
> > > Subject: cvsup
> > > Date: Tue, 16 Mar 2010 12:32:49 +0000
> > >
> > > I can reproduce the cvsup problem.
> > > An important point to consider is that cvsupd forks a server?
> > > Maybe that messes up initialization?
> > > I'll dig further.
> > >
> > >
> > > I think these steps are complete.
> > > I'll test them on a second clean system.
> > > You don't need any real data.
> > >
> > >
> > > jay at xlin2:~$ cat cvsupd.cfg
> > > *default tag=.
> > > *default host=xlin2.
> > > *default prefix=/home/jay
> > > *default base=/home/jay/var/db
> > > *default release=cvs delete use-rel-suffix compress
> > >
> > > src-all
> > >
> > >
> > > mkdir -p $HOME/var/db
> > > mkdir -p $HOME/data/cvsupd
> > > pkill cvsupd # cleanup any previous
> > >
> > >
> > > You don't really need any data.
> > > cvsup/cvsupd will issue an error in the "working" case, and
> > > hang in the non-working case.
> > >
> > >
> > >
> > >
> > > run the server:
> > >
> > >
> > > jay at xlin2:~$ /cm3/bin/cvsupd -b $HOME/data/cvsupd
> > > 2010.03.16 05:24:25 PDT [22555]: CVSup server started
> > > 2010.03.16 05:24:25 PDT [22555]: Software version: 2010-03-05 10:55:15
> > > 2010.03.16 05:24:25 PDT [22555]: Protocol version: 17.0
> > > 2010.03.16 05:24:25 PDT [22555]: Ready to service requests
> > >
> > >
> > > run the client:
> > >
> > > /cm3/bin/cvsup -g -L 2 $HOME/cvsupd.cfg
> > >
> > >
> > > Parsing supfile "/home/jay/cvsupd.cfg"
> > > Connecting to xlin2.
> > > Connected to xlin2.
> > > Server software version: 2010-03-05
> > > Negotiating file attribute support
> > > Exchanging collection information
> > > Server message: Unknown collection "src-all"
> > > Establishing multiplexed-mode data connection
> > > Running
> > > Skipping collection src-all/cvs
> > > Shutting down connection to server
> > > Finished successfully
> > >
> > > it is able to talk to the server, and then fails.
> > > Ok -- at least it talked to the server.
> > >
> > > The server then exits too.
> > > I think that is correct (read the usage on the -C option).
> > >
> > >
> > > Then try with -C.
> > > The bug says -C 2. Let's use -C 1.
> > > They behave the same for our purposes.
> > >
> > >
> > > Just add -C 1 to the above server.
> > > Run the same client.
> > > The client hangs -- it never connects to the server.
> > >
> > >
> > > I reproduced this on LINUXLIBC6 and PPC_LINUX.
> > > Maybe more soon.
> > > I'm just rebuilding first.
> > >
> > >
> > > - Jay
> > >
> > >
> > 
> > 
> > 
> > -- 
> > Olaf Wagner -- elego Software Solutions GmbH
> > Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
> > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
> > http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
> > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
> > 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100316/ead43c26/attachment-0002.html>