<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Verdana
}
--></style>
</head>
<body class='hmmessage'>
---<BR>
bad news:<BR>
It doesn't completely work. It works a bunch of times in a row, like 9, then hangs.<BR>
Restart manually. Works again. Around 9 times. Then hangs again.<BR>
That is on Linux/x86 and Solaris/sparc.<BR>
Doesn't work at all on Mac/amd64, just hangs.<BR>
<BR>
---<BR>
sketch:<BR>
m3core uses pthread_atfork to selectively reinitialize<BR>
Mainly to only have one thread.<BR>
<BR>
<BR>
common Thread.PThreadAtFork is provided for others to do the same<BR>
It is deliberately in a portable interface.<BR>
<BR>
<BR>
Thread.ReforkThreadAfterProcessFork<BR>
Is provided for users to restart threads from their child AtFork hander.<BR>
This is used by the allocator/collector.<BR><BR>
<BR>
Thread.ForkProcessAndAllThreads()<BR>
Is used by "lazy" clients who want to restart all their threads<BR>
but didn't keep track of them. The runtime can do it for them.<BR>
<BR>
<BR>
This allows for "fork + do work" folks do call or not call ForkProcessAndAllThreads<BR>
or not, depending on if they need their threads restarted.<BR>
The runtime takes care of its threads either way.<BR>
<BR>
<BR>
---<BR>
What'd I'd written up:<BR>
<BR>
attached works typically 9 times on Linux and Solaris<BR>before server hangs again.<BR><BR>
<BR>
No improvement on Darwin, just hangs.<BR>Can't see much in debuggers for some reason.<BR>
<BR>
<BR>There is extra allowance in the m3core change such<BR> that users of fork + do work (as opposed to fork + exec)<BR> may or may not call ForkAll, depending on if they<BR> feel a need for their own threads to be recreated,<BR> and if they've kept track of how to recreate them,<BR> or just rely on the runtime to know all the threads.<BR>
<BR>
<BR>There are three runtime threads that are sometimes<BR>created in the parent, and if so, recreated in the child.<BR>
background collector, foreground collector, weak ref thread<BR>
<BR>
<BR>
I'll try to poke at it some more.<BR>
<BR>
<BR>I'm not sure what is the best way to suspend all threads.<BR>I tried a few differnt ways.<BR> SuspendOthers<BR> LockHeap<BR> pthread_mutex_lock<BR> various combinations<BR>
<BR> <BR>
It is deliberate that pthread specific code is in common/Thread.i3.<BR>That way code can be portable, at least among the two Posix thread implementations.<BR>
<BR>
<BR> - Jay<BR><BR><BR> <BR>
<HR id=stopSpelling>
From: hosking@cs.purdue.edu<BR>Date: Wed, 17 Mar 2010 14:01:31 -0400<BR>To: jay.krell@cornell.edu<BR>CC: m3devel@elegosoft.com<BR>Subject: Re: [M3devel] fork/cvsup<BR><BR><BASE>Can you sketch the approach you've taken?
<DIV><BR><BR>
<DIV>
<DIV>On 17 Mar 2010, at 11:39, Jay K wrote:</DIV><BR class=ecxApple-interchange-newline>
<BLOCKQUOTE><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: medium Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; WORD-SPACING: 0px" class=ecxApple-style-span>
<DIV style="FONT-FAMILY: Verdana; FONT-SIZE: 10pt" class=ecxhmmessage>I have something working on Solaris now.<BR>More details after testing on Linux and Darwin.<BR> <BR> - Jay<BR> <BR>
<HR id=ecxstopSpelling>
From:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR>To:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</A><BR>Date: Wed, 17 Mar 2010 14:01:15 +0000<BR>CC:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>Subject: Re: [M3devel] fork/cvsup<BR><BR>Exec what?<BR>You'd have to change the code to carefully reach the same place.<BR> <BR> - Jay<BR> <BR>
<HR id=ecxecxstopSpelling>
Subject: Re: [M3devel] fork/cvsup<BR>From:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</A><BR>Date: Wed, 17 Mar 2010 09:28:14 -0400<BR>CC:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>To:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR><BR>
<DIV><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span>
<DIV style="WORD-WRAP: break-word"><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxApple-style-span>
<DIV><SPAN style="FONT-SIZE: medium" class=ecxecxecxApple-style-span><FONT class=ecxecxecxApple-style-span color=#0000ff face="'Gill Sans'">Why not just exec in the child?</FONT></SPAN></DIV></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></DIV></SPAN></SPAN></DIV><BR>
<DIV>
<DIV>On 17 Mar 2010, at 03:47, Jay K wrote:</DIV><BR class=ecxecxecxApple-interchange-newline>
<BLOCKQUOTE><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: medium Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; WORD-SPACING: 0px" class=ecxecxecxApple-style-span>
<DIV style="FONT-FAMILY: Verdana; FONT-SIZE: 10pt" class=ecxecxecxhmmessage><A href="http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/fork.2.html">http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/fork.2.html</A><BR> <BR><FONT face=""></FONT> <BR><FONT face="">There are limits to what you can do in the child process. To be totally safe you should restrict your-<FONT class=ecxecxecxwhiteout color=#ffffff size=1>self yourself</FONT><BR> self to only executing async-signal safe operations until such time as one of the exec functions is<BR> called. All APIs, including global data symbols, in any framework or library should be assumed to be<BR> unsafe after a<SPAN class=ecxecxecxApple-converted-space> </SPAN><B>fork</B>() unless explicitly documented to be safe or async-signal safe. If you need to use<BR> these frameworks in the child process, you must exec. In this situation it is reasonable to exec your-<FONT class=ecxecxecxwhiteout color=#ffffff size=1>self. yourself.</FONT><BR> self.<BR><BR></FONT><FONT face=""></FONT> <BR><FONT face=""><A href="http://www.opengroup.org/onlinepubs/000095399/functions/fork.html">http://www.opengroup.org/onlinepubs/000095399/functions/fork.html</A></FONT><BR><FONT face=""></FONT> <BR><FONT face="">Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the<I><A href="http://www.opengroup.org/onlinepubs/000095399/functions/exec.html">exec</A></I><SPAN class=ecxecxecxApple-converted-space> </SPAN>functions is called.<SPAN class=ecxecxecxApple-converted-space> </SPAN><SUP>[<A>THR</A>]</SUP><SPAN class=ecxecxecxApple-converted-space> </SPAN><IMG border=0 alt="[Option Start]" src="http://www.opengroup.org/onlinepubs/000095399/images/opt-start.gif"><SPAN class=ecxecxecxApple-converted-space> </SPAN> Fork handlers may be established by means of the<SPAN class=ecxecxecxApple-converted-space> </SPAN><A href="http://www.opengroup.org/onlinepubs/000095399/functions/pthread_atfork.html"><I>pthread_atfork</I>()</A><SPAN class=ecxecxecxApple-converted-space> </SPAN>function in order to maintain application invariants across<SPAN class=ecxecxecxApple-converted-space> </SPAN><I>fork</I>() calls.<SPAN class=ecxecxecxApple-converted-space> </SPAN><IMG border=0 alt="[Option End]" src="http://www.opengroup.org/onlinepubs/000095399/images/opt-end.gif"><BR> <BR> <BR>I've run through a few theories so far.<BR>Current thinking is related to what Tony said:<BR> use pthread_atfork:<SPAN class=ecxecxecxApple-converted-space> </SPAN><BR> in prepare, stopworld<SPAN class=ecxecxecxApple-converted-space> </SPAN><BR> in parent, resumeworld<SPAN class=ecxecxecxApple-converted-space> </SPAN><BR> You don't want the child to be mid-gc for example, on another thread. Or mid-anything.<BR> in child, reinitialize -- current thread is the only thread<BR> <BR> <BR>Also in the cvsup code, ShutDown should just call DoShutDown immediately.<BR>I did that, without m3core changes, and it hits an error in the pthread code, signaling a nonexistant thread.<BR>pthread_atfork/child should address that -- child shouldn't retain a record of all the threads in the parent.<BR> <BR> <BR>I don't have a theory as to why user threads work.<BR> <BR> <BR>I experimented with malloc vs. static alloc vs. sbrk vs. mmap(private) vs. mmap(shared).<BR>I was expecting more cases to act like mmap(shared), but none did, only it.<BR> <BR> <BR>I experimented with having mutexes and condition variables be initialize up front instead of on-demand.<BR>Via changing cvsup to lock/unlock or broadcast immediately upon creating them.<BR>On the theory that might let them work across process.<BR>That didn't make a difference.<BR> <BR> <BR> - Jay<BR></FONT></DIV></SPAN></BLOCKQUOTE></DIV><BR></DIV></SPAN><BR class=ecxApple-interchange-newline></BLOCKQUOTE></DIV><BR></DIV> </body>
</html>