<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Verdana
}
--></style>
</head>
<body class='hmmessage'>
I was going to ask:<BR>
Do you mean pthreads or Modula-3 threads?<BR>
How you think it is or how you think it should be?<BR>
<BR>
<BR>
Modula-3 user threads are inherited.<BR>
That is what this function would say.<BR>
At least they are currently.<BR>
<BR>
<BR>
However, as you assert, I've also wondered many times to myself the past few days:<BR>
Should Modula-3 userthreads and pthreads act the same way? Always? Optionally?<BR>
<BR>
<BR>
What stopped me was I couldn't convince myself it is trivial to implement.<BR>
I was thinking, like, you'd associate a pid with each thread. If getpid doesn't match thread.pid, the thread is from before a fork. I then worried that you'd build up all the data about threads "forever", across multiple forks. However that is probably false.<BR>
<BR>
<BR>
As well, if you provide RTProcess.Fork() as the replacement for fork(), it affords more power or at least ease of implementation.<BR>
<BR>
<BR>
I guess where you end is like:<BR>
<BR>
<BR>
RTProcess.Fork()<BR>
does nothing on Windows <BR>
in fact is <* EXTERNAL *> and fails to link most likely <BR>
You could have the convention of returning ENOSYS but I say no.<BR>
Or it might be a NIL pointer, following the convention you know about, have used several times. <BR>
just calls fork() on pthreads <BR>
on user threads, described later <BR>
<BR>
<BR>
RTProcess.AtFork() (maybe RegisterAtFork or RegisterForkCallbacks?)<BR>
does nothing on Windows (but exists and links, usable by portable code) <BR>
might be NIL <BR>
on pthreads just calls pthread_atfork <BR>
on user threads, described later <BR>
<BR>
<BR>
And then RTProcess.Fork(), RTProcess.AtFork on user threads work together to provide basically the same meaning as fork() + pthread_fork() has on pthreads.<BR>
Easy to imagine how to code it actually. AtFork maintains an array of function pointers. Fork() calls them and fork in the proscribed order. The array is inherited across processes (like pthread_atfork()).<BR>
Only the calling thread survives, like fork().<BR>
<BR>
<BR>
I didn't realize it at first, but killing of all other threads is pretty easy.<BR>
You just reinitialize like I did for pthreads.<BR>
Then the next time the timer expires, it doesn't find them to be scheduled.<BR>
<BR>
<BR>
I also strongly suspect that in libm3 where fork+exec is called, some of what it does -- in particular turning off the timer -- would be moved to RTProcess.Fork().<BR>
Or perhaps that is in ThreadPosix.m3's AtForkPrepare.<BR>
<BR>
<BR>
Either way, the general idea is that fork is easier to use, be it fork + exec, or even fork + do work. You replace it with RTProcess.Fork().<BR>
<BR>
<BR>
As well however, fork + exec on pthreads continues to work.<BR>
You don't have to call RTProcess.Fork().<BR>
However RTProcess.Fork() is needed to provide matching semantics in user threads if you don't exec after fork. Geez this is a mouthful. I think it is somewhat important to dwell on -- what semantics change, which external libraries just work and don't have to be wrapped.<BR>
<BR>
if you are doing fork + exec, you can keep doing the same, I think<BR>
if you don't care about user threads, and want them all to be inherited, you can keep doing the same, I think<BR>
<BR>
<BR>
No working code should have an altered meaning.<BR>
cvsupd on pthreads doesn't count as working code, of course.<BR>
In fact no fork + do work code on pthreads probably worked.<BR>
<BR>
<BR>
My assertion that "we should use pthread_atfork and be good pthread citizens" remains true and respected by these changes.<BR>
<BR>
<BR>
I'll code this up over the next few days/week.<BR>
<BR>
<BR>
I'm not sure anything ends up in Unix, could all end up in RTProcessC.c or such.<BR>
<BR>
<BR>
If RTProcess.i3 isn't public, it will be.<BR>
<BR>
<BR>
Or we can introduce a new interface.<BR>
<BR>
<BR>
This all sounds about right?<BR>
<BR>
<BR>
Notice that, like my second set of diffs, existing pthreads fork + exec paths will be "significantly" affected -- taking all the locks in the prepare call. It should work.<BR>
<BR>
<BR>
- Jay<BR><BR><BR>
<HR id=stopSpelling>
Subject: Re: [M3devel] cvsup fix refinements<BR>From: hosking@cs.purdue.edu<BR>Date: Fri, 19 Mar 2010 16:17:34 -0400<BR>CC: m3devel@elegosoft.com<BR>To: jay.krell@cornell.edu<BR><BR><BASE>I meant for both user and system threads.<BR><BR>
<DIV>
<DIV>On 19 Mar 2010, at 15:56, Jay K wrote:</DIV><BR class=ecxApple-interchange-newline>
<BLOCKQUOTE><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: medium Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; WORD-SPACING: 0px" class=ecxApple-style-span>
<DIV style="FONT-FAMILY: Verdana; FONT-SIZE: 10pt" class=ecxhmmessage>Agreed. Details later.<BR><BR>
<HR>
From:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</A><BR>Date: Fri, 19 Mar 2010 14:50:01 -0400<BR>To:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR>CC:<SPAN class=ecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>Subject: Re: [M3devel] cvsup fix refinements<BR><BR>
<DIV>The way fork is defined on modern platforms I don't think the child should ever inherit all the threads.</DIV><BR>
<DIV>
<DIV>On 19 Mar 2010, at 14:47, Jay K wrote:</DIV><BR class=ecxecxApple-interchange-newline>
<BLOCKQUOTE><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: medium Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; WORD-SPACING: 0px" class=ecxecxApple-style-span>
<DIV style="FONT-FAMILY: Verdana; FONT-SIZE: 10pt" class=ecxecxhmmessage>How about DoesForkExist, DoesForkInheritAllThreads?<BR><BR><BR>
<HR>
From:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR>To:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</A><BR>Date: Fri, 19 Mar 2010 18:15:34 +0000<BR>CC:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>Subject: Re: [M3devel] cvsup fix refinements<BR><BR>RTProcess is fine.<BR>Unix is a little wierd because I don't think it should do anything if using userthreads.<BR>Unix implies a fairly thin wrapper?<BR><BR> - Jay<BR><BR><BR>
<HR id=ecxecxecxstopSpelling>
Subject: Re: [M3devel] cvsup fix refinements<BR>From:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</A><BR>Date: Fri, 19 Mar 2010 14:13:32 -0400<BR>CC:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>To:<SPAN class=ecxecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR><BR>Why not put it into RTProcess?
<DIV>Or into Unix?<BR>
<DIV>
<DIV>I want to avoid confusion between Thread.Fork (fork a thread) and process fork.</DIV>
<DIV><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span>
<DIV style="WORD-WRAP: break-word"><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: 12px Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; COLOR: rgb(0,0,0); WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span>
<DIV><SPAN style="FONT-SIZE: medium" class=ecxecxecxecxApple-style-span><FONT class=ecxecxecxecxApple-style-span color=#0000ff face="'Gill Sans'"><BR></FONT></SPAN></DIV></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></SPAN></DIV></SPAN></SPAN></DIV>
<DIV>
<DIV>On 19 Mar 2010, at 14:08, Jay K wrote:</DIV><BR class=ecxecxecxecxApple-interchange-newline>
<BLOCKQUOTE><SPAN style="TEXT-TRANSFORM: none; TEXT-INDENT: 0px; BORDER-COLLAPSE: separate; FONT: medium Helvetica; WHITE-SPACE: normal; LETTER-SPACING: normal; WORD-SPACING: 0px" class=ecxecxecxecxApple-style-span>
<DIV style="FONT-FAMILY: Verdana; FONT-SIZE: 10pt" class=ecxecxecxecxhmmessage> > could go in any of ThreadF, RTOS, RTProcess?<BR><BR><BR>Or RTThread. Or almost anywhere.<BR><BR><BR>Some typos in the diffs:<BR> in a comment, ThreadF__ should be RTOS__<BR> function PThreadForkHandler named inconsistently,<BR> should be PThreadAtForkHandler<BR><BR><BR>Again, "PThread" appears in these names to indicate<BR>their meaning is not portable. Their existance is, but not their meaning.<BR>We have no way of saying "only call this function for pthreads",<BR>instead as I understand, we provide a portable interface with<BR>drastically varying semantics, such as "do something" vs. "do nothing".<BR><BR><BR>Given that Init in ThreadPThread.m3 is private, we could probably<BR>take the stackbase parameter from the caller instead.<BR><BR><BR>The call to AtFork should probably follow InitWithStackbase,<BR>in case it fails under low memory, the Die/assert more likely to "work".<BR><BR><BR>I'm not completely happy making RTOS public.<BR> So maybe ThreadF?<BR> I had gone to RTOS because SetNextForkWillExec required LockHeap/UnlockHeap.<BR> But for now they don't exist.<BR><BR><BR>Maybe a new interface? ThreadPThread.i3, but is still<BR>present on all platforms, so mostly portable can call it.<BR><BR><BR>ThreadPThread.AtFork?<BR>PThread.AtFork? That seems right.<BR><BR><BR> - Jay<BR><BR><BR>
<HR id=ecxecxecxecxstopSpelling>
From:<SPAN class=ecxecxecxecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR>To:<SPAN class=ecxecxecxecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>Date: Fri, 19 Mar 2010 17:21:13 +0000<BR>Subject: Re: [M3devel] cvsup fix refinements<BR><BR>diffs attached<BR><BR><BR>For now I've removed the Get/SetNextForkWillExec.<BR> I also think truly get/set is right, not inc/dec.<BR> It was confusing enough to interpret and initialize<BR> the value. And it is also not clear what the default<BR> should be. The usual behavor is exec does follow fork.<BR> The safer default if the handlers work ok is to assume<BR> exec will not follow. It is a perf hint or a don't change<BR> how things were hint? Hint to speed up fork+exec or hint<BR> to avoid running the new code that might not work?<BR><BR><BR>(For now I've removed the Get/SetNextForkWillExec.)<BR> They'd have to be implemented three times, and<BR> I had trouble defining them.<BR> Obviously Win32/userthreads just return a<BR> hardcoded true, but it is hard to explain<BR> from the caller of Get's point of view.<BR><BR>Maybe:<BR> SetNextForkWillExec<BR> GetNextForkWillExec<BR> <SPAN class=ecxecxecxecxApple-converted-space> </SPAN><BR>and then:<BR> PThreadAtForkNeeded() = GetNextForkWillExec = FALSE ?<BR><BR>That is,<BR> someone is who is calling fork and possibly exec,<BR> can immediately tell what these functions mean.<BR><BR> <SPAN class=ecxecxecxecxApple-converted-space> </SPAN><BR>But the implementer of an atfork handler has to<BR> do quite a semantic translation I think.<BR> I wrote it backwards the first time. That either<BR> implies it is confusing, or I wasn't thinking.<BR> <SPAN class=ecxecxecxecxApple-converted-space> </SPAN><BR><BR>If it really is a problem to run this stuff when exec<BR>will follow, we should know quickly, as building cm3<BR>is an aggressive user of this code.<BR><BR><BR>This version has worked repeatedly on Darwin.<BR>I didn't test user threads but it is structured such<BR>as to make that obviously work.<BR> The cvsup changes are in pthreaad_atfork handlers,<BR> so no change for userthreads.<BR><BR><BR>I didn't test on others but confidence is quite high now.<BR><BR>description of changes at least where they are hard to read:<BR><BR>m3core:<BR> Init split into Init and InitWithStackBase,<BR> We have to provide the old stack base in the child fork handler<BR> instead of estimating from the address of a local. I don't<BR> know what stack is used, maybe the caller of fork?<BR> i.e. we might have eaten a fair amount of stack and<BR> there could be arbitrary references above the current stack.<BR><BR><BR>WeakRefFromRef split into WeakRefFromRef and StartWeakCleaner<BR><BR>There is a resulting double lock in WeakRefFromRef, every time.<BR>Could instead copy/paste more code around, i.e.:<BR> EVAL Thread.Fork(NEW(Thread.Closure, apply := WeakCleaner));<BR><BR><BR> - Jay<BR><BR><BR><BR><BR><BR><BR>
<HR id=ecxecxecxecxecxstopSpelling>
From:<SPAN class=ecxecxecxecxApple-converted-space> </SPAN><A href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</A><BR>To:<SPAN class=ecxecxecxecxApple-converted-space> </SPAN><A href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</A><BR>Date: Fri, 19 Mar 2010 16:01:03 +0000<BR>Subject: [M3devel] cvsup fix refinements<BR><BR>refinements:<BR><BR><BR>new public functions in m3core:<BR><BR><BR>TYPE PThreadForkHandler = PROCEDURE();<BR><BR><* EXTERNAL RTOS__PThreadAtFork *><BR>PROCEDURE PThreadAtFork(prep, parent, child: PThreadForkHandler): INTEGER;<BR><BR><BR>(That's not a change yet.)<BR><BR><BR>PROCEDURE SetNextForkWillExec(value: BOOLEAN);<BR><BR>PROCEDURE GetNextForkWillExec(): BOOLEAN;<BR><BR> These are only an optimization and should<BR> perhaps be removed? They should be used<BR> under LockHeap/UnlockHeap? which wasn't previously<BR> public. This way, existing fork/exec paths<BR> not affected, though maybe though might as well<BR> ought to be?<BR><BR><BR>could go in any of ThreadF, RTOS, RTProcess?<BR><BR><BR>Should be called under RTOS.LockHeap?<BR> Therefore I put in RTOS.<BR> Also helps that RTOS is exported from *Thread.m3.<BR>RTOS not previously public.<BR><BR><BR>Should Get/Set be Inc/Dec? (therefore just Inc with +1,-1?)<BR> I implemented them that way: true incs, false decs.<BR><BR><BR>Or don't bother with them at all?<BR><BR><BR>Furthermore, in RTCollector, instead of claiming the threads<BR>aren't started, if they were started, restart them.<BR>This makes more sense for the weak cleaner to me, and<BR>seems reasonable for the other two.<BR><BR><BR>Diffs not yet available.<BR><BR><BR> - Jay<BR></DIV></SPAN></BLOCKQUOTE></DIV><BR></DIV></DIV></DIV></SPAN><BR class=ecxecxApple-interchange-newline></BLOCKQUOTE></DIV><BR></DIV></SPAN><BR class=ecxApple-interchange-newline></BLOCKQUOTE></DIV><BR> </body>
</html>