<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
</head>
<body class='hmmessage'>
> This change depends on pthread_atfork, for pthreads and user threads.<br><br><br>Clarification: There's no change described there for pthreads, and indeed, getting<br>this behavior with pthreads doesn't depend on using pthread_atfork. That is just<br>how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs. fork()).<br>The Solaris default used to be fork => forkall().<br>The current Solaris default is fork => fork1().<br>There really do exist on Solaris to functions you can call, fork1() and forkall().<br>All other systems and the Posix specification have the "fork1" semantic for fork.<br>Therefore "fork1" is what we should provide unconditionally.<br>If you really want "forkall", then you write:<br>#ifdef __sun<br> forkall()<br>#else<br> #error<br>#endif<br><br><br><br>and/or you go and hack up m3core/src/thread/POSIX, branch off m3core/src/thread/POSIX_FORKALL or such.<br>I don't expect anyone to ever do that.<br><br><br><br>The need for pthread_atfork is sort of different for user threads vs. pthreads, but it is needed either way.<br>For pthreads, there is this problem -- at the time a thread calls fork(), what are other threads doing?<br>Answer: Anything.<br>Follow up question: What locks might they hold?<br>Answer: Any.<br>Follow up: So, if arbitrary locks are held, and you fork, and don't exec, and the child process<br>attempts to use some of the locks that were held in the parent, and only the forking thread survived,<br>then the other threads will never leave their locks, and later attempts to acquire them in the children<br>will deadlock. Therefore, what pthread_atfork provides for, what you are supposed to use it for,<br>is roughly speaking, right before fork(), in the parent, acquire all locks, and then fork(), and then in the<br>parent and children, release all locks. Therefore, you have to come up with a locking order.<br><br><br><br>Things are a little different for user threads.<br>In user threads, the existance of threads is established by virtue of a global array or list of data describing the threads.<br>And a timer to occasionally preempt. When the timer interrupts, the scheduler picks another thread from the global<br>array/list. Previously, the data went unchanged when fork() was called. Therefore all threads survived.<br>What we do now is that in the child "handler" for pthread_atfork, is reinitialize the various data, so as to forget about<br>all but the current thread. Something like that.<br><br><br><br>I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork, then ForkBeforeExec<br>might get away with just fork() and ignore all this.<br>So far we have not drawn this line -- there is just the one fork and it is used prior to exec or do-more-work<br>and does the same thing w/o knowing what is to follow.<br><br><br>The Posix documentation I think describes this all fairly well.<br>Bing for "posix opengroup pthread_atfork".<br>Really. I think it is saying much of the same thing I am saying, but in language that has received<br>much more thought. Be sure to read the "rationale" part.<br><br><br> - Jay<br><br><br><hr id="stopSpelling">From: jay.krell@cornell.edu<br>To: mika@async.caltech.edu<br>CC: m3devel@elegosoft.com<br>Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4<br>Date: Tue, 19 Apr 2011 15:06:38 +0000<br><br>
<meta http-equiv="Content-Type" content="text/html; charset=unicode">
<meta name="Generator" content="Microsoft SafeHTML">
<style>
.ExternalClass .ecxhmmessage P
{padding:0px;}
.ExternalClass body.ecxhmmessage
{font-size:10pt;font-family:Tahoma;}
</style>
Previously: user threads: all threads survived fork()<br>Previously: pthreads: only the thread calling fork() survived fork()<br>Now: user threads and pthreads: only the thread calling fork() survives fork()<br>This must be how pthreads behave, and this makes user threads and pthreads consistent.<br>
This change depends on pthread_atfork, for pthreads and user threads.<br>It only really matters to the rare "fork + do more work" program, such as cvsupd.<br>Most programs either never fork, or exec almost immediately after fork.<br><br><br>pthread_atfork offers a good model.<br>A sort of "distributed" model.<br>You don't have to go and change all the calls to fork().<br>Each module with a need to do something before/after fork, calls the central pthread_atfork,<br>and fork and pthread_atfork cooperate to do what is needed.<br><br><br>No function pointer is needed.<br>Instead move the code to m3core/thread.<br><br><br>If you really must not use -pthread, then you must implement pthread_atfork functionality<br>yourself and have all fork() calls go through your own fork() wrapper that cooperates<br>with your pthread_atfork replacement.<br>There is no free lunch -- there is a downside to this approach, as plain fork() calls<br>are ok and correct if pthread_atfork is used, but now become incorrect.<br>Pick your poison:<br> user thread/pthread inconsistency<br> cvsupd incompatibility with pthreads <br> user threads using pthread_atfork/-pthread <br> fork() calls having to go through a wrapper (ok -- you could miss this and <br> not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). <br><br><br>(Most of this has been explained multiple times, but people only pay attention<br>when they think it affects them. I'm guilty of the same thing.)<br><br><br> - Jay<br><br><br>> To: jay.krell@cornell.edu<br>> Date: Mon, 18 Apr 2011 19:30:58 -0700<br>> From: mika@async.caltech.edu<br>> CC: m3devel@elegosoft.com<br>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4<br>> <br>> <br>> If you or Tony could describe roughly what you think needs to be done<br>> I'd be happy to look into it.<br>> <br>> The basic problem is that a decision that's made/described in<br>> m3core/thread/m3makefile needs to someone find its way to controlling<br>> what C code gets compiled elsewhere in the system. Or maybe there should<br>> be an indirection into the thread library to pick up pthread_atfork (or not).<br>> <br>> But then again you still haven't explained why you made the user threads use<br>> pthread_atfork. I just remove it from my installations, but then again I'm<br>> not trying to run CVSup so I don't know if it breaks that program to do so.<br>> <br>> Mika<br>> <br>> Jay K writes:<br>> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_<br>> >Content-Type: text/plain; charset="iso-8859-1"<br>> >Content-Transfer-Encoding: quoted-printable<br>> ><br>> ><br>> >> The following code from RTProcessC.c ensures that it is neeeded on every =<br>> >Unix except<br>> >> > /* NOTE: Even userthreads now depends<br>> >> > * on availability of pthreads.<br>> >> > * This can be fixed if need be.<br>> >> > */<br>> ><br>> >=20<br>> >Ok=2C "we" should probably go ahead and fix that.<br>> >I'll try to=2C but no promises=2C sorry.<br>> >=20<br>> > - Jay<br>> >=20<br>> >> From: hosking@cs.purdue.edu<br>> >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400<br>> >> To: mika@async.caltech.edu<br>> >> CC: m3devel@elegosoft.com<br>> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib=<br>> >c-2.6-4<br>> >>=20<br>> >> Probably unnecessary=2C given that I think there is another entry point t=<br>> >o forking a process (I forget where) in the thread-specific portion of m3co=<br>> >re. In which case the necessary work might be done there?<br>> >>=20<br>> >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote:<br>> >>=20<br>> >> > Tony Hosking writes:<br>> >> > ...<br>> >> >> pthread_atfork should not be needed under user threads.<br>> >> > ...<br>> >> >=20<br>> >> > The following code from RTProcessC.c ensures that it is neeeded on ever=<br>> >y Unix except<br>> >> > FreeBSD before 6. The comment is from the checked-in source file.<br>> >> >=20<br>> >> > /* NOTE: Even userthreads now depends<br>> >> > * on availability of pthreads.<br>> >> > * This can be fixed if need be.<br>> >> > */<br>> >> >=20<br>> >> > INTEGER<br>> >> > __cdecl<br>> >> > RTProcess__RegisterForkHandlers(<br>> >> > ForkHandler prepare=2C<br>> >> > ForkHandler parent=2C<br>> >> > ForkHandler child)<br>> >> > {<br>> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf.<br>> >> > * VMS lacks pthread_atfork? Would be good to use autoconf.<br>> >> > * Win32 lacks pthread_atfork and fork. OK.<br>> >> > *<br>> >> > * As well=2C for all Posix systems=2C we could implement<br>> >> > * atfork ourselves=2C as long as we provide a fork()<br>> >> > * wrapper that code uses.<br>> >> > */<br>> >> > #if defined(_WIN32) \<br>> >> > || defined(__vms) \<br>> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ )<br>> >> > return 0=3B<br>> >> > #else<br>> >> > while (1)<br>> >> > {<br>> >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B<br>> >> > if (i !=3D EAGAIN)<br>> >> > return i=3B<br>> >> > sleep(0)=3B<br>> >> > }<br>> >> > #endif<br>> >> > }<br>> >>=20<br>> > =<br>> ><br>> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_<br>> >Content-Type: text/html; charset="iso-8859-1"<br>> >Content-Transfer-Encoding: quoted-printable<br>> ><br>> ><html><br>> ><head><br>> ><style><!--<br>> >.hmmessage P<br>> >{<br>> >margin:0px=3B<br>> >padding:0px<br>> >}<br>> >body.hmmessage<br>> >{<br>> >font-size: 10pt=3B<br>> >font-family:Tahoma<br>> >}<br>> >--></style><br>> ></head><br>> ><body class=3D'hmmessage'><br>> >>=3B The following code from RTProcessC.c ensures that it is neeeded on e=<br>> >very Unix except<BR>>=3B >=3B /* NOTE: Even userthreads now depends<BR>=<br>> >>=3B >=3B * on availability of pthreads.<BR>>=3B >=3B * This can be=<br>> > fixed if need be.<BR>>=3B >=3B */<BR><BR><br>> > =3B<BR><br>> >Ok=2C "we" should probably go ahead and fix that.<BR><br>> >I'll try to=2C but no =3Bpromises=2C sorry.<BR><br>> > =3B<BR><br>> > =3B- Jay<BR> =3B<BR><br>> >>=3B From: hosking@cs.purdue.edu<BR>>=3B Date: Mon=2C 18 Apr 2011 18:11=<br>> >:26 -0400<BR>>=3B To: mika@async.caltech.edu<BR>>=3B CC: m3devel@elegos=<br>> >oft.com<BR>>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne=<br>> >l 2.6.23=2C glibc-2.6-4<BR>>=3B <BR>>=3B Probably unnecessary=2C given =<br>> >that I think there is another entry point to forking a process (I forget wh=<br>> >ere) in the thread-specific portion of m3core. In which case the necessary =<br>> >work might be done there?<BR>>=3B <BR>>=3B On Apr 18=2C 2011=2C at 2:45=<br>> > PM=2C Mika Nystrom wrote:<BR>>=3B <BR>>=3B >=3B Tony Hosking writes:=<br>> ><BR>>=3B >=3B ...<BR>>=3B >=3B>=3B pthread_atfork should not be n=<br>> >eeded under user threads.<BR>>=3B >=3B ...<BR>>=3B >=3B <BR>>=3B =<br>> >>=3B The following code from RTProcessC.c ensures that it is neeeded on e=<br>> >very Unix except<BR>>=3B >=3B FreeBSD before 6. The comment is from the=<br>> > checked-in source file.<BR>>=3B >=3B <BR>>=3B >=3B /* NOTE: Even u=<br>> >serthreads now depends<BR>>=3B >=3B * on availability of pthreads.<BR>&=<br>> >gt=3B >=3B * This can be fixed if need be.<BR>>=3B >=3B */<BR>>=3B =<br>> >>=3B <BR>>=3B >=3B INTEGER<BR>>=3B >=3B __cdecl<BR>>=3B >=3B =<br>> >RTProcess__RegisterForkHandlers(<BR>>=3B >=3B ForkHandler prepare=2C<BR=<br>> >>>=3B >=3B ForkHandler parent=2C<BR>>=3B >=3B ForkHandler child)<BR=<br>> >>>=3B >=3B {<BR>>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.=<br>> > Would be good to use autoconf.<BR>>=3B >=3B * VMS lacks pthread_atfork=<br>> >? Would be good to use autoconf.<BR>>=3B >=3B * Win32 lacks pthread_atf=<br>> >ork and fork. OK.<BR>>=3B >=3B *<BR>>=3B >=3B * As well=2C for all =<br>> >Posix systems=2C we could implement<BR>>=3B >=3B * atfork ourselves=2C =<br>> >as long as we provide a fork()<BR>>=3B >=3B * wrapper that code uses.<B=<br>> >R>>=3B >=3B */<BR>>=3B >=3B #if defined(_WIN32) \<BR>>=3B >=3B =<br>> >|| defined(__vms) \<BR>>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am=<br>> >p=3B (__FreeBSD__ <=3B 6)*/ )<BR>>=3B >=3B return 0=3B<BR>>=3B >=<br>> >=3B #else<BR>>=3B >=3B while (1)<BR>>=3B >=3B {<BR>>=3B >=3B in=<br>> >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B<BR>>=3B >=3B if (=<br>> >i !=3D EAGAIN)<BR>>=3B >=3B return i=3B<BR>>=3B >=3B sleep(0)=3B<BR=<br>> >>>=3B >=3B }<BR>>=3B >=3B #endif<BR>>=3B >=3B }<BR>>=3B <BR> =<br>> > </body><br>> ></html>=<br>> ><br>> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_--<br> </body>
</html>