[M3devel] cvsup

[Jay K]

Please start here:

http://www.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html

  "There are at least two serious problems with the semantics of fork() in a multi-threaded program"

 

 

I've been looking at cvsup a while now.

Lots of RTIO.PutText, etc.

pthread_atfork usage in RTCollector and ThreadPThread.m3.

   Reinitialize in the child, mark the gc threads as not being created yet.

Been using @M3nogc. It helps. I think I understand why.

 

 

I have a somewhat wild educated guess as to the problem.

It is "fork1" vs. "forkall", sort of.

 

 

In user threads, when you fork(), you get all the user Modula-3 threads continuing

to run in the child. Because their existance is an artifact of a bunch of data

that we maintain and gets carried into the new process.

 

 

With kernel threads (with the exception of using Solaris forkall()), you get

just the thread that called fork().

 

 

See my reference to the RTCollector/Allocator atfork change, which I show here (not yet commited):

 

 

PROCEDURE AtForkPrepare() =
  BEGIN
  END AtForkPrepare;

 

PROCEDURE AtForkParent() =
  BEGIN
  END AtForkParent;

 

PROCEDURE AtForkChild() =
  VAR r: INTEGER;
  BEGIN
    r := Thread.PThreadAtFork(AtForkPrepare, AtForkParent, AtForkChild);
    <* ASSERT r = 0 *>
    startedBackground := FALSE;
    startedForeground := FALSE;
  END AtForkChild;

 

PROCEDURE Init () =
  VAR r: INTEGER;
  BEGIN
    r := Thread.PThreadAtFork(AtForkPrepare, AtForkParent, AtForkChild);
    <* ASSERT r = 0 *>


 

Some threads may need to be recreated in the child, some not.

The default is not.

I think we mainly have to adjust cvsup to recreate its threads.

And a little bit of pthread_atfork use in m3core (a bunch of

  it shown believe, and reinitialization in ThreadPThread.m3).

 

 

Like the Apple/Darwin warnings, I don't think most libraries can be considered

"fork safe". That is you can fork, but only if you exec soon thereafter.

Making code "fork safe" I believe equates to reviewing it a bunch

and using pthread_atfork selectively. One thing to watch for is if

the library creates any worker threads during "initialization" or

even "on demand" -- to either recreate them in the child, or note

that they don't exist in the child.

We can make m3core fork safe, I guess, to satisfy cvsup.

 

 

Maybe write the code to duplicate all threads in a child, subject to

some configuration setting? Something like @M3forkall, but it'd

be something that e.g. cvsup would specify when it builds, and

then user wouldn't have to list it anywhere.

Kind of like the flags for incremental and vm gc that get recorded

by the compiler.

 

 

Anyway, let me see about recreating cvsup's threads manually.

Probably by using the Thread.PThreadAtFork I added.

 

I think the initial hang I described fits my theory -- the dispatcher thread

doesn't exist in the child, so hang. And then when I avoid that

with a local edit I get a hang later. I compared without -C and I 

see other stuff happening..I think on other threads that fork() is abandoning..

 

 

Thread.PThreadAtFork is wierd, but I think reasonable.

It will do nothing on user threads and Win32.

It will only do anything on PThreads.

Libraries can use it to achieve compatibility with pthreads programs

that use fork.

 

 - Jay

 
> Date: Wed, 17 Mar 2010 10:19:59 +0100
> From: wagner at elegosoft.com
> To: m3devel at elegosoft.com
> Subject: Re: [M3devel] cvsup
> 
> Quoting Daniel Solaz <m3 at sol42.com>:
> 
> > On 16 Mar 2010, at 20:32, Tony Hosking wrote:
> >> fork should still work with pthreads. Why wouldn't it?
> > Chapter 6.1 of Butenhof's pthreads book deals with this, and well, 
> > it is a mess. According to him only the calling thread exists on 
> > return from fork() in the child process, but the other pthread 
> > *states* (mutexes, conditions, data keys, etc.) are replicated as 
> > well. And I'm sure the details vary from system to system.
> > This is the chapter's first sentence: "Avoid using fork in a 
> > threaded program (if you can) unless you intend to exec a new 
> > program immediately."
> 
> Is this about kernel threads or user level threads? Or about a specific
> implementation of one or the other?
> 
> I don't know the book, but I always thought about pthreads as being the
> POSIX specification of threads, and I don't think there's anything
> in that relating to fork, AFAIR, but that was years ago.
> 
> Can anyone enlighten us about the POSIX specs and if they define the
> semantics of fork in presence of threaded processes? Or does it really
> boil down to the above `avoid it if you can'?
> 
> Olaf
> -- 
> Olaf Wagner -- elego Software Solutions GmbH
> Gustav-Meyer-Allee 25 / Gebäude 12, 13355 Berlin, Germany
> phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95
> http://www.elegosoft.com | Geschäftsführer: Olaf Wagner | Sitz: Berlin
> Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100317/7c61c1f9/attachment-0002.html>