[M3devel] Ho hum... AtFork...

[mika at async.caltech.edu]

Question... is there something odd about my pthreads?  Are pthreads normally reentrant?  I didn't think so.

My compiler is much happier with the following changes I already outlined:

1. a dirty, disgusting hack to keep from locking against myself going from XWait with self.mutex locked to m.release().

2. the change to LockHeap I described in previous email

But now...

(gdb) cont
Continuing.
ERROR: pthread_mutex_lock:11
ERROR: pthread_mutex_lock:11

Program received signal SIGABRT, Aborted.
0x000000080107626a in thr_kill () from /lib/libc.so.7
(gdb) where
#0  0x000000080107626a in thr_kill () from /lib/libc.so.7
#1  0x000000080113dac9 in abort () from /lib/libc.so.7
#2  0x000000000071e37a in ThreadPThread__pthread_mutex_lock (mutex=Error accessing memory address 0x8000ffffb508: Bad address.
) at ../src/thread/PTHREAD/ThreadPThreadC.c:543
#3  0x000000000071d48d in RTOS__LockHeap () at ../src/thread/PTHREAD/ThreadPThread.m3:1377
#4  0x0000000000706b9d in RTHooks__CheckLoadTracedRef (M3_Af40ku_ref=Error accessing memory address 0x8000ffffb568: Bad address.
) at ../src/runtime/common/RTCollector.m3:2234
#5  0x000000000071d284 in ThreadPThread__AtForkParent () at ../src/thread/PTHREAD/ThreadPThread.m3:1348
#6  0x0000000800df8733 in fork () from /lib/libthr.so.3
#7  0x000000000070dd8b in RTProcess__Fork () at ../src/runtime/common/RTProcessC.c:152
#8  0x00000000006c52f2 in ProcessPosixCommon__Create_ForkExec (M3_Bd56fi_cmd=Error accessing memory address 0x8000ffffb6f8: Bad address.
) at ../src/os/POSIX/ProcessPosixCommon.m3:75
#9  0x00000000006c6c6c in Process__Create (M3_Bd56fi_cmd=Error accessing memory address 0x8000ffffb7f8: Bad address.
) at ../src/os/POSIX/ProcessPosix.m3:21
#10 0x00000000004d6826 in QMachine__FulfilExecPromise (M3_D6rRrg_ep=Error accessing memory address 0x8000ffffb838: Bad address.
) at ../src/QMachine.m3:1666
#11 0x00000000004d6220 in QMachine__ExecCommand (M3_An02H2_t=Error accessing memory address 0x8000ffffb9f8: Bad address.
) at ../src/QMachine.m3:1605
#12 0x00000000004d537e in QMachine__DoTryExec (M3_An02H2_t=Error accessing memory address 0x8000ffffbee8: Bad address.
) at ../src/QMachine.m3:1476

What am I doing wrong here?

The error doesn't look unreasonable!  Looking more closely at the code:

First, AtForkPrepare has been called:

PROCEDURE AtForkPrepare() =
  VAR me := GetActivation();
      act: Activation;
  BEGIN
    PThreadLockMutex(slotsMu, ThisLine());
    PThreadLockMutex(perfMu, ThisLine());
    PThreadLockMutex(initMu, ThisLine()); (* InitMutex => RegisterFinalCleanup => LockHeap *)
    PThreadLockMutex(heapMu, ThisLine());
    PThreadLockMutex(activeMu, ThisLine()); (* LockHeap => SuspendOthers => activeMu *)
    (* Walk activations and lock all threads.
     * NOTE: We have initMu, activeMu, so slots won't change, conditions and
     * mutexes won't be initialized on-demand.
     *)
    act := me;
    REPEAT
      PThreadLockMutex(act.mutex, ThisLine());
      act := act.next;
    UNTIL act = me;
  END AtForkPrepare;

a postcondition of this routine is that heapMu is locked.

now we get into AtForkParent:

PROCEDURE AtForkParent() =
  VAR me := GetActivation();
      act: Activation;
      cond: Condition;
  BEGIN
    (* Walk activations and unlock all threads, conditions. *)
    act := me;
    REPEAT
      cond := slots[act.slot].join;
      IF cond # NIL THEN PThreadUnlockMutex(cond.mutex, ThisLine()) END;
      PThreadUnlockMutex(act.mutex, ThisLine());
      act := act.next;
    UNTIL act = me;
    PThreadUnlockMutex(activeMu, ThisLine());
    PThreadUnlockMutex(heapMu, ThisLine());
    PThreadUnlockMutex(initMu, ThisLine());
    PThreadUnlockMutex(perfMu, ThisLine());
    PThreadUnlockMutex(slotsMu, ThisLine());
  END AtForkParent;

We can see by inspecting the code that a necessary precondition for
this routine is that heapMu is locked!  (Since it's going to unlock it,
it had BETTER be locked on entry.)

But the cond := ... causes a RTHooks.CheckLoadTracedRef

which causes an RTOS.LockHeap

the code of which we just saw:

PROCEDURE LockHeap () =
  VAR self := pthread_self();
  BEGIN
    WITH r = pthread_mutex_lock(heapMu,ThisLine()) DO <*ASSERT r=0*> END;
...

we try to lock heapMu.  kaboom!  No surprise there, really?

Am I going about this totally the wrong way?  Other people are running Modula-3
with pthreads, right?  Right??  Somewhere out there in m3devel-land?

     Mika