[M3devel] Ho hum... AtFork...
Tony Hosking
hosking at cs.purdue.edu
Wed Aug 13 18:40:22 CEST 2014
On Aug 13, 2014, at 12:37 PM, mika at async.caltech.edu wrote:
> 1.262
>
> Short summary: 1.262 seems to work fine on AMD64_LINUX except for the
> fork issue (a bummer because my parallelization of the backends of cm3
> uses fork a lot...and it does speed up the compiler significantly if
> you have fast disks and lots of cores) It does the attached on AMD64_FREEBSD.
Yes, that was the state of things. Subsequent revs were to address fork issues.
>
> I see the intent of what you were doing in later version/s, but it was
> a bit rough, I think... I think the three issues I reported (1. locking
> heapMu from AtFork; 2. the try-only-once bug in LockHeap itself; 3. not
> passing the lock to release from Wait) are all more or less real.
Working on these right now. Sorry for hacked versions — no time right now to do real testing before push.
Getting closer now I think.
>
> Mika
>
> Tony Hosking writes:
>> Which revision of ThreadPThread.m3 is this output from?
>>
>> On Aug 13, 2014, at 2:30 AM, mika at async.caltech.edu wrote:
>>
>>> =20
>>> OK... deleted all the derived directories and rebuilt. Result on =
>> FreeBSD is similar (or identical, hard to tell):
>>> =20
>>> I get a partial deadlock:
>>> =20
>>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU =
>> COMMAND
>>> 48760 root 98 0 87836K 29104K CPU1 1 12:32 83.89% =
>> threadtest{threadtest}
>>> 48760 root 94 0 87836K 29104K CPU3 3 11:02 74.66% =
>> threadtest{threadtest}
>>> 48760 root 94 0 87836K 29104K CPU0 0 11:03 71.97% =
>> threadtest{threadtest}
>>> 48760 root 32 0 87836K 29104K umtxn 2 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 20 0 87836K 29104K umtxn 2 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 35 0 87836K 29104K uwait 2 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 27 0 87836K 29104K uwait 0 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 27 0 87836K 29104K umtxn 3 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 27 0 87836K 29104K umtxn 2 0:01 0.00% =
>> threadtest{threadtest}
>>> 48760 root 20 0 87836K 29104K umtxn 0 0:00 0.00% =
>> threadtest{threadtest}
>>> 48759 root 20 0 30276K 11064K wait 0 0:00 0.00% gdb
>>> =20
>>> one of the dead threads is the main thread, so now output. But three
>>> threads are still alive, interestingly enough.
>>> =20
>>> Looks like the same issue as before:
>>> =20
>>> Thread 5 (Thread 801807400 (LWP 100751/threadtest)):
>>> #0 0x0000000800ae089c in __error () from /lib/libthr.so.3
>>> #1 0x0000000800ad6261 in pthread_suspend_np () from /lib/libthr.so.3
>>> #2 0x00000000004511bc in ThreadPThread__SuspendThread (mt=3DError =
>> accessing memory address 0x8000ff5faa28: Bad address.
>>> ) at ../src/thread/PTHREAD/ThreadFreeBSD.c:33
>>> #3 0x000000000044ed8e in ThreadPThread__StopThread =
>> (M3_DMxDjQ_act=3DError accessing memory address 0x8000ff5faa48: Bad =
>> address.
>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:909
>>> #4 0x000000000044ef31 in ThreadPThread__StopWorld () at =
>> ../src/thread/PTHREAD/ThreadPThread.m3:948
>>> #5 0x000000000044e44e in RTThread__SuspendOthers () at =
>> ../src/thread/PTHREAD/ThreadPThread.m3:713
>>> #6 0x0000000000436386 in RTCollector__CollectSomeInStateZero () at =
>> ../src/runtime/common/RTCollector.m3:749
>>> #7 0x0000000000436331 in RTCollector__CollectSome () at =
>> ../src/runtime/common/RTCollector.m3:723
>>> #8 0x0000000000435ff9 in RTHeapRep__CollectEnough () at =
>> ../src/runtime/common/RTCollector.m3:657
>>> #9 0x0000000000433233 in RTAllocator__AllocTraced =
>> (M3_Cwb5VA_dataSize=3DError accessing memory address 0x8000ff5fac88: Bad =
>> address.
>>> ) at ../src/runtime/common/RTAllocator.m3:367
>>> #10 0x0000000000432b55 in RTAllocator__GetOpenArray =
>> (M3_Eic7CK_def=3DError accessing memory address 0x8000ff5fad48: Bad =
>> address.
>>> ) at ../src/runtime/common/RTAllocator.m3:296
>>> #11 0x0000000000431e8f in RTHooks__AllocateOpenArray =
>> (M3_AJWxb1_defn=3DError accessing memory address 0x8000ff5fada8: Bad =
>> address.
>>> ) at ../src/runtime/common/RTAllocator.m3:143
>>> #12 0x00000000004052ea in Main__AApply (M3_AP7a1g_cl=3DError accessing =
>> memory address 0x8000ff5fade8: Bad address.
>>> ) at ../src/Main.m3:283
>>> #13 0x000000000044d243 in ThreadPThread__RunThread (M3_DMxDjQ_me=3DError=
>> accessing memory address 0x8000ff5faeb8: Bad address.
>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>> #14 0x000000000044cf00 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3DError accessing memory address 0x8000ff5faf68: Bad =
>> address.
>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>> #15 0x0000000800ad54a4 in pthread_create () from /lib/libthr.so.3
>>> #16 0x0000000000000000 in ?? ()
>>> =20
>>> Interestingly the same test seems to work on AMD64_LINUX. That is
>>> promising, at least. But of course, software testing never reveals =
>> the
>>> absence of bugs.
>>> =20
>>> Mika
>>> =20
>>> Peter McKinna writes:
>>>> --089e01183a1813188705007c3498
>>>> Content-Type: text/plain; charset=3DUTF-8
>>>> =20
>>>> Oh, one other thing that seemed to help me in testing this stuff =
>> especially
>>>> things to do with m3core and libm3 which seem to get out of whack
>>>> sometimes, is to totally remove the target directory of m3core then =
>> cm3;
>>>> cm3 -ship Same with libm3. I'm not sure that clean actually removes =
>> the
>>>> targets and I think there are leftover files that stuff things up =
>> since I
>>>> would get segv in FileRd for no obvious reason.
>>>> =20
>>>> Peter
>>>> =20
>>>> =20
>>>> =20
>>>> On Wed, Aug 13, 2014 at 3:33 PM, Peter McKinna =
>> <peter.mckinna at gmail.com>
>>>> wrote:
>>>> =20
>>>>> That is weird. The thread tester works fine with the old version of
>>>>> ThreadPThread.m3 on linux amd_64 except if you have the fork test in =
>> the
>>>>> mix of tests you see the pthread_mutex_destroy every now and again, =
>> the
>>>>> fork bug in fact.
>>>>> I was testing the pthread changes Tony has made over the past few =
>> days
>>>>> using the thread tester program until I stupidly rebuilt the =
>> compiler and
>>>>> was getting all sorts of hangs as it dragged in the new version of
>>>>> pthreads. So I rebuilt everything from a backup to get back where I =
>> was
>>>>> with a decent compiler, then reintroduced the latest changes to
>>>>> ThreadPThread.m3, which yesterday was just hanging in sigsuspend but =
>> today
>>>>> is giving me an assert failure at line 1387 in UnlockHeap. So its a =
>> bit
>>>>> unstable to say the least. Maybe there is something else wrong with =
>> freebsd.
>>>>> =20
>>>>> Peter
>>>>> =20
>>>>> =20
>>>>> =20
>>>>> On Wed, Aug 13, 2014 at 2:54 PM, <mika at async.caltech.edu> wrote:
>>>>> =20
>>>>>> =20
>>>>>> Yeah OK 1.262 makes a lot more sense to me but I'm still not =
>> getting
>>>>>> things to work and now I'm really baffled because I think I =
>> understand
>>>>>> this code!
>>>>>> =20
>>>>>> What I did was I threw away all my changes and reverted =
>> ThreadPThread.m3
>>>>>> to 1.262 as you suggested.
>>>>>> =20
>>>>>> Rebuilt the compiler with upgrade.sh
>>>>>> =20
>>>>>> Then rebuilt the compiler again with itself.
>>>>>> =20
>>>>>> Then I realcleaned the world and buildshipped it. (I was afraid
>>>>>> that parseparams, also imported by the thread tester, would pollute
>>>>>> it somehow.)
>>>>>> =20
>>>>>> When I look at the code in 1.262 it looks quite straightforward. =
>> heapMu
>>>>>> is just a mutex, (Modula-3) mutexes are just (pthreads) mutexes.
>>>>>> Condition
>>>>>> variables are mutexes too, but that's no big deal.
>>>>>> =20
>>>>>> So, rebuild thread tester, run it:
>>>>>> =20
>>>>>> new source -> compiling Main.m3
>>>>>> -> linking threadtest
>>>>>> root at rover:~mika/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread #
>>>>>> AMD64_FREEBSD/threadtest
>>>>>> Writing file...done
>>>>>> Creating read threads...done
>>>>>> Creating fork threads...done
>>>>>> Creating alloc threads...done
>>>>>> Creating lock threads...done
>>>>>> running...printing oldest/median age/newest
>>>>>> .
>>>>>> =20
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> *** Segmentation violation - possible attempt to dereference
>>>>>> NIL.........laziest thread is 1407901189/1407901189/9 (tests: read
>>>>>> 1407901189/1407901189/1407901189 fork =
>> 1407901189/1407901189/1407901189
>>>>>> alloc 9/9/9 lock 1407901189/1407901189/9)
>>>>>> =20
>>>>>> =20
>>>>>> =20
>>>>>> =20
>>>>>> ^C
>>>>>> root at rover:~mika/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread # gdb =
>> !$
>>>>>> gdb AMD64_FREEBSD/threadtest
>>>>>> GNU gdb 6.1.1 [FreeBSD]
>>>>>> Copyright 2004 Free Software Foundation, Inc.
>>>>>> GDB is free software, covered by the GNU General Public License, =
>> and you
>>>>>> are
>>>>>> welcome to change it and/or distribute copies of it under certain
>>>>>> conditions.
>>>>>> Type "show copying" to see the conditions.
>>>>>> There is absolutely no warranty for GDB. Type "show warranty" for
>>>>>> details.
>>>>>> This GDB was configured as "amd64-marcel-freebsd"...
>>>>>> (gdb) run
>>>>>> Starting program:
>>>>>> =
>> /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread/AMD64_FREEBSD/th=
>> readtest
>>>>>> [New LWP 100121]
>>>>>> Writing file...done
>>>>>> Creating read threads...done
>>>>>> Creating fork threads...done
>>>>>> Creating alloc threads...done
>>>>>> Creating lock threads...done
>>>>>> running...printing oldest/median age/newest
>>>>>> .[New Thread 801807000 (LWP 100343/threadtest)]
>>>>>> [New Thread 801807c00 (LWP 100667/threadtest)]
>>>>>> [New Thread 801808800 (LWP 100816/threadtest)]
>>>>>> [New Thread 801807400 (LWP 100349/threadtest)]
>>>>>> [New Thread 801808400 (LWP 100815/threadtest)]
>>>>>> [New Thread 801807800 (LWP 100352/threadtest)]
>>>>>> [New Thread 801808c00 (LWP 100817/threadtest)]
>>>>>> [New Thread 801809000 (LWP 100818/threadtest)]
>>>>>> [New Thread 801809800 (LWP 100820/threadtest)]
>>>>>> [New Thread 801808000 (LWP 100678/threadtest)]
>>>>>> [New Thread 801806400 (LWP 100121/threadtest)]
>>>>>> [New Thread 801806800 (LWP 100341/threadtest)]
>>>>>> [New Thread 801806c00 (LWP 100342/threadtest)]
>>>>>> .ERROR: pthread_mutex_lock:11
>>>>>> =20
>>>>>> Program received signal SIGABRT, Aborted.
>>>>>> [Switching to Thread 801809800 (LWP 100820/threadtest)]
>>>>>> 0x0000000800d5c26a in thr_kill () from /lib/libc.so.7
>>>>>> (gdb) where
>>>>>> #0 0x0000000800d5c26a in thr_kill () from /lib/libc.so.7
>>>>>> #1 0x0000000800e23ac9 in abort () from /lib/libc.so.7
>>>>>> #2 0x000000000045101f in ThreadPThread__pthread_mutex_lock =
>> (mutex=3DError
>>>>>> accessing memory address 0x8000fe5f2d98: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:513
>>>>>> #3 0x000000000044b370 in ThreadPThread__LockMutex =
>> (M3_AYIbX3_m=3DError
>>>>>> accessing memory address 0x8000fe5f2dc8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:119
>>>>>> #4 0x0000000000405b5d in Main__LApply (M3_AP7a1g_cl=3DError =
>> accessing
>>>>>> memory address 0x8000fe5f2df8: Bad address.
>>>>>> ) at ../src/Main.m3:319
>>>>>> #5 0x000000000044d243 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3DError
>>>>>> accessing memory address 0x8000fe5f2eb8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>>>>> #6 0x000000000044cf00 in ThreadPThread__ThreadBase
>>>>>> (M3_AJWxb1_param=3DError accessing memory address 0x8000fe5f2f68: =
>> Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>>>>> #7 0x0000000800ad54a4 in pthread_create () from /lib/libthr.so.3
>>>>>> #8 0x0000000000000000 in ?? ()
>>>>>> (gdb) set lang c
>>>>>> (gdb) thread apply all bt
>>>>>> ... not that interesting ...
>>>>>> =20
>>>>>> Segfault is segfault---error 11 is EDEADLK (locking against =
>> myself?)
>>>>>> =20
>>>>>> ????? the code is very straightforward, as I said.
>>>>>> =20
>>>>>> I thought people had the thread tester working with pthreads? =
>> Which set
>>>>>> of files, then? Anyone on FreeBSD/amd64?
>>>>>> =20
>>>>>> Could it be an issue with "volatile"? Not even sure where to look.
>>>>>> =20
>>>>>> The code calling the lock is just this:
>>>>>> =20
>>>>>> PROCEDURE GetChar (rd: T): CHAR
>>>>>> RAISES {EndOfFile, Failure, Alerted} =3D
>>>>>> BEGIN
>>>>>> LOCK rd DO
>>>>>> RETURN FastGetChar(rd);
>>>>>> END
>>>>>> END GetChar;
>>>>>> =20
>>>>>> No mysteries there...
>>>>>> =20
>>>>>> Ah is this the fork bug? Looks like the Init routine is called on =
>> a
>>>>>> lot of NIL mutexes around the fork. But the subprocesses aren't =
>> meant
>>>>>> to accesses the mutexes... humm
>>>>>> =20
>>>>>> Hmm, and it's not entirely a fork issue. Even with "threadtest =
>> -tests
>>>>>> STD,-fork,-forktoomuch" it misbehaves. Hangs....
>>>>>> =20
>>>>>> Looks like a deadlock this time.
>>>>>> =20
>>>>>> Some stacks...
>>>>>> =20
>>>>>> Thread 4 (Thread 800c0b400 (LWP 100477/threadtest)):
>>>>>> #0 0x00000000004b4e2b in __vdso_gettimeofday ()
>>>>>> #1 0x00000000004ab8d2 in gettimeofday ()
>>>>>> #2 0x0000000000452e4b in TimePosix__Now () at
>>>>>> ../src/time/POSIX/TimePosixC.c:50
>>>>>> #3 0x0000000000452d72 in Time__Now () at
>>>>>> ../src/time/POSIX/TimePosix.m3:14
>>>>>> #4 0x00000000004029a3 in Main__LApply (M3_AP7a1g_cl=3DError =
>> accessing
>>>>>> memory address 0x8000fedf6df8: Bad address.
>>>>>> ) at ../src/Main.m3:327
>>>>>> #5 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3DError
>>>>>> accessing memory address 0x8000fedf6eb8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>>>>> #6 0x0000000000449c50 in ThreadPThread__ThreadBase
>>>>>> (M3_AJWxb1_param=3DError accessing memory address 0x8000fedf6f68: =
>> Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>>>>> #7 0x000000000047c7d4 in thread_start ()
>>>>>> #8 0x0000000000000000 in ?? ()
>>>>>> =20
>>>>>> Thread 3 (Thread 800c0b000 (LWP 100476/threadtest)):
>>>>>> #0 0x000000000047e53c in _umtx_op_err ()
>>>>>> #1 0x0000000000475f14 in __thr_umutex_lock ()
>>>>>> #2 0x0000000000479404 in mutex_lock_common ()
>>>>>> #3 0x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3DError
>>>>>> accessing memory address 0x8000feff7d98: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506
>>>>>> #4 0x00000000004480c0 in ThreadPThread__LockMutex =
>> (M3_AYIbX3_m=3DError
>>>>>> accessing memory address 0x8000feff7dc8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:119
>>>>>> #5 0x00000000004028ad in Main__LApply (M3_AP7a1g_cl=3DError =
>> accessing
>>>>>> memory address 0x8000feff7df8: Bad address.
>>>>>> ) at ../src/Main.m3:319
>>>>>> #6 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3DError
>>>>>> accessing memory address 0x8000feff7eb8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>>>>> #7 0x0000000000449c50 in ThreadPThread__ThreadBase
>>>>>> (M3_AJWxb1_param=3DError accessing memory address 0x8000feff7f68: =
>> Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>>>>> #8 0x000000000047c7d4 in thread_start ()
>>>>>> #9 0x0000000000000000 in ?? ()
>>>>>> =20
>>>>>> Thread 6 (Thread 800c09800 (LWP 100470/threadtest)):
>>>>>> #0 0x000000000047e53c in _umtx_op_err ()
>>>>>> #1 0x0000000000475759 in suspend_common ()
>>>>>> #2 0x00000000004755c1 in pthread_suspend_np ()
>>>>>> #3 0x000000000044df0c in ThreadPThread__SuspendThread (mt=3DError
>>>>>> accessing memory address 0x8000ffbfd6f8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadFreeBSD.c:33
>>>>>> #4 0x000000000044bade in ThreadPThread__StopThread =
>> (M3_DMxDjQ_act=3DError
>>>>>> accessing memory address 0x8000ffbfd718: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:909
>>>>>> #5 0x000000000044bc81 in ThreadPThread__StopWorld () at
>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:948
>>>>>> #6 0x000000000044b19e in RTThread__SuspendOthers () at
>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:713
>>>>>> #7 0x00000000004330d6 in RTCollector__CollectSomeInStateZero () at
>>>>>> ../src/runtime/common/RTCollector.m3:749
>>>>>> #8 0x0000000000433081 in RTCollector__CollectSome () at
>>>>>> ../src/runtime/common/RTCollector.m3:723
>>>>>> #9 0x0000000000432d49 in RTHeapRep__CollectEnough () at
>>>>>> ../src/runtime/common/RTCollector.m3:657
>>>>>> #10 0x000000000042ff83 in RTAllocator__AllocTraced
>>>>>> (M3_Cwb5VA_dataSize=3DError accessing memory address =
>> 0x8000ffbfd958: Bad
>>>>>> address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:367
>>>>>> #11 0x000000000042f8a5 in RTAllocator__GetOpenArray =
>> (M3_Eic7CK_def=3DError
>>>>>> accessing memory address 0x8000ffbfda18: Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:296
>>>>>> #12 0x000000000042ebdf in RTHooks__AllocateOpenArray
>>>>>> (M3_AJWxb1_defn=3DError accessing memory address 0x8000ffbfda78: =
>> Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:143
>>>>>> #13 0x000000000040ca4e in Rd__NextBuff (M3_EkTcCb_rd=3DError =
>> accessing
>>>>>> memory address 0x8000ffbfdab8: Bad address.
>>>>>> ) at ../src/rw/Rd.m3:159
>>>>>> #14 0x000000000040cd43 in UnsafeRd__FastGetChar (M3_EkTcCb_rd=3DError=
>>
>>>>>> accessing memory address 0x8000ffbfdb88: Bad address.
>>>>>> ) at ../src/rw/Rd.m3:187
>>>>>> #15 0x000000000040cc4a in Rd__GetChar (M3_EkTcCb_rd=3DError =
>> accessing
>>>>>> memory address 0x8000ffbfdbd8: Bad address.
>>>>>> ) at ../src/rw/Rd.m3:176
>>>>>> #16 0x000000000040095c in Main__RApply (M3_AP7a1g_cl=3DError =
>> accessing
>>>>>> memory address 0x8000ffbfdc58: Bad address.
>>>>>> ) at ../src/Main.m3:185
>>>>>> =20
>>>>>> #17 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3DError
>>>>>> accessing memory address 0x8000ffbfdeb8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>>>>> #18 0x0000000000449c50 in ThreadPThread__ThreadBase
>>>>>> (M3_AJWxb1_param=3DError accessing memory address 0x8000ffbfdf68: =
>> Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>>>>> #19 0x000000000047c7d4 in thread_start ()
>>>>>> #20 0x0000000000000000 in ?? ()
>>>>>> =20
>>>>>> Thread 5 (Thread 800c09400 (LWP 101035/threadtest)):
>>>>>> #0 0x000000000047e53c in _umtx_op_err ()
>>>>>> #1 0x0000000000475f14 in __thr_umutex_lock ()
>>>>>> #2 0x0000000000479404 in mutex_lock_common ()
>>>>>> #3 0x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3DError
>>>>>> accessing memory address 0x8000ffffc678: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506
>>>>>> #4 0x000000000044d039 in RTOS__LockHeap () at
>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:1337
>>>>>> ---Type <return> to continue, or q <return> to quit---
>>>>>> #5 0x000000000042ff79 in RTAllocator__AllocTraced
>>>>>> (M3_Cwb5VA_dataSize=3DError accessing memory address =
>> 0x8000ffffc6e8: Bad
>>>>>> address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:365
>>>>>> #6 0x000000000042f15b in RTAllocator__GetTracedObj =
>> (M3_Eic7CK_def=3DError
>>>>>> accessing memory address 0x8000ffffc7a8: Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:224
>>>>>> #7 0x000000000042eb23 in RTHooks__AllocateTracedObj
>>>>>> (M3_AJWxb1_defn=3DError accessing memory address 0x8000ffffc7f8: =
>> Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:122
>>>>>> #8 0x000000000045fbe4 in TextCat__Flat (M3_Bd56fi_LText=3DError =
>> accessing
>>>>>> memory address 0x8000ffffc858: Bad address.
>>>>>> ) at ../src/text/TextCat.m3:562
>>>>>> #9 0x000000000045ed5d in TextCat__Balance =
>> (M3_Bd56fi_LText=3D0x800c490b0,
>>>>>> M3_BUgnwf_LInfo=3DError accessing memory address 0x8000ffffc8f8: =
>> Bad address.
>>>>>> ) at ../src/text/TextCat.m3:488
>>>>>> #10 0x000000000045d64f in RTHooks__Concat (M3_Bd56fi_t=3DError =
>> accessing
>>>>>> memory address 0x8000ffffcbb8: Bad address.
>>>>>> ) at ../src/text/TextCat.m3:40
>>>>>> #11 0x000000000040639e in Main_M3 (M3_AcxOUs_mode=3DError accessing =
>> memory
>>>>>> address 0x8000ffffcc38: Bad address.
>>>>>> ) at ../src/Main.m3:593
>>>>>> #12 0x000000000043d55d in RTLinker__RunMainBody (M3_DjPxE3_m=3DError
>>>>>> accessing memory address 0x8000ffffcf88: Bad address.
>>>>>> ) at ../src/runtime/common/RTLinker.m3:408
>>>>>> #13 0x000000000043c8e8 in RTLinker__AddUnitI (M3_DjPxE3_m=3DError =
>> accessing
>>>>>> memory address 0x8000ffffd008: Bad address.
>>>>>> ) at ../src/runtime/common/RTLinker.m3:115
>>>>>> #14 0x000000000043c97c in RTLinker__AddUnit (M3_DjPxE5_b=3DError =
>> accessing
>>>>>> memory address 0x8000ffffd028: Bad address.
>>>>>> ) at ../src/runtime/common/RTLinker.m3:124
>>>>>> #15 0x00000000004004a6 in main (argc=3DError accessing memory =
>> address
>>>>>> 0x8000ffffd07c: Bad address.
>>>>>> ) at _m3main.c:22
>>>>>> =20
>>>>>> Thread 9 (Thread 800c0a400 (LWP 100473/threadtest)):
>>>>>> #0 0x000000000047e53c in _umtx_op_err ()
>>>>>> #1 0x0000000000477e8a in check_suspend ()
>>>>>> #2 0x00000000004780a2 in sigcancel_handler ()
>>>>>> #3 <signal handler called>
>>>>>> #4 0x000000000047e53c in _umtx_op_err ()
>>>>>> #5 0x0000000000475f14 in __thr_umutex_lock ()
>>>>>> #6 0x0000000000479404 in mutex_lock_common ()
>>>>>> #7 0x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3DError
>>>>>> accessing memory address 0x8000ff5fac18: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506
>>>>>> #8 0x000000000044d039 in RTOS__LockHeap () at
>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:1337
>>>>>> #9 0x000000000042ff79 in RTAllocator__AllocTraced
>>>>>> (M3_Cwb5VA_dataSize=3DError accessing memory address =
>> 0x8000ff5fac88: Bad
>>>>>> address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:365
>>>>>> #10 0x000000000042f8a5 in RTAllocator__GetOpenArray =
>> (M3_Eic7CK_def=3DError
>>>>>> accessing memory address 0x8000ff5fad48: Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:296
>>>>>> #11 0x000000000042ebdf in RTHooks__AllocateOpenArray
>>>>>> (M3_AJWxb1_defn=3DError accessing memory address 0x8000ff5fada8: =
>> Bad address.
>>>>>> ) at ../src/runtime/common/RTAllocator.m3:143
>>>>>> #12 0x000000000040203a in Main__AApply (M3_AP7a1g_cl=3DError =
>> accessing
>>>>>> memory address 0x8000ff5fade8: Bad address.
>>>>>> ) at ../src/Main.m3:283
>>>>>> #13 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3DError
>>>>>> accessing memory address 0x8000ff5faeb8: Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449
>>>>>> #14 0x0000000000449c50 in ThreadPThread__ThreadBase
>>>>>> (M3_AJWxb1_param=3DError accessing memory address 0x8000ff5faf68: =
>> Bad address.
>>>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422
>>>>>> #15 0x000000000047c7d4 in thread_start ()
>>>>>> #16 0x0000000000000000 in ?? ()
>>>>>> =20
>>>>>> (others are similar)
>>>>>> =20
>>>>>> Hmm looks like a FreeBSD issue now. It's here...
>>>>>> =20
>>>>>> int
>>>>>> __cdecl
>>>>>> ThreadPThread__SuspendThread (m3_pthread_t mt)
>>>>>> {
>>>>>> ThreadFreeBSD__Fatal(pthread_suspend_np(PTHREAD_FROM_M3(mt)),
>>>>>> "pthread_suspend_np");
>>>>>> return 1;
>>>>>> }
>>>>>> =20
>>>>>> Now this suspend can wait:
>>>>>> =20
>>>>>> static int
>>>>>> suspend_common(struct pthread *curthread, struct pthread *thread,
>>>>>> int waitok)
>>>>>> {
>>>>>> uint32_t tmp;
>>>>>> =20
>>>>>> while (thread->state !=3D PS_DEAD &&
>>>>>> !(thread->flags & THR_FLAGS_SUSPENDED)) {
>>>>>> thread->flags |=3D THR_FLAGS_NEED_SUSPEND;
>>>>>> /* Thread is in creation. */
>>>>>> if (thread->tid =3D=3D TID_TERMINATED)
>>>>>> return (1);
>>>>>> tmp =3D thread->cycle;
>>>>>> _thr_send_sig(thread, SIGCANCEL);
>>>>>> THR_THREAD_UNLOCK(curthread, thread);
>>>>>> if (waitok) {
>>>>>> _thr_umtx_wait_uint(&thread->cycle, tmp, =
>> NULL,
>>>>>> 0); <=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>>>>> THR_THREAD_LOCK(curthread, thread);
>>>>>> } else {
>>>>>> THR_THREAD_LOCK(curthread, thread);
>>>>>> return (0);
>>>>>> }
>>>>>> }
>>>>>> =20
>>>>>> return (1);
>>>>>> }
>>>>>> =20
>>>>>> ... but what it can wait for I am not clear on.
>>>>>> =20
>>>>>> Do things work better on Linux?
>>>>>> =20
>>>>>> What is the status of the fork bug? Can it be worked around by =
>> only
>>>>>> forking via Process.Create with no mutexes held (outside of all =
>> LOCK
>>>>>> blocks)?
>>>>>> =20
>>>>>> Mika
>>>>>> =20
>>>>>> =20
>>>>>> =20
>>>>>> Peter McKinna writes:
>>>>>>> --001a11c2ced4471a2e050079db82
>>>>>>> Content-Type: text/plain; charset=3DUTF-8
>>>>>>> =20
>>>>>>> Mika
>>>>>>> =20
>>>>>>> I think you need to back out Tony's changes to fix the fork bug, =
>> at
>>>>>> least
>>>>>>> for now. Need to reload from cvs version 1.262 for =
>> ThreadPThread.m3 If
>>>>>>> you're using git you're on your own.
>>>>>>> Do a cvs log ThreadPThread.m3 for an explanation for some of the =
>> design
>>>>>>> principles.
>>>>>>> Also if you use gdb then you need to set lanc c before backtraces =
>> so at
>>>>>>> least you can see address names and values even if they are =
>> contorted you
>>>>>>> can extract the M3 name in the parm lists. Also in gdb thread =
>> apply all
>>>>>> bt
>>>>>>> gives all thread backtraces which can be handy to see whose got =
>> the locks
>>>>>>> held.
>>>>>>> =20
>>>>>>> Regards Peter
>>>>>>> =20
>>>>>>> =20
>>>>>>> =20
>>>>>>> On Wed, Aug 13, 2014 at 12:14 PM, <mika at async.caltech.edu> wrote:
>>>>>>> =20
>>>>>>>> =20
>>>>>>>> Question... is there something odd about my pthreads? Are =
>> pthreads
>>>>>>>> normally reentrant? I didn't think so.
>>>>>>>> =20
>>>>>>>> My compiler is much happier with the following changes I already
>>>>>> outlined:
>>>>>>>> =20
>>>>>>>> 1. a dirty, disgusting hack to keep from locking against myself =
>> going
>>>>>> from
>>>>>>>> XWait with self.mutex locked to m.release().
>>>>>>>> =20
>>>>>>>> 2. the change to LockHeap I described in previous email
>>>>>>>> =20
>>>>>>>> But now...
>>>>>>>> =20
>>>>>>>> (gdb) cont
>>>>>>>> Continuing.
>>>>>>>> ERROR: pthread_mutex_lock:11
>>>>>>>> ERROR: pthread_mutex_lock:11
>>>>>>>> =20
>>>>>>>> Program received signal SIGABRT, Aborted.
>>>>>>>> 0x000000080107626a in thr_kill () from /lib/libc.so.7
>>>>>>>> (gdb) where
>>>>>>>> #0 0x000000080107626a in thr_kill () from /lib/libc.so.7
>>>>>>>> #1 0x000000080113dac9 in abort () from /lib/libc.so.7
>>>>>>>> #2 0x000000000071e37a in ThreadPThread__pthread_mutex_lock
>>>>>> (mutex=3DError
>>>>>>>> accessing memory address 0x8000ffffb508: Bad address.
>>>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:543
>>>>>>>> #3 0x000000000071d48d in RTOS__LockHeap () at
>>>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:1377
>>>>>>>> #4 0x0000000000706b9d in RTHooks__CheckLoadTracedRef
>>>>>> (M3_Af40ku_ref=3DError
>>>>>>>> accessing memory address 0x8000ffffb568: Bad address.
>>>>>>>> ) at ../src/runtime/common/RTCollector.m3:2234
>>>>>>>> #5 0x000000000071d284 in ThreadPThread__AtForkParent () at
>>>>>>>> ../src/thread/PTHREAD/ThreadPThread.m3:1348
>>>>>>>> #6 0x0000000800df8733 in fork () from /lib/libthr.so.3
>>>>>>>> #7 0x000000000070dd8b in RTProcess__Fork () at
>>>>>>>> ../src/runtime/common/RTProcessC.c:152
>>>>>>>> #8 0x00000000006c52f2 in ProcessPosixCommon__Create_ForkExec
>>>>>>>> (M3_Bd56fi_cmd=3DError accessing memory address 0x8000ffffb6f8: =
>> Bad
>>>>>> address.
>>>>>>>> ) at ../src/os/POSIX/ProcessPosixCommon.m3:75
>>>>>>>> #9 0x00000000006c6c6c in Process__Create (M3_Bd56fi_cmd=3DError
>>>>>> accessing
>>>>>>>> memory address 0x8000ffffb7f8: Bad address.
>>>>>>>> ) at ../src/os/POSIX/ProcessPosix.m3:21
>>>>>>>> #10 0x00000000004d6826 in QMachine__FulfilExecPromise
>>>>>> (M3_D6rRrg_ep=3DError
>>>>>>>> accessing memory address 0x8000ffffb838: Bad address.
>>>>>>>> ) at ../src/QMachine.m3:1666
>>>>>>>> #11 0x00000000004d6220 in QMachine__ExecCommand =
>> (M3_An02H2_t=3DError
>>>>>>>> accessing memory address 0x8000ffffb9f8: Bad address.
>>>>>>>> ) at ../src/QMachine.m3:1605
>>>>>>>> #12 0x00000000004d537e in QMachine__DoTryExec (M3_An02H2_t=3DError
>>>>>> accessing
>>>>>>>> memory address 0x8000ffffbee8: Bad address.
>>>>>>>> ) at ../src/QMachine.m3:1476
>>>>>>>> =20
>>>>>>>> What am I doing wrong here?
>>>>>>>> =20
>>>>>>>> The error doesn't look unreasonable! Looking more closely at the =
>> code:
>>>>>>>> =20
>>>>>>>> First, AtForkPrepare has been called:
>>>>>>>> =20
>>>>>>>> PROCEDURE AtForkPrepare() =3D
>>>>>>>> VAR me :=3D GetActivation();
>>>>>>>> act: Activation;
>>>>>>>> BEGIN
>>>>>>>> PThreadLockMutex(slotsMu, ThisLine());
>>>>>>>> PThreadLockMutex(perfMu, ThisLine());
>>>>>>>> PThreadLockMutex(initMu, ThisLine()); (* InitMutex =3D>
>>>>>>>> RegisterFinalCleanup =3D> LockHeap *)
>>>>>>>> PThreadLockMutex(heapMu, ThisLine());
>>>>>>>> PThreadLockMutex(activeMu, ThisLine()); (* LockHeap =3D>
>>>>>> SuspendOthers
>>>>>>>> =3D> activeMu *)
>>>>>>>> (* Walk activations and lock all threads.
>>>>>>>> * NOTE: We have initMu, activeMu, so slots won't change,
>>>>>> conditions
>>>>>>>> and
>>>>>>>> * mutexes won't be initialized on-demand.
>>>>>>>> *)
>>>>>>>> act :=3D me;
>>>>>>>> REPEAT
>>>>>>>> PThreadLockMutex(act.mutex, ThisLine());
>>>>>>>> act :=3D act.next;
>>>>>>>> UNTIL act =3D me;
>>>>>>>> END AtForkPrepare;
>>>>>>>> =20
>>>>>>>> a postcondition of this routine is that heapMu is locked.
>>>>>>>> =20
>>>>>>>> now we get into AtForkParent:
>>>>>>>> =20
>>>>>>>> PROCEDURE AtForkParent() =3D
>>>>>>>> VAR me :=3D GetActivation();
>>>>>>>> act: Activation;
>>>>>>>> cond: Condition;
>>>>>>>> BEGIN
>>>>>>>> (* Walk activations and unlock all threads, conditions. *)
>>>>>>>> act :=3D me;
>>>>>>>> REPEAT
>>>>>>>> cond :=3D slots[act.slot].join;
>>>>>>>> IF cond # NIL THEN PThreadUnlockMutex(cond.mutex, =
>> ThisLine())
>>>>>> END;
>>>>>>>> PThreadUnlockMutex(act.mutex, ThisLine());
>>>>>>>> act :=3D act.next;
>>>>>>>> UNTIL act =3D me;
>>>>>>>> PThreadUnlockMutex(activeMu, ThisLine());
>>>>>>>> PThreadUnlockMutex(heapMu, ThisLine());
>>>>>>>> PThreadUnlockMutex(initMu, ThisLine());
>>>>>>>> PThreadUnlockMutex(perfMu, ThisLine());
>>>>>>>> PThreadUnlockMutex(slotsMu, ThisLine());
>>>>>>>> END AtForkParent;
>>>>>>>> =20
>>>>>>>> We can see by inspecting the code that a necessary precondition =
>> for
>>>>>>>> this routine is that heapMu is locked! (Since it's going to =
>> unlock it,
>>>>>>>> it had BETTER be locked on entry.)
>>>>>>>> =20
>>>>>>>> But the cond :=3D ... causes a RTHooks.CheckLoadTracedRef
>>>>>>>> =20
>>>>>>>> which causes an RTOS.LockHeap
>>>>>>>> =20
>>>>>>>> the code of which we just saw:
>>>>>>>> =20
>>>>>>>> PROCEDURE LockHeap () =3D
>>>>>>>> VAR self :=3D pthread_self();
>>>>>>>> BEGIN
>>>>>>>> WITH r =3D pthread_mutex_lock(heapMu,ThisLine()) DO <*ASSERT =
>> r=3D0*>
>>>>>> END;
>>>>>>>> ...
>>>>>>>> =20
>>>>>>>> we try to lock heapMu. kaboom! No surprise there, really?
>>>>>>>> =20
>>>>>>>> Am I going about this totally the wrong way? Other people are =
>> running
>>>>>>>> Modula-3
>>>>>>>> with pthreads, right? Right?? Somewhere out there in =
>> m3devel-land?
>>>>>>>> =20
>>>>>>>> Mika
>>>>>>>> =20
>>>>>>>> =20
>>>>>>> =20
>>>>>>> --001a11c2ced4471a2e050079db82
>>>>>>> Content-Type: text/html; charset=3DUTF-8
>>>>>>> Content-Transfer-Encoding: quoted-printable
>>>>>>> =20
>>>>>>> <div dir=3D3D"ltr">Mika<div><br></div><div>=3DC2=3DA0 I think you =
>> need to back
>>>>>> ou=3D
>>>>>>> t Tony's changes to fix the fork bug, at least for now. Need =
>> to
>>>>>> reload =3D
>>>>>>> from cvs version 1.262 for ThreadPThread.m3 If you're using =
>> git
>>>>>> you'=3D
>>>>>>> ;re on your own.</div>
>>>>>>> =20
>>>>>>> <div>=3DC2=3DA0 Do a cvs log ThreadPThread.m3 for an explanation =
>> for some of
>>>>>> th=3D
>>>>>>> e design principles.=3DC2=3DA0</div><div>=3DC2=3DA0 Also if you =
>> use gdb then you
>>>>>> ne=3D
>>>>>>> ed to set lanc c before backtraces so at least you can see address =
>> names
>>>>>> an=3D
>>>>>>> d values even if they are contorted you can extract the M3 name in =
>> the
>>>>>> parm=3D
>>>>>>> lists. Also in gdb thread apply all bt gives all thread backtraces
>>>>>> which c=3D
>>>>>>> an be handy to see whose got the locks held.</div>
>>>>>>> =20
>>>>>>> <div><br></div><div>Regards Peter</div><div><br></div><div
>>>>>> class=3D3D"gmail_e=3D
>>>>>>> xtra"><br><br><div class=3D3D"gmail_quote">On Wed, Aug 13, 2014 at =
>> 12:14
>>>>>> PM, =3D
>>>>>>> <span dir=3D3D"ltr"><<a href=3D3D"mailto:mika at async.caltech.edu"
>>>>>> target=3D3D"=3D
>>>>>>> _blank">mika at async.caltech.edu</a>></span> wrote:<br>
>>>>>>> =20
>>>>>>> <blockquote class=3D3D"gmail_quote" style=3D3D"margin:0 0 0
>>>>>> .8ex;border-left:1p=3D
>>>>>>> x #ccc solid;padding-left:1ex"><br>
>>>>>>> Question... is there something odd about my pthreads? =3DC2=3DA0Are =
>> pthreads
>>>>>> no=3D
>>>>>>> rmally reentrant? =3DC2=3DA0I didn't think so.<br>
>>>>>>> <br>
>>>>>>> My compiler is much happier with the following changes I already
>>>>>> outlined:<=3D
>>>>>>> br>
>>>>>>> <br>
>>>>>>> 1. a dirty, disgusting hack to keep from locking against myself =
>> going
>>>>>> from =3D
>>>>>>> XWait with self.mutex locked to m.release().<br>
>>>>>>> <br>
>>>>>>> 2. the change to LockHeap I described in previous email<br>
>>>>>>> <br>
>>>>>>> But now...<br>
>>>>>>> <br>
>>>>>>> (gdb) cont<br>
>>>>>>> Continuing.<br>
>>>>>>> ERROR: pthread_mutex_lock:11<br>
>>>>>>> ERROR: pthread_mutex_lock:11<br>
>>>>>>> <br>
>>>>>>> Program received signal SIGABRT, Aborted.<br>
>>>>>>> 0x000000080107626a in thr_kill () from /lib/libc.so.7<br>
>>>>>>> (gdb) where<br>
>>>>>>> #0 =3DC2=3DA00x000000080107626a in thr_kill () from =
>> /lib/libc.so.7<br>
>>>>>>> #1 =3DC2=3DA00x000000080113dac9 in abort () from =
>> /lib/libc.so.7<br>
>>>>>>> #2 =3DC2=3DA00x000000000071e37a in =
>> ThreadPThread__pthread_mutex_lock
>>>>>> (mutex=3D3DE=3D
>>>>>>> rror accessing memory address 0x8000ffffb508: Bad address.<br>
>>>>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:543<br>
>>>>>>> #3 =3DC2=3DA00x000000000071d48d in RTOS__LockHeap () at
>>>>>> ../src/thread/PTHREAD/T=3D
>>>>>>> hreadPThread.m3:1377<br>
>>>>>>> #4 =3DC2=3DA00x0000000000706b9d in RTHooks__CheckLoadTracedRef
>>>>>> (M3_Af40ku_ref=3D
>>>>>>> =3D3DError accessing memory address 0x8000ffffb568: Bad =
>> address.<br>
>>>>>>> ) at ../src/runtime/common/RTCollector.m3:2234<br>
>>>>>>> #5 =3DC2=3DA00x000000000071d284 in ThreadPThread__AtForkParent () =
>> at
>>>>>> ../src/thr=3D
>>>>>>> ead/PTHREAD/ThreadPThread.m3:1348<br>
>>>>>>> #6 =3DC2=3DA00x0000000800df8733 in fork () from =
>> /lib/libthr.so.3<br>
>>>>>>> #7 =3DC2=3DA00x000000000070dd8b in RTProcess__Fork () at
>>>>>> ../src/runtime/common/=3D
>>>>>>> RTProcessC.c:152<br>
>>>>>>> #8 =3DC2=3DA00x00000000006c52f2 in =
>> ProcessPosixCommon__Create_ForkExec
>>>>>> (M3_Bd56=3D
>>>>>>> fi_cmd=3D3DError accessing memory address 0x8000ffffb6f8: Bad =
>> address.<br>
>>>>>>> ) at ../src/os/POSIX/ProcessPosixCommon.m3:75<br>
>>>>>>> #9 =3DC2=3DA00x00000000006c6c6c in Process__Create =
>> (M3_Bd56fi_cmd=3D3DError
>>>>>> acces=3D
>>>>>>> sing memory address 0x8000ffffb7f8: Bad address.<br>
>>>>>>> ) at ../src/os/POSIX/ProcessPosix.m3:21<br>
>>>>>>> #10 0x00000000004d6826 in QMachine__FulfilExecPromise
>>>>>> (M3_D6rRrg_ep=3D3DError=3D
>>>>>>> accessing memory address 0x8000ffffb838: Bad address.<br>
>>>>>>> ) at ../src/QMachine.m3:1666<br>
>>>>>>> #11 0x00000000004d6220 in QMachine__ExecCommand =
>> (M3_An02H2_t=3D3DError
>>>>>> access=3D
>>>>>>> ing memory address 0x8000ffffb9f8: Bad address.<br>
>>>>>>> ) at ../src/QMachine.m3:1605<br>
>>>>>>> #12 0x00000000004d537e in QMachine__DoTryExec (M3_An02H2_t=3D3DError=
>>
>>>>>> accessin=3D
>>>>>>> g memory address 0x8000ffffbee8: Bad address.<br>
>>>>>>> ) at ../src/QMachine.m3:1476<br>
>>>>>>> <br>
>>>>>>> What am I doing wrong here?<br>
>>>>>>> <br>
>>>>>>> The error doesn't look unreasonable! =3DC2=3DA0Looking more =
>> closely at
>>>>>> the =3D
>>>>>>> code:<br>
>>>>>>> <br>
>>>>>>> First, AtForkPrepare has been called:<br>
>>>>>>> <br>
>>>>>>> PROCEDURE AtForkPrepare() =3D3D<br>
>>>>>>> =3DC2=3DA0 VAR me :=3D3D GetActivation();<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act: Activation;<br>
>>>>>>> =3DC2=3DA0 BEGIN<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(slotsMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(perfMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(initMu, ThisLine()); (* =
>> InitMutex =3D3D>
>>>>>> Re=3D
>>>>>>> gisterFinalCleanup =3D3D> LockHeap *)<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(heapMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(activeMu, ThisLine()); (* =
>> LockHeap
>>>>>> =3D3D> S=3D
>>>>>>> uspendOthers =3D3D> activeMu *)<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 (* Walk activations and lock all =
>> threads.<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0* NOTE: We have initMu, activeMu, =
>> so slots won't
>>>>>> ch=3D
>>>>>>> ange, conditions and<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0* mutexes won't be initialized =
>> on-demand.<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0*)<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 act :=3D3D me;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 REPEAT<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(act.mutex, =
>> ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act :=3D3D act.next;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 UNTIL act =3D3D me;<br>
>>>>>>> =3DC2=3DA0 END AtForkPrepare;<br>
>>>>>>> <br>
>>>>>>> a postcondition of this routine is that heapMu is locked.<br>
>>>>>>> <br>
>>>>>>> now we get into AtForkParent:<br>
>>>>>>> <br>
>>>>>>> PROCEDURE AtForkParent() =3D3D<br>
>>>>>>> =3DC2=3DA0 VAR me :=3D3D GetActivation();<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act: Activation;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 cond: Condition;<br>
>>>>>>> =3DC2=3DA0 BEGIN<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 (* Walk activations and unlock all threads, =
>> conditions.
>>>>>> *)<br=3D
>>>>>>>> =20
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 act :=3D3D me;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 REPEAT<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 cond :=3D3D =
>> slots[act.slot].join;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 IF cond # NIL THEN =
>> PThreadUnlockMutex(cond.mutex,
>>>>>> This=3D
>>>>>>> Line()) END;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(act.mutex, =
>> ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act :=3D3D act.next;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 UNTIL act =3D3D me;<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(activeMu, =
>> ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(heapMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(initMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(perfMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(slotsMu, ThisLine());<br>
>>>>>>> =3DC2=3DA0 END AtForkParent;<br>
>>>>>>> <br>
>>>>>>> We can see by inspecting the code that a necessary precondition =
>> for<br>
>>>>>>> this routine is that heapMu is locked! =3DC2=3DA0(Since it's =
>> going to
>>>>>> unloc=3D
>>>>>>> k it,<br>
>>>>>>> it had BETTER be locked on entry.)<br>
>>>>>>> <br>
>>>>>>> But the cond :=3D3D ... causes a RTHooks.CheckLoadTracedRef<br>
>>>>>>> <br>
>>>>>>> which causes an RTOS.LockHeap<br>
>>>>>>> <br>
>>>>>>> the code of which we just saw:<br>
>>>>>>> <br>
>>>>>>> PROCEDURE LockHeap () =3D3D<br>
>>>>>>> =3DC2=3DA0 VAR self :=3D3D pthread_self();<br>
>>>>>>> =3DC2=3DA0 BEGIN<br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 WITH r =3D3D =
>> pthread_mutex_lock(heapMu,ThisLine()) DO
>>>>>> <*ASSE=3D
>>>>>>> RT r=3D3D0*> END;<br>
>>>>>>> ...<br>
>>>>>>> <br>
>>>>>>> we try to lock heapMu. =3DC2=3DA0kaboom! =3DC2=3DA0No surprise =
>> there, really?<br>
>>>>>>> <br>
>>>>>>> Am I going about this totally the wrong way? =3DC2=3DA0Other =
>> people are
>>>>>> running=3D
>>>>>>> Modula-3<br>
>>>>>>> with pthreads, right? =3DC2=3DA0Right?? =3DC2=3DA0Somewhere out =
>> there in
>>>>>> m3devel-la=3D
>>>>>>> nd?<br>
>>>>>>> <span><font color=3D3D"#888888"><br>
>>>>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0Mika<br>
>>>>>>> <br>
>>>>>>> </font></span></blockquote></div><br></div></div>
>>>>>>> =20
>>>>>>> --001a11c2ced4471a2e050079db82--
>>>>>> =20
>>>>> =20
>>>>> =20
>>>> =20
>>>> --089e01183a1813188705007c3498
>>>> Content-Type: text/html; charset=3DUTF-8
>>>> Content-Transfer-Encoding: quoted-printable
>>>> =20
>>>> <div dir=3D3D"ltr">Oh, one other thing that seemed to help me in =
>> testing this=3D
>>>> stuff especially things to do with m3core and libm3 which seem to get =
>> out =3D
>>>> of whack sometimes, is to totally remove the target directory of =
>> m3core the=3D
>>>> n cm3; cm3 -ship Same with libm3. I'm not sure that clean =
>> actually remo=3D
>>>> ves the targets and I think there are leftover files that stuff =
>> things up s=3D
>>>> ince I would get segv in FileRd for no obvious reason.<div>
>>>> <br></div><div>Peter</div><div><br></div></div><div =
>> class=3D3D"gmail_extra"><=3D
>>>> br><br><div class=3D3D"gmail_quote">On Wed, Aug 13, 2014 at 3:33 PM, =
>> Peter Mc=3D
>>>> Kinna <span dir=3D3D"ltr"><<a =
>> href=3D3D"mailto:peter.mckinna at gmail.com" targ=3D
>>>> et=3D3D"_blank">peter.mckinna at gmail.com</a>></span> wrote:<br>
>>>> <blockquote class=3D3D"gmail_quote" style=3D3D"margin:0 0 0 =
>> .8ex;border-left:1p=3D
>>>> x #ccc solid;padding-left:1ex"><div dir=3D3D"ltr">That is weird. The =
>> thread t=3D
>>>> ester works fine with the old version of ThreadPThread.m3 on linux =
>> amd_64 e=3D
>>>> xcept if you have the fork test in the mix of tests you see the =
>> pthread_mut=3D
>>>> ex_destroy every now and again, the fork bug in fact.=3DC2=3DA0<div>
>>>> =20
>>>> I was testing the pthread changes Tony has made over the past few =
>> days usin=3D
>>>> g the thread tester program until I stupidly rebuilt the compiler and =
>> was g=3D
>>>> etting all sorts of hangs as it dragged in the new version of =
>> pthreads. So =3D
>>>> I rebuilt everything from a backup to get back where I was with a =
>> decent co=3D
>>>> mpiler, then reintroduced the latest changes to ThreadPThread.m3, =
>> which yes=3D
>>>> terday was just hanging in sigsuspend but today is giving me an =
>> assert fail=3D
>>>> ure at line 1387 in UnlockHeap. So its a bit unstable to say the =
>> least. May=3D
>>>> be there is something else wrong with freebsd.</div>
>>>> <span class=3D3D"HOEnZb"><font color=3D3D"#888888">
>>>> =
>> <div><br></div><div>Peter</div><div><br></div></font></span></div><div =
>> clas=3D
>>>> s=3D3D"HOEnZb"><div class=3D3D"h5"><div =
>> class=3D3D"gmail_extra"><br><br><div clas=3D
>>>> s=3D3D"gmail_quote">On Wed, Aug 13, 2014 at 2:54 PM, <span =
>> dir=3D3D"ltr"><<=3D
>>>> a href=3D3D"mailto:mika at async.caltech.edu" =
>> target=3D3D"_blank">mika at async.calte=3D
>>>> ch.edu</a>></span> wrote:<br>
>>>> =20
>>>> <blockquote class=3D3D"gmail_quote" style=3D3D"margin:0 0 0 =
>> .8ex;border-left:1p=3D
>>>> x #ccc solid;padding-left:1ex"><br>
>>>> Yeah OK 1.262 makes a lot more sense to me but I'm still not =
>> getting<br=3D
>>>>> =20
>>>> things to work and now I'm really baffled because I think I =
>> understand<=3D
>>>> br>
>>>> this code!<br>
>>>> <br>
>>>> What I did was I threw away all my changes and reverted =
>> ThreadPThread.m3<br=3D
>>>>> =20
>>>> to 1.262 as you suggested.<br>
>>>> <br>
>>>> Rebuilt the compiler with upgrade.sh<br>
>>>> <br>
>>>> Then rebuilt the compiler again with itself.<br>
>>>> <br>
>>>> Then I realcleaned the world and buildshipped it. =3DC2=3DA0(I was =
>> afraid<br>
>>>> that parseparams, also imported by the thread tester, would =
>> pollute<br>
>>>> it somehow.)<br>
>>>> <br>
>>>> When I look at the code in 1.262 it looks quite straightforward. =
>> =3DC2=3DA0heap=3D
>>>> Mu<br>
>>>> is just a mutex, (Modula-3) mutexes are just (pthreads) mutexes. =
>> =3DC2=3DA0Cond=3D
>>>> ition<br>
>>>> variables are mutexes too, but that's no big deal.<br>
>>>> <br>
>>>> So, rebuild thread tester, run it:<br>
>>>> <br>
>>>> new source -> compiling Main.m3<br>
>>>> =3DC2=3DA0-> linking threadtest<br>
>>>> root at rover:~mika/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread # =
>> AMD64_FREEB=3D
>>>> SD/threadtest<br>
>>>> Writing file...done<br>
>>>> Creating read threads...done<br>
>>>> Creating fork threads...done<br>
>>>> Creating alloc threads...done<br>
>>>> Creating lock threads...done<br>
>>>> running...printing oldest/median age/newest<br>
>>>> .<br>
>>>> <br>
>>>> ***<br>
>>>> *** runtime error:<br>
>>>> *** =3DC2=3DA0 =3DC2=3DA0Segmentation violation - possible attempt to =
>> dereference N=3D
>>>> IL.........laziest thread is 1407901189/<a =
>> href=3D3D"tel:1407901189%2F9" valu=3D
>>>> e=3D3D"+14079011899" target=3D3D"_blank">1407901189/9</a> (tests: =
>> read 14079011=3D
>>>> 89/1407901189/1407901189 fork 1407901189/1407901189/1407901189 alloc =
>> 9/9/9 =3D
>>>> lock 1407901189/1407901189/9)<br>
>>>> =20
>>>> =20
>>>> <br>
>>>> <br>
>>>> <br>
>>>> <br>
>>>> ^C<br>
>>>> root at rover:~mika/cm3-cvs-anon/cm3/m3-libs/m3core/tests/thread # gdb =
>> !$<br>
>>>> gdb AMD64_FREEBSD/threadtest<br>
>>>> GNU gdb 6.1.1 [FreeBSD]<br>
>>>> Copyright 2004 Free Software Foundation, Inc.<br>
>>>> GDB is free software, covered by the GNU General Public License, and =
>> you ar=3D
>>>> e<br>
>>>> welcome to change it and/or distribute copies of it under certain =
>> condition=3D
>>>> s.<br>
>>>> Type "show copying" to see the conditions.<br>
>>>> There is absolutely no warranty for GDB. =3DC2=3DA0Type "show =
>> warranty&quo=3D
>>>> t; for details.<br>
>>>> This GDB was configured as "amd64-marcel-freebsd"...<br>
>>>> (gdb) run<br>
>>>> Starting program: =
>> /big/home2/mika/2/cm3-cvs/cm3/m3-libs/m3core/tests/thread=3D
>>>> /AMD64_FREEBSD/threadtest<br>
>>>> [New LWP 100121]<br>
>>>> Writing file...done<br>
>>>> Creating read threads...done<br>
>>>> Creating fork threads...done<br>
>>>> Creating alloc threads...done<br>
>>>> Creating lock threads...done<br>
>>>> running...printing oldest/median age/newest<br>
>>>> .[New Thread 801807000 (LWP 100343/threadtest)]<br>
>>>> [New Thread 801807c00 (LWP 100667/threadtest)]<br>
>>>> [New Thread 801808800 (LWP 100816/threadtest)]<br>
>>>> [New Thread 801807400 (LWP 100349/threadtest)]<br>
>>>> [New Thread 801808400 (LWP 100815/threadtest)]<br>
>>>> [New Thread 801807800 (LWP 100352/threadtest)]<br>
>>>> [New Thread 801808c00 (LWP 100817/threadtest)]<br>
>>>> [New Thread 801809000 (LWP 100818/threadtest)]<br>
>>>> [New Thread 801809800 (LWP 100820/threadtest)]<br>
>>>> [New Thread 801808000 (LWP 100678/threadtest)]<br>
>>>> [New Thread 801806400 (LWP 100121/threadtest)]<br>
>>>> [New Thread 801806800 (LWP 100341/threadtest)]<br>
>>>> [New Thread 801806c00 (LWP 100342/threadtest)]<br>
>>>> .ERROR: pthread_mutex_lock:11<br>
>>>> <div><br>
>>>> Program received signal SIGABRT, Aborted.<br>
>>>> </div>[Switching to Thread 801809800 (LWP 100820/threadtest)]<br>
>>>> 0x0000000800d5c26a in thr_kill () from /lib/libc.so.7<br>
>>>> (gdb) where<br>
>>>> #0 =3DC2=3DA00x0000000800d5c26a in thr_kill () from =
>> /lib/libc.so.7<br>
>>>> #1 =3DC2=3DA00x0000000800e23ac9 in abort () from /lib/libc.so.7<br>
>>>> #2 =3DC2=3DA00x000000000045101f in ThreadPThread__pthread_mutex_lock =
>> (mutex=3D3DE=3D
>>>> rror accessing memory address 0x8000fe5f2d98: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:513<br>
>>>> #3 =3DC2=3DA00x000000000044b370 in ThreadPThread__LockMutex =
>> (M3_AYIbX3_m=3D3DErro=3D
>>>> r accessing memory address 0x8000fe5f2dc8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:119<br>
>>>> #4 =3DC2=3DA00x0000000000405b5d in Main__LApply (M3_AP7a1g_cl=3D3DError=
>> accessing=3D
>>>> memory address 0x8000fe5f2df8: Bad address.<br>
>>>> ) at ../src/Main.m3:319<br>
>>>> #5 =3DC2=3DA00x000000000044d243 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3D3DErr=3D
>>>> or accessing memory address 0x8000fe5f2eb8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449<br>
>>>> #6 =3DC2=3DA00x000000000044cf00 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3D
>>>> =3D3DError accessing memory address 0x8000fe5f2f68: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422<br>
>>>> #7 =3DC2=3DA00x0000000800ad54a4 in pthread_create () from =
>> /lib/libthr.so.3<br>
>>>> #8 =3DC2=3DA00x0000000000000000 in ?? ()<br>
>>>> (gdb) set lang c<br>
>>>> (gdb) thread apply all bt<br>
>>>> ... not that interesting ...<br>
>>>> <br>
>>>> Segfault is segfault---error 11 is EDEADLK (locking against =
>> myself?)<br>
>>>> <br>
>>>> ????? the code is very straightforward, as I said.<br>
>>>> <br>
>>>> I thought people had the thread tester working with pthreads? =
>> =3DC2=3DA0Which s=3D
>>>> et of files, then? =3DC2=3DA0Anyone on FreeBSD/amd64?<br>
>>>> <br>
>>>> Could it be an issue with "volatile"? =3DC2=3DA0Not even =
>> sure where t=3D
>>>> o look.<br>
>>>> <br>
>>>> The code calling the lock is just this:<br>
>>>> <br>
>>>> PROCEDURE GetChar (rd: T): CHAR<br>
>>>> =3DC2=3DA0 RAISES {EndOfFile, Failure, Alerted} =3D3D<br>
>>>> =3DC2=3DA0 BEGIN<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 LOCK rd DO<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 RETURN FastGetChar(rd);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 END<br>
>>>> =3DC2=3DA0 END GetChar;<br>
>>>> <br>
>>>> No mysteries there...<br>
>>>> <br>
>>>> Ah is this the fork bug? =3DC2=3DA0Looks like the Init routine is =
>> called on a<b=3D
>>>> r>
>>>> lot of NIL mutexes around the fork. =3DC2=3DA0But the subprocesses =
>> aren't m=3D
>>>> eant<br>
>>>> to accesses the mutexes... humm<br>
>>>> <br>
>>>> Hmm, and it's not entirely a fork issue. =3DC2=3DA0Even with =
>> "threadte=3D
>>>> st -tests STD,-fork,-forktoomuch" it misbehaves. =
>> =3DC2=3DA0Hangs....<br>
>>>> <br>
>>>> Looks like a deadlock this time.<br>
>>>> <br>
>>>> Some stacks...<br>
>>>> <br>
>>>> Thread 4 (Thread 800c0b400 (LWP 100477/threadtest)):<br>
>>>> #0 =3DC2=3DA00x00000000004b4e2b in __vdso_gettimeofday ()<br>
>>>> #1 =3DC2=3DA00x00000000004ab8d2 in gettimeofday ()<br>
>>>> #2 =3DC2=3DA00x0000000000452e4b in TimePosix__Now () at =
>> ../src/time/POSIX/TimeP=3D
>>>> osixC.c:50<br>
>>>> #3 =3DC2=3DA00x0000000000452d72 in Time__Now () at =
>> ../src/time/POSIX/TimePosix.=3D
>>>> m3:14<br>
>>>> #4 =3DC2=3DA00x00000000004029a3 in Main__LApply (M3_AP7a1g_cl=3D3DError=
>> accessing=3D
>>>> memory address 0x8000fedf6df8: Bad address.<br>
>>>> ) at ../src/Main.m3:327<br>
>>>> #5 =3DC2=3DA00x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3D3DErr=3D
>>>> or accessing memory address 0x8000fedf6eb8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449<br>
>>>> #6 =3DC2=3DA00x0000000000449c50 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3D
>>>> =3D3DError accessing memory address 0x8000fedf6f68: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422<br>
>>>> #7 =3DC2=3DA00x000000000047c7d4 in thread_start ()<br>
>>>> #8 =3DC2=3DA00x0000000000000000 in ?? ()<br>
>>>> <br>
>>>> Thread 3 (Thread 800c0b000 (LWP 100476/threadtest)):<br>
>>>> #0 =3DC2=3DA00x000000000047e53c in _umtx_op_err ()<br>
>>>> #1 =3DC2=3DA00x0000000000475f14 in __thr_umutex_lock ()<br>
>>>> #2 =3DC2=3DA00x0000000000479404 in mutex_lock_common ()<br>
>>>> #3 =3DC2=3DA00x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3D3DE=3D
>>>> rror accessing memory address 0x8000feff7d98: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506<br>
>>>> #4 =3DC2=3DA00x00000000004480c0 in ThreadPThread__LockMutex =
>> (M3_AYIbX3_m=3D3DErro=3D
>>>> r accessing memory address 0x8000feff7dc8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:119<br>
>>>> #5 =3DC2=3DA00x00000000004028ad in Main__LApply (M3_AP7a1g_cl=3D3DError=
>> accessing=3D
>>>> memory address 0x8000feff7df8: Bad address.<br>
>>>> ) at ../src/Main.m3:319<br>
>>>> #6 =3DC2=3DA00x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3D3DErr=3D
>>>> or accessing memory address 0x8000feff7eb8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449<br>
>>>> #7 =3DC2=3DA00x0000000000449c50 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3D
>>>> =3D3DError accessing memory address 0x8000feff7f68: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422<br>
>>>> #8 =3DC2=3DA00x000000000047c7d4 in thread_start ()<br>
>>>> #9 =3DC2=3DA00x0000000000000000 in ?? ()<br>
>>>> <br>
>>>> Thread 6 (Thread 800c09800 (LWP 100470/threadtest)):<br>
>>>> #0 =3DC2=3DA00x000000000047e53c in _umtx_op_err ()<br>
>>>> #1 =3DC2=3DA00x0000000000475759 in suspend_common ()<br>
>>>> #2 =3DC2=3DA00x00000000004755c1 in pthread_suspend_np ()<br>
>>>> #3 =3DC2=3DA00x000000000044df0c in ThreadPThread__SuspendThread =
>> (mt=3D3DError acc=3D
>>>> essing memory address 0x8000ffbfd6f8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadFreeBSD.c:33<br>
>>>> #4 =3DC2=3DA00x000000000044bade in ThreadPThread__StopThread =
>> (M3_DMxDjQ_act=3D3DE=3D
>>>> rror accessing memory address 0x8000ffbfd718: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:909<br>
>>>> #5 =3DC2=3DA00x000000000044bc81 in ThreadPThread__StopWorld () at =
>> ../src/thread=3D
>>>> /PTHREAD/ThreadPThread.m3:948<br>
>>>> #6 =3DC2=3DA00x000000000044b19e in RTThread__SuspendOthers () at =
>> ../src/thread/=3D
>>>> PTHREAD/ThreadPThread.m3:713<br>
>>>> #7 =3DC2=3DA00x00000000004330d6 in =
>> RTCollector__CollectSomeInStateZero () at ..=3D
>>>> /src/runtime/common/RTCollector.m3:749<br>
>>>> #8 =3DC2=3DA00x0000000000433081 in RTCollector__CollectSome () at =
>> ../src/runtim=3D
>>>> e/common/RTCollector.m3:723<br>
>>>> #9 =3DC2=3DA00x0000000000432d49 in RTHeapRep__CollectEnough () at =
>> ../src/runtim=3D
>>>> e/common/RTCollector.m3:657<br>
>>>> #10 0x000000000042ff83 in RTAllocator__AllocTraced =
>> (M3_Cwb5VA_dataSize=3D3DEr=3D
>>>> ror accessing memory address 0x8000ffbfd958: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:367<br>
>>>> #11 0x000000000042f8a5 in RTAllocator__GetOpenArray =
>> (M3_Eic7CK_def=3D3DError =3D
>>>> accessing memory address 0x8000ffbfda18: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:296<br>
>>>> #12 0x000000000042ebdf in RTHooks__AllocateOpenArray =
>> (M3_AJWxb1_defn=3D3DErro=3D
>>>> r accessing memory address 0x8000ffbfda78: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:143<br>
>>>> #13 0x000000000040ca4e in Rd__NextBuff (M3_EkTcCb_rd=3D3DError =
>> accessing memo=3D
>>>> ry address 0x8000ffbfdab8: Bad address.<br>
>>>> ) at ../src/rw/Rd.m3:159<br>
>>>> #14 0x000000000040cd43 in UnsafeRd__FastGetChar (M3_EkTcCb_rd=3D3DError=
>> acces=3D
>>>> sing memory address 0x8000ffbfdb88: Bad address.<br>
>>>> ) at ../src/rw/Rd.m3:187<br>
>>>> #15 0x000000000040cc4a in Rd__GetChar (M3_EkTcCb_rd=3D3DError =
>> accessing memor=3D
>>>> y address 0x8000ffbfdbd8: Bad address.<br>
>>>> ) at ../src/rw/Rd.m3:176<br>
>>>> #16 0x000000000040095c in Main__RApply (M3_AP7a1g_cl=3D3DError =
>> accessing memo=3D
>>>> ry address 0x8000ffbfdc58: Bad address.<br>
>>>> ) at ../src/Main.m3:185<br>
>>>> <br>
>>>> #17 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3D3DError ac=3D
>>>> cessing memory address 0x8000ffbfdeb8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449<br>
>>>> #18 0x0000000000449c50 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3D3DErro=3D
>>>> r accessing memory address 0x8000ffbfdf68: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422<br>
>>>> #19 0x000000000047c7d4 in thread_start ()<br>
>>>> #20 0x0000000000000000 in ?? ()<br>
>>>> <br>
>>>> Thread 5 (Thread 800c09400 (LWP 101035/threadtest)):<br>
>>>> #0 =3DC2=3DA00x000000000047e53c in _umtx_op_err ()<br>
>>>> #1 =3DC2=3DA00x0000000000475f14 in __thr_umutex_lock ()<br>
>>>> #2 =3DC2=3DA00x0000000000479404 in mutex_lock_common ()<br>
>>>> #3 =3DC2=3DA00x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3D3DE=3D
>>>> rror accessing memory address 0x8000ffffc678: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506<br>
>>>> #4 =3DC2=3DA00x000000000044d039 in RTOS__LockHeap () at =
>> ../src/thread/PTHREAD/T=3D
>>>> hreadPThread.m3:1337<br>
>>>> ---Type <return> to continue, or q <return> to =
>> quit---<br>
>>>> #5 =3DC2=3DA00x000000000042ff79 in RTAllocator__AllocTraced =
>> (M3_Cwb5VA_dataSize=3D
>>>> =3D3DError accessing memory address 0x8000ffffc6e8: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:365<br>
>>>> #6 =3DC2=3DA00x000000000042f15b in RTAllocator__GetTracedObj =
>> (M3_Eic7CK_def=3D3DE=3D
>>>> rror accessing memory address 0x8000ffffc7a8: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:224<br>
>>>> #7 =3DC2=3DA00x000000000042eb23 in RTHooks__AllocateTracedObj =
>> (M3_AJWxb1_defn=3D
>>>> =3D3DError accessing memory address 0x8000ffffc7f8: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:122<br>
>>>> #8 =3DC2=3DA00x000000000045fbe4 in TextCat__Flat =
>> (M3_Bd56fi_LText=3D3DError acces=3D
>>>> sing memory address 0x8000ffffc858: Bad address.<br>
>>>> ) at ../src/text/TextCat.m3:562<br>
>>>> #9 =3DC2=3DA00x000000000045ed5d in TextCat__Balance =
>> (M3_Bd56fi_LText=3D3D0x800c49=3D
>>>> 0b0, M3_BUgnwf_LInfo=3D3DError accessing memory address =
>> 0x8000ffffc8f8: Bad a=3D
>>>> ddress.<br>
>>>> ) at ../src/text/TextCat.m3:488<br>
>>>> #10 0x000000000045d64f in RTHooks__Concat (M3_Bd56fi_t=3D3DError =
>> accessing me=3D
>>>> mory address 0x8000ffffcbb8: Bad address.<br>
>>>> ) at ../src/text/TextCat.m3:40<br>
>>>> #11 0x000000000040639e in Main_M3 (M3_AcxOUs_mode=3D3DError accessing =
>> memory =3D
>>>> address 0x8000ffffcc38: Bad address.<br>
>>>> ) at ../src/Main.m3:593<br>
>>>> #12 0x000000000043d55d in RTLinker__RunMainBody (M3_DjPxE3_m=3D3DError =
>> access=3D
>>>> ing memory address 0x8000ffffcf88: Bad address.<br>
>>>> ) at ../src/runtime/common/RTLinker.m3:408<br>
>>>> #13 0x000000000043c8e8 in RTLinker__AddUnitI (M3_DjPxE3_m=3D3DError =
>> accessing=3D
>>>> memory address 0x8000ffffd008: Bad address.<br>
>>>> ) at ../src/runtime/common/RTLinker.m3:115<br>
>>>> #14 0x000000000043c97c in RTLinker__AddUnit (M3_DjPxE5_b=3D3DError =
>> accessing =3D
>>>> memory address 0x8000ffffd028: Bad address.<br>
>>>> ) at ../src/runtime/common/RTLinker.m3:124<br>
>>>> #15 0x00000000004004a6 in main (argc=3D3DError accessing memory =
>> address 0x800=3D
>>>> 0ffffd07c: Bad address.<br>
>>>> ) at _m3main.c:22<br>
>>>> <br>
>>>> Thread 9 (Thread 800c0a400 (LWP 100473/threadtest)):<br>
>>>> #0 =3DC2=3DA00x000000000047e53c in _umtx_op_err ()<br>
>>>> #1 =3DC2=3DA00x0000000000477e8a in check_suspend ()<br>
>>>> #2 =3DC2=3DA00x00000000004780a2 in sigcancel_handler ()<br>
>>>> #3 =3DC2=3DA0<signal handler called><br>
>>>> #4 =3DC2=3DA00x000000000047e53c in _umtx_op_err ()<br>
>>>> #5 =3DC2=3DA00x0000000000475f14 in __thr_umutex_lock ()<br>
>>>> #6 =3DC2=3DA00x0000000000479404 in mutex_lock_common ()<br>
>>>> #7 =3DC2=3DA00x000000000044dd15 in ThreadPThread__pthread_mutex_lock =
>> (mutex=3D3DE=3D
>>>> rror accessing memory address 0x8000ff5fac18: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:506<br>
>>>> #8 =3DC2=3DA00x000000000044d039 in RTOS__LockHeap () at =
>> ../src/thread/PTHREAD/T=3D
>>>> hreadPThread.m3:1337<br>
>>>> #9 =3DC2=3DA00x000000000042ff79 in RTAllocator__AllocTraced =
>> (M3_Cwb5VA_dataSize=3D
>>>> =3D3DError accessing memory address 0x8000ff5fac88: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:365<br>
>>>> #10 0x000000000042f8a5 in RTAllocator__GetOpenArray =
>> (M3_Eic7CK_def=3D3DError =3D
>>>> accessing memory address 0x8000ff5fad48: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:296<br>
>>>> #11 0x000000000042ebdf in RTHooks__AllocateOpenArray =
>> (M3_AJWxb1_defn=3D3DErro=3D
>>>> r accessing memory address 0x8000ff5fada8: Bad address.<br>
>>>> ) at ../src/runtime/common/RTAllocator.m3:143<br>
>>>> #12 0x000000000040203a in Main__AApply (M3_AP7a1g_cl=3D3DError =
>> accessing memo=3D
>>>> ry address 0x8000ff5fade8: Bad address.<br>
>>>> ) at ../src/Main.m3:283<br>
>>>> #13 0x0000000000449f93 in ThreadPThread__RunThread =
>> (M3_DMxDjQ_me=3D3DError ac=3D
>>>> cessing memory address 0x8000ff5faeb8: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:449<br>
>>>> #14 0x0000000000449c50 in ThreadPThread__ThreadBase =
>> (M3_AJWxb1_param=3D3DErro=3D
>>>> r accessing memory address 0x8000ff5faf68: Bad address.<br>
>>>> ) at ../src/thread/PTHREAD/ThreadPThread.m3:422<br>
>>>> #15 0x000000000047c7d4 in thread_start ()<br>
>>>> #16 0x0000000000000000 in ?? ()<br>
>>>> <br>
>>>> (others are similar)<br>
>>>> <br>
>>>> Hmm looks like a FreeBSD issue now. =3DC2=3DA0It's here...<br>
>>>> <br>
>>>> int<br>
>>>> __cdecl<br>
>>>> ThreadPThread__SuspendThread (m3_pthread_t mt)<br>
>>>> {<br>
>>>> =3DC2=3DA0 =
>> ThreadFreeBSD__Fatal(pthread_suspend_np(PTHREAD_FROM_M3(mt)), "=3D
>>>> pthread_suspend_np");<br>
>>>> =3DC2=3DA0 return 1;<br>
>>>> }<br>
>>>> <br>
>>>> Now this suspend can wait:<br>
>>>> <br>
>>>> static int<br>
>>>> suspend_common(struct pthread *curthread, struct pthread *thread,<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 int waitok)<br>
>>>> {<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 uint32_t tmp;<br>
>>>> <br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 while (thread->state =
>> !=3D3D PS_DEAD &&=3D
>>>> <br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 !(thread->flags & T=3D
>>>> HR_FLAGS_SUSPENDED)) {<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 thread->flags |=3D
>>>> =3D3D THR_FLAGS_NEED_SUSPEND;<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 /* Thread is in cre=3D
>>>> ation. */<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 if (thread->tid =3D
>>>> =3D3D=3D3D TID_TERMINATED)<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3D
>>>> =3DA0 =3DC2=3DA0 return (1);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 tmp =3D3D thread->=3D
>>>> cycle;<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 _thr_send_sig(threa=3D
>>>> d, SIGCANCEL);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 THR_THREAD_UNLOCK(c=3D
>>>> urthread, thread);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 if (waitok) {<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3D
>>>> =3DA0 =3DC2=3DA0 _thr_umtx_wait_uint(&thread->cycle, tmp, =
>> NULL, 0); =3DC2=3DA0=3D
>>>> <=3D3D=3D3D=3D3D=3D3D=3D3D=3D3D=3D3D=3D3D=3D3D=3D3D<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3D
>>>> =3DA0 =3DC2=3DA0 THR_THREAD_LOCK(curthread, thread);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 } else {<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3D
>>>> =3DA0 =3DC2=3DA0 THR_THREAD_LOCK(curthread, thread);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3D
>>>> =3DA0 =3DC2=3DA0 return (0);<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> =3DC2=3DA0 =3DC2=3DA0 }<br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 }<br>
>>>> <br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 return (1);<br>
>>>> }<br>
>>>> <br>
>>>> ... but what it can wait for I am not clear on.<br>
>>>> <br>
>>>> Do things work better on Linux?<br>
>>>> <br>
>>>> What is the status of the fork bug? =3DC2=3DA0Can it be worked around =
>> by only f=3D
>>>> orking via Process.Create with no mutexes held (outside of all LOCK =
>> blocks)=3D
>>>> ?<br>
>>>> <br>
>>>> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0Mika<br>
>>>> <br>
>>>> <br>
>>>> <br>
>>>> Peter McKinna writes:<br>
>>>> >--001a11c2ced4471a2e050079db82<br>
>>>> >Content-Type: text/plain; charset=3D3DUTF-8<br>
>>>> <div><div>><br>
>>>> >Mika<br>
>>>> ><br>
>>>> > =3DC2=3DA0I think you need to back out Tony's changes to fix =
>> the fork =3D
>>>> bug, at least<br>
>>>> >for now. Need to reload from cvs version 1.262 for =
>> ThreadPThread.m3 If<=3D
>>>> br>
>>>> >you're using git you're on your own.<br>
>>>> > =3DC2=3DA0Do a cvs log ThreadPThread.m3 for an explanation for =
>> some of the=3D
>>>> design<br>
>>>> >principles.<br>
>>>> > =3DC2=3DA0Also if you use gdb then you need to set lanc c before =
>> backtrace=3D
>>>> s so at<br>
>>>> >least you can see address names and values even if they are =
>> contorted y=3D
>>>> ou<br>
>>>> >can extract the M3 name in the parm lists. Also in gdb thread =
>> apply all=3D
>>>> bt<br>
>>>> >gives all thread backtraces which can be handy to see whose got =
>> the loc=3D
>>>> ks<br>
>>>> >held.<br>
>>>> ><br>
>>>> >Regards Peter<br>
>>>> ><br>
>>>> ><br>
>>>> ><br>
>>>> >On Wed, Aug 13, 2014 at 12:14 PM, <<a =
>> href=3D3D"mailto:mika at async.calt=3D
>>>> ech.edu" target=3D3D"_blank">mika at async.caltech.edu</a>> =
>> wrote:<br>
>>>> ><br>
>>>> >><br>
>>>> >> Question... is there something odd about my pthreads? =
>> =3DC2=3DA0Are pt=3D
>>>> hreads<br>
>>>> >> normally reentrant? =3DC2=3DA0I didn't think so.<br>
>>>> >><br>
>>>> >> My compiler is much happier with the following changes I =
>> already o=3D
>>>> utlined:<br>
>>>> >><br>
>>>> >> 1. a dirty, disgusting hack to keep from locking against =
>> myself go=3D
>>>> ing from<br>
>>>> >> XWait with self.mutex locked to m.release().<br>
>>>> >><br>
>>>> >> 2. the change to LockHeap I described in previous email<br>
>>>> >><br>
>>>> >> But now...<br>
>>>> >><br>
>>>> >> (gdb) cont<br>
>>>> >> Continuing.<br>
>>>> >> ERROR: pthread_mutex_lock:11<br>
>>>> >> ERROR: pthread_mutex_lock:11<br>
>>>> >><br>
>>>> >> Program received signal SIGABRT, Aborted.<br>
>>>> >> 0x000000080107626a in thr_kill () from /lib/libc.so.7<br>
>>>> >> (gdb) where<br>
>>>> >> #0 =3DC2=3DA00x000000080107626a in thr_kill () from =
>> /lib/libc.so.7<br>
>>>> >> #1 =3DC2=3DA00x000000080113dac9 in abort () from =
>> /lib/libc.so.7<br>
>>>> >> #2 =3DC2=3DA00x000000000071e37a in =
>> ThreadPThread__pthread_mutex_lock (=3D
>>>> mutex=3D3DError<br>
>>>> >> accessing memory address 0x8000ffffb508: Bad address.<br>
>>>> >> ) at ../src/thread/PTHREAD/ThreadPThreadC.c:543<br>
>>>> >> #3 =3DC2=3DA00x000000000071d48d in RTOS__LockHeap () at<br>
>>>> >> ../src/thread/PTHREAD/ThreadPThread.m3:1377<br>
>>>> >> #4 =3DC2=3DA00x0000000000706b9d in =
>> RTHooks__CheckLoadTracedRef (M3_Af4=3D
>>>> 0ku_ref=3D3DError<br>
>>>> >> accessing memory address 0x8000ffffb568: Bad address.<br>
>>>> >> ) at ../src/runtime/common/RTCollector.m3:2234<br>
>>>> >> #5 =3DC2=3DA00x000000000071d284 in =
>> ThreadPThread__AtForkParent () at<b=3D
>>>> r>
>>>> >> ../src/thread/PTHREAD/ThreadPThread.m3:1348<br>
>>>> >> #6 =3DC2=3DA00x0000000800df8733 in fork () from =
>> /lib/libthr.so.3<br>
>>>> >> #7 =3DC2=3DA00x000000000070dd8b in RTProcess__Fork () at<br>
>>>> >> ../src/runtime/common/RTProcessC.c:152<br>
>>>> >> #8 =3DC2=3DA00x00000000006c52f2 in =
>> ProcessPosixCommon__Create_ForkExec=3D
>>>> <br>
>>>> >> (M3_Bd56fi_cmd=3D3DError accessing memory address =
>> 0x8000ffffb6f8: Ba=3D
>>>> d address.<br>
>>>> >> ) at ../src/os/POSIX/ProcessPosixCommon.m3:75<br>
>>>> >> #9 =3DC2=3DA00x00000000006c6c6c in Process__Create =
>> (M3_Bd56fi_cmd=3D3DEr=3D
>>>> ror accessing<br>
>>>> >> memory address 0x8000ffffb7f8: Bad address.<br>
>>>> >> ) at ../src/os/POSIX/ProcessPosix.m3:21<br>
>>>> >> #10 0x00000000004d6826 in QMachine__FulfilExecPromise =
>> (M3_D6rRrg_e=3D
>>>> p=3D3DError<br>
>>>> >> accessing memory address 0x8000ffffb838: Bad address.<br>
>>>> >> ) at ../src/QMachine.m3:1666<br>
>>>> >> #11 0x00000000004d6220 in QMachine__ExecCommand =
>> (M3_An02H2_t=3D3DErr=3D
>>>> or<br>
>>>> >> accessing memory address 0x8000ffffb9f8: Bad address.<br>
>>>> >> ) at ../src/QMachine.m3:1605<br>
>>>> >> #12 0x00000000004d537e in QMachine__DoTryExec =
>> (M3_An02H2_t=3D3DError=3D
>>>> accessing<br>
>>>> >> memory address 0x8000ffffbee8: Bad address.<br>
>>>> >> ) at ../src/QMachine.m3:1476<br>
>>>> >><br>
>>>> >> What am I doing wrong here?<br>
>>>> >><br>
>>>> >> The error doesn't look unreasonable! =3DC2=3DA0Looking =
>> more closel=3D
>>>> y at the code:<br>
>>>> >><br>
>>>> >> First, AtForkPrepare has been called:<br>
>>>> >><br>
>>>> >> PROCEDURE AtForkPrepare() =3D3D<br>
>>>> >> =3DC2=3DA0 VAR me :=3D3D GetActivation();<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act: Activation;<br>
>>>> >> =3DC2=3DA0 BEGIN<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(slotsMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(perfMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(initMu, ThisLine()); =
>> (* InitMutex =3D
>>>> =3D3D><br>
>>>> >> RegisterFinalCleanup =3D3D> LockHeap *)<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(heapMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(activeMu, =
>> ThisLine()); (* LockHeap =3D
>>>> =3D3D> SuspendOthers<br>
>>>> >> =3D3D> activeMu *)<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 (* Walk activations and lock all =
>> threads.<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0* NOTE: We have initMu, =
>> activeMu, so slots won=3D
>>>> 't change, conditions<br>
>>>> >> and<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0* mutexes won't be =
>> initialized on-demand.<=3D
>>>> br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0*)<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 act :=3D3D me;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 REPEAT<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 PThreadLockMutex(act.mutex, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act :=3D3D act.next;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 UNTIL act =3D3D me;<br>
>>>> >> =3DC2=3DA0 END AtForkPrepare;<br>
>>>> >><br>
>>>> >> a postcondition of this routine is that heapMu is =
>> locked.<br>
>>>> >><br>
>>>> >> now we get into AtForkParent:<br>
>>>> >><br>
>>>> >> PROCEDURE AtForkParent() =3D3D<br>
>>>> >> =3DC2=3DA0 VAR me :=3D3D GetActivation();<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act: Activation;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 cond: Condition;<br>
>>>> >> =3DC2=3DA0 BEGIN<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 (* Walk activations and unlock all =
>> threads, conditio=3D
>>>> ns. *)<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 act :=3D3D me;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 REPEAT<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 cond :=3D3D =
>> slots[act.slot].join;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 IF cond # NIL THEN =
>> PThreadUnlockMutex(cond.mu=3D
>>>> tex, ThisLine()) END;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 =
>> PThreadUnlockMutex(act.mutex, ThisLine());<br=3D
>>>>> =20
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0 act :=3D3D act.next;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 UNTIL act =3D3D me;<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(activeMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(heapMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(initMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(perfMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 PThreadUnlockMutex(slotsMu, =
>> ThisLine());<br>
>>>> >> =3DC2=3DA0 END AtForkParent;<br>
>>>> >><br>
>>>> >> We can see by inspecting the code that a necessary =
>> precondition fo=3D
>>>> r<br>
>>>> >> this routine is that heapMu is locked! =3DC2=3DA0(Since =
>> it's going=3D
>>>> to unlock it,<br>
>>>> >> it had BETTER be locked on entry.)<br>
>>>> >><br>
>>>> >> But the cond :=3D3D ... causes a =
>> RTHooks.CheckLoadTracedRef<br>
>>>> >><br>
>>>> >> which causes an RTOS.LockHeap<br>
>>>> >><br>
>>>> >> the code of which we just saw:<br>
>>>> >><br>
>>>> >> PROCEDURE LockHeap () =3D3D<br>
>>>> >> =3DC2=3DA0 VAR self :=3D3D pthread_self();<br>
>>>> >> =3DC2=3DA0 BEGIN<br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 WITH r =3D3D =
>> pthread_mutex_lock(heapMu,ThisLine()) DO =3D
>>>> <*ASSERT r=3D3D0*> END;<br>
>>>> >> ...<br>
>>>> >><br>
>>>> >> we try to lock heapMu. =3DC2=3DA0kaboom! =3DC2=3DA0No =
>> surprise there, real=3D
>>>> ly?<br>
>>>> >><br>
>>>> >> Am I going about this totally the wrong way? =3DC2=3DA0Other =
>> people ar=3D
>>>> e running<br>
>>>> >> Modula-3<br>
>>>> >> with pthreads, right? =3DC2=3DA0Right?? =3DC2=3DA0Somewhere =
>> out there in m=3D
>>>> 3devel-land?<br>
>>>> >><br>
>>>> >> =3DC2=3DA0 =3DC2=3DA0 =3DC2=3DA0Mika<br>
>>>> >><br>
>>>> >><br>
>>>> ><br>
>>>> </div></div>>--001a11c2ced4471a2e050079db82<br>
>>>> >Content-Type: text/html; charset=3D3DUTF-8<br>
>>>> >Content-Transfer-Encoding: quoted-printable<br>
>>>> ><br>
>>>> ><div =
>> dir=3D3D3D"ltr">Mika<div><br></div>=3D
>>>> ;<div>=3D3DC2=3D3DA0 I think you need to back ou=3D3D<br>
>>>> >t Tony's changes to fix the fork bug, at least for now. =
>> Need to=3D
>>>> reload =3D3D<br>
>>>> >from cvs version 1.262 for ThreadPThread.m3 If you're =
>> using git=3D
>>>> you'=3D3D<br>
>>>> >;re on your own.</div><br>
>>>> ><br>
>>>> ><div>=3D3DC2=3D3DA0 Do a cvs log ThreadPThread.m3 for an =
>> explanation =3D
>>>> for some of th=3D3D<br>
>>>> >e design principles.=3D3DC2=3D3DA0</div><div>=3D3DC2=3D=
>> 3DA0 Also if=3D
>>>> you use gdb then you ne=3D3D<br>
>>>> >ed to set lanc c before backtraces so at least you can see =
>> address name=3D
>>>> s an=3D3D<br>
>>>> >d values even if they are contorted you can extract the M3 name =
>> in the =3D
>>>> parm=3D3D<br>
>>>> > lists. Also in gdb thread apply all bt gives all thread =
>> backtraces whi=3D
>>>> ch c=3D3D<br>
>>>> >an be handy to see whose got the locks held.</div><br>
>>>> ><br>
>>>> ><div><br></div><div>Regards =
>> Peter</div>&l=3D
>>>> t;div><br></div><div class=3D3D3D"gmail_e=3D3D<b=
>> r>
>>>> >xtra"><br><br><div =
>> class=3D3D3D"gmail_quote&q=3D
>>>> uot;>On Wed, Aug 13, 2014 at 12:14 PM, =3D3D<br>
>>>> > <span dir=3D3D3D"ltr"><<a =
>> href=3D3D3D"mailt=3D
>>>> o:<a href=3D3D"mailto:mika at async.caltech.edu" =
>> target=3D3D"_blank">mika at async.ca=3D
>>>> ltech.edu</a>" target=3D3D3D"=3D3D<br>
>>>> >_blank"><a href=3D3D"mailto:mika at async.caltech.edu" =
>> target=3D3D"_bl=3D
>>>> ank">mika at async.caltech.edu</a></a>></span> =
>> wrote:<br=3D
>>>> ><br>
>>>> ><br>
>>>> ><blockquote class=3D3D3D"gmail_quote" =
>> style=3D3D3D"margin=3D
>>>> :0 0 0 .8ex;border-left:1p=3D3D<br>
>>>> >x #ccc solid;padding-left:1ex"><br><br>
>>>> >Question... is there something odd about my pthreads? =
>> =3D3DC2=3D3DA0Are pth=3D
>>>> reads no=3D3D<br>
>>>> >rmally reentrant? =3D3DC2=3D3DA0I didn't think =
>> so.<br><br>
>>>> ><br><br>
>>>> >My compiler is much happier with the following changes I already =
>> outlin=3D
>>>> ed:<=3D3D<br>
>>>> >br><br>
>>>> ><br><br>
>>>> >1. a dirty, disgusting hack to keep from locking against myself =
>> going f=3D
>>>> rom =3D3D<br>
>>>> >XWait with self.mutex locked to m.release().<br><br>
>>>> ><br><br>
>>>> >2. the change to LockHeap I described in previous =
>> email<br><br>
>>>> ><br><br>
>>>> >But now...<br><br>
>>>> ><br><br>
>>>> >(gdb) cont<br><br>
>>>> >Continuing.<br><br>
>>>> >ERROR: pthread_mutex_lock:11<br><br>
>>>> >ERROR: pthread_mutex_lock:11<br><br>
>>>> ><br><br>
>>>> >Program received signal SIGABRT, Aborted.<br><br>
>>>> >0x000000080107626a in thr_kill () from =
>> /lib/libc.so.7<br><br>
>>>> >(gdb) where<br><br>
>>>> >#0 =3D3DC2=3D3DA00x000000080107626a in thr_kill () from =
>> /lib/libc.so.7<b=3D
>>>> r><br>
>>>> >#1 =3D3DC2=3D3DA00x000000080113dac9 in abort () from =
>> /lib/libc.so.7<br&g=3D
>>>> t;<br>
>>>> >#2 =3D3DC2=3D3DA00x000000000071e37a in =
>> ThreadPThread__pthread_mutex_lock (m=3D
>>>> utex=3D3D3DE=3D3D<br>
>>>> >rror accessing memory address 0x8000ffffb508: Bad =
>> address.<br><br=3D
>>>>> =20
>>>> >) at ../src/thread/PTHREAD/ThreadPThreadC.c:543<br><br>
>>>> >#3 =3D3DC2=3D3DA00x000000000071d48d in RTOS__LockHeap () at =
>> ../src/thread/P=3D
>>>> THREAD/T=3D3D<br>
>>>> >hreadPThread.m3:1377<br><br>
>>>> >#4 =3D3DC2=3D3DA00x0000000000706b9d in =
>> RTHooks__CheckLoadTracedRef (M3_Af40=3D
>>>> ku_ref=3D3D<br>
>>>> >=3D3D3DError accessing memory address 0x8000ffffb568: Bad =
>> address.<br&=3D
>>>> gt;<br>
>>>> >) at ../src/runtime/common/RTCollector.m3:2234<br><br>
>>>> >#5 =3D3DC2=3D3DA00x000000000071d284 in =
>> ThreadPThread__AtForkParent () at ..=3D
>>>> /src/thr=3D3D<br>
>>>> >ead/PTHREAD/ThreadPThread.m3:1348<br><br>
>>>> >#6 =3D3DC2=3D3DA00x0000000800df8733 in fork () from =
>> /lib/libthr.so.3<br&=3D
>>>> gt;<br>
>>>> >#7 =3D3DC2=3D3DA00x000000000070dd8b in RTProcess__Fork () at =
>> ../src/runtime=3D
>>>> /common/=3D3D<br>
>>>> >RTProcessC.c:152<br><br>
>>>> >#8 =3D3DC2=3D3DA00x00000000006c52f2 in =
>> ProcessPosixCommon__Create_ForkExec =3D
>>>> (M3_Bd56=3D3D<br>
>>>> >fi_cmd=3D3D3DError accessing memory address 0x8000ffffb6f8: Bad =
>> address.&=3D
>>>> lt;br><br>
>>>> >) at ../src/os/POSIX/ProcessPosixCommon.m3:75<br><br>
>>>> >#9 =3D3DC2=3D3DA00x00000000006c6c6c in Process__Create =
>> (M3_Bd56fi_cmd=3D3D3DE=3D
>>>> rror acces=3D3D<br>
>>>> >sing memory address 0x8000ffffb7f8: Bad address.<br><br>
>>>> >) at ../src/os/POSIX/ProcessPosix.m3:21<br><br>
>>>> >#10 0x00000000004d6826 in QMachine__FulfilExecPromise =
>> (M3_D6rRrg_ep=3D3D3=3D
>>>> DError=3D3D<br>
>>>> > accessing memory address 0x8000ffffb838: Bad =
>> address.<br><br>
>>>> >) at ../src/QMachine.m3:1666<br><br>
>>>> >#11 0x00000000004d6220 in QMachine__ExecCommand =
>> (M3_An02H2_t=3D3D3DError =3D
>>>> access=3D3D<br>
>>>> >ing memory address 0x8000ffffb9f8: Bad address.<br><br>
>>>> >) at ../src/QMachine.m3:1605<br><br>
>>>> >#12 0x00000000004d537e in QMachine__DoTryExec =
>> (M3_An02H2_t=3D3D3DError ac=3D
>>>> cessin=3D3D<br>
>>>> >g memory address 0x8000ffffbee8: Bad address.<br><br>
>>>> >) at ../src/QMachine.m3:1476<br><br>
>>>> ><br><br>
>>>> >What am I doing wrong here?<br><br>
>>>> ><br><br>
>>>> >The error doesn't look unreasonable! =3D3DC2=3D3DA0Looking =
>> more clo=3D
>>>> sely at the =3D3D<br>
>>>> >code:<br><br>
>>>> ><br><br>
>>>> >First, AtForkPrepare has been called:<br><br>
>>>> ><br><br>
>>>> >PROCEDURE AtForkPrepare() =3D3D3D<br><br>
>>>> >=3D3DC2=3D3DA0 VAR me :=3D3D3D GetActivation();<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act: =
>> Activation;<br><br>
>>>> >=3D3DC2=3D3DA0 BEGIN<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadLockMutex(slotsMu, =
>> ThisLine());<br><=3D
>>>> br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadLockMutex(perfMu, =
>> ThisLine());<br><b=3D
>>>> r>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadLockMutex(initMu, =
>> ThisLine()); (* InitMute=3D
>>>> x =3D3D3D> Re=3D3D<br>
>>>> >gisterFinalCleanup =3D3D3D> LockHeap *)<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadLockMutex(heapMu, =
>> ThisLine());<br><b=3D
>>>> r>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadLockMutex(activeMu, =
>> ThisLine()); (* LockHe=3D
>>>> ap =3D3D3D> S=3D3D<br>
>>>> >uspendOthers =3D3D3D> activeMu *)<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 (* Walk activations and lock all =
>> threads.<br&g=3D
>>>> t;<br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0* NOTE: We have =
>> initMu, activeMu, so sl=3D
>>>> ots won't ch=3D3D<br>
>>>> >ange, conditions and<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0* mutexes =
>> won't be initialized =3D
>>>> on-demand.<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0*)<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act :=3D3D3D me;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 REPEAT<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =
>> PThreadLockMutex(act.mutex, ThisLine()=3D
>>>> );<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act :=3D3D3D =
>> act.next;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 UNTIL act =3D3D3D me;<br><br>
>>>> >=3D3DC2=3D3DA0 END AtForkPrepare;<br><br>
>>>> ><br><br>
>>>> >a postcondition of this routine is that heapMu is =
>> locked.<br><br>
>>>> ><br><br>
>>>> >now we get into AtForkParent:<br><br>
>>>> ><br><br>
>>>> >PROCEDURE AtForkParent() =3D3D3D<br><br>
>>>> >=3D3DC2=3D3DA0 VAR me :=3D3D3D GetActivation();<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act: =
>> Activation;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 cond: =
>> Condition;<br><br>
>>>> >=3D3DC2=3D3DA0 BEGIN<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 (* Walk activations and unlock all =
>> threads, condi=3D
>>>> tions. *)<br=3D3D<br>
>>>> >><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act :=3D3D3D me;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 REPEAT<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 cond :=3D3D3D =
>> slots[act.slot].join;<b=3D
>>>> r><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 IF cond # NIL THEN =
>> PThreadUnlockMutex(=3D
>>>> cond.mutex, This=3D3D<br>
>>>> >Line()) END;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =
>> PThreadUnlockMutex(act.mutex, ThisLine=3D
>>>> ());<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0 act :=3D3D3D =
>> act.next;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 UNTIL act =3D3D3D me;<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadUnlockMutex(activeMu, =
>> ThisLine());<br&g=3D
>>>> t;<br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadUnlockMutex(heapMu, =
>> ThisLine());<br>=3D
>>>> <br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadUnlockMutex(initMu, =
>> ThisLine());<br>=3D
>>>> <br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadUnlockMutex(perfMu, =
>> ThisLine());<br>=3D
>>>> <br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 PThreadUnlockMutex(slotsMu, =
>> ThisLine());<br>=3D
>>>> ;<br>
>>>> >=3D3DC2=3D3DA0 END AtForkParent;<br><br>
>>>> ><br><br>
>>>> >We can see by inspecting the code that a necessary precondition =
>> for<=3D
>>>> br><br>
>>>> >this routine is that heapMu is locked! =3D3DC2=3D3DA0(Since =
>> it's go=3D
>>>> ing to unloc=3D3D<br>
>>>> >k it,<br><br>
>>>> >it had BETTER be locked on entry.)<br><br>
>>>> ><br><br>
>>>> >But the cond :=3D3D3D ... causes a =
>> RTHooks.CheckLoadTracedRef<br><b=3D
>>>> r>
>>>> ><br><br>
>>>> >which causes an RTOS.LockHeap<br><br>
>>>> ><br><br>
>>>> >the code of which we just saw:<br><br>
>>>> ><br><br>
>>>> >PROCEDURE LockHeap () =3D3D3D<br><br>
>>>> >=3D3DC2=3D3DA0 VAR self :=3D3D3D pthread_self();<br><br>
>>>> >=3D3DC2=3D3DA0 BEGIN<br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 WITH r =3D3D3D =
>> pthread_mutex_lock(heapMu,ThisLine()=3D
>>>> ) DO <*ASSE=3D3D<br>
>>>> >RT r=3D3D3D0*> END;<br><br>
>>>> >...<br><br>
>>>> ><br><br>
>>>> >we try to lock heapMu. =3D3DC2=3D3DA0kaboom! =3D3DC2=3D3DA0No =
>> surprise there, r=3D
>>>> eally?<br><br>
>>>> ><br><br>
>>>> >Am I going about this totally the wrong way? =3D3DC2=3D3DA0Other =
>> people are=3D
>>>> running=3D3D<br>
>>>> > Modula-3<br><br>
>>>> >with pthreads, right? =3D3DC2=3D3DA0Right?? =3D3DC2=3D3DA0Somewhere=
>> out there i=3D
>>>> n m3devel-la=3D3D<br>
>>>> >nd?<br><br>
>>>> ><span><font =
>> color=3D3D3D"#888888"><br><br>
>>>> >=3D3DC2=3D3DA0 =3D3DC2=3D3DA0 =3D3DC2=3D3DA0Mika<br><br>
>>>> ><br><br>
>>>> =
>> ></font></span></blockquote></div><br><=
>> ;=3D
>>>> /div></div><br>
>>>> ><br>
>>>> >--001a11c2ced4471a2e050079db82--<br>
>>>> </blockquote></div><br></div>
>>>> </div></div></blockquote></div><br></div>
>>>> =20
>>>> --089e01183a1813188705007c3498--
More information about the M3devel
mailing list