[M3devel] deadlock... in ThreadPThread.m3?

Mika Nystrom mika at async.caltech.edu
Fri Jan 7 16:14:11 CET 2011 

Hi m3devel (especially Tony), 

I'm having a problem with a Modula-3 program that seems to deadlock
with one of my 60+ threads stuck in Process.Create.  Is it possible
there is a bug in ThreadPThread.m3?  (Would be really nice to have a
deadlock detector here, you know... rather than have the program go
catatonic when there's a deadlock.  Deadlock is stable so the detector
could run very infrequently---i.e., be very inefficient---and still be
extremely useful....)

I don't think there is anything I ought to be able to do from "user
code" that would cause Process.Create to be part of a deadlock cycle.
Or could a "user code" deadlock cycle (cycle of Modula-3 MUTEXes)
possibly exhibit itself as a failure of Process.Create to complete??

Below are backtraces of the threads stuck in some sort of locking
activity, platform is I386_DARWIN at head as of a couple of weeks ago.
(I deleted the threads that are up to other things than locking.)

     Mika

Thread 68 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613c20) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003c36e1 in ThreadPThread__PThreadLockMutex (mutex=0x613c20, line=1279) at ../src/thread/PTHREAD/ThreadPThread.m3:1256
#4  0x003c37e3 in ThreadPThread__AtForkPrepare () at ../src/thread/PTHREAD/ThreadPThread.m3:1279
#5  0x97137217 in _cthread_fork_prepare ()
#6  0x97137155 in fork ()
#7  0x00352910 in ProcessPosixCommon__Create_ForkExec (cmd=0x242c2ac, params=0x242c3ac, env=0x0, wd=0x57bf44, stdin=0x0, stdout=0x239f9a4, stderr=0x2433508) at ../src/os/POSIX/ProcessPosixC
ommon.m3:75
#8  0x00353de5 in Process__Create (cmd=0x242c2ac, params=0x242c3ac, env=0x0, wd=0x57bf44, stdin=0x0, stdout=0x239f9a4, stderr=0x2433508) at ../src/os/POSIX/ProcessPosix.m3:21
#9  0x00316384 in ProcUtils__Apply__Exec.1072 () at ../src/ProcUtils.m3:180
#10 0x00316d6e in ProcUtils__Apply (self=0x242f160) at ../src/ProcUtils.m3:274
#11 0x003c0ea5 in ThreadPThread__RunThread (me=0x2009600) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#12 0x003c0b9a in ThreadPThread__ThreadBase (param=0x2009600) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#13 0x9713885d in _pthread_start ()
#14 0x971386e2 in thread_start ()

Thread 65 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613c20) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003beefd in ThreadPThread__InitMutex (m=0x242a6f8, root=0x242a6f4, Clean=0x3bed6f) at ../src/thread/PTHREAD/ThreadPThread.m3:101
#4  0x003befce in ThreadPThread__LockMutex (m=0x242a6f4) at ../src/thread/PTHREAD/ThreadPThread.m3:117
#5  0x00355ff6 in Rd__GetChar (rd=0x242a6f4) at ../src/rw/Rd.m3:33
#6  0x00364b5d in Lex__Skip (rd=0x242a6f4, cs=0x587ff8) at ../src/fmtlex/Lex.m3:46
#7  0x00366e53 in Lex__ReadReal (rd=0x242a6f4, sign=0xb1cf0603, digits=0xb1cf061c, digCnt=0xb1cf0608, exp=0xb1cf060c) at ../src/fmtlex/Lex.m3:569
#8  0x003678f0 in Lex__LongReal (rd=0x242a6f4) at ../src/fmtlex/Lex.m3:662
#9  0x0036834e in Scan__LongReal (txt=0x1753494) at ../src/fmtlex/Scan.m3:77
#10 0x002603f5 in SchemeInputPort__NextToken (t=0x2380de4, wx=0x242a4a8) at ../src/SchemeInputPort.m3:439
#11 0x00260756 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x242a4a8) at ../src/SchemeInputPort.m3:240
#12 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#13 0x0025edad in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:191
#14 0x00260738 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x242a40c) at ../src/SchemeInputPort.m3:240
#15 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#16 0x00260738 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x239ddc4) at ../src/SchemeInputPort.m3:240
#17 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#18 0x00260738 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x239dc4c) at ../src/SchemeInputPort.m3:240
#19 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#20 0x00260738 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x239da8c) at ../src/SchemeInputPort.m3:240
#21 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#22 0x00260738 in SchemeInputPort__ReadTail2 (t=0x2380de4, wx=0x239d9b4) at ../src/SchemeInputPort.m3:240
#23 0x0025ebdd in SchemeInputPort__Read (t=0x2380de4) at ../src/SchemeInputPort.m3:179
#24 0x0024db82 in Scheme__LoadPort (t=0x237018c, in=0x2380de4) at ../src/Scheme.m3:317
#25 0x0024ae56 in Scheme__LoadRd (t=0x237018c, rd=0x2379cfc, fn=0x0) at ../src/Scheme.m3:174
#26 0x0024e659 in Scheme__ReadInitialFiles (t=0x237018c, files=0x4afc8c) at ../src/Scheme.m3:134
#27 0x0024ea46 in Scheme__Init2 (t=0x237018c, input=0x15ac4ec, output=0x15ac514, files=0x4afc8c, env=0x0) at ../src/Scheme.m3:127
#28 0x0024a89a in Scheme__Init (t=0x237018c, files=0x4afc8c, env=0x0) at ../src/Scheme.m3:102
#29 0x0002abbc in SchemeCalculator__Init (t=0x237017c, code=0x225541c) at ../src/SchemeCalculator.m3:28
#30 0x0000e373 in EuroDriver__OApply (cl=0x2367b70) at ../src/EuroDriver.m3:865
#31 0x003c0ea5 in ThreadPThread__RunThread (me=0x2008b80) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#32 0x003c0b9a in ThreadPThread__ThreadBase (param=0x2008b80) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#33 0x9713885d in _pthread_start ()
#34 0x971386e2 in thread_start ()

Thread 64 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613ca0) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003c3bb3 in RTOS__LockHeap () at ../src/thread/PTHREAD/ThreadPThread.m3:1336
#4  0x003b023b in RTHeapRep__RegisterFinalCleanup (r=0x2430f38, p=0x3bed6f) at ../src/runtime/common/RTCollector.m3:2148
#5  0x003bef65 in ThreadPThread__InitMutex (m=0x2430f3c, root=0x2430f38, Clean=0x3bed6f) at ../src/thread/PTHREAD/ThreadPThread.m3:106
#6  0x003befce in ThreadPThread__LockMutex (m=0x2430f38) at ../src/thread/PTHREAD/ThreadPThread.m3:117
#7  0x00355ff6 in Rd__GetChar (rd=0x2430f38) at ../src/rw/Rd.m3:33
#8  0x00364b5d in Lex__Skip (rd=0x2430f38, cs=0x587ff8) at ../src/fmtlex/Lex.m3:46
#9  0x00366e53 in Lex__ReadReal (rd=0x2430f38, sign=0xb1c6e7c3, digits=0xb1c6e7dc, digCnt=0xb1c6e7c8, exp=0xb1c6e7cc) at ../src/fmtlex/Lex.m3:569
#10 0x003678f0 in Lex__LongReal (rd=0x2430f38) at ../src/fmtlex/Lex.m3:662
#11 0x0036834e in Scan__LongReal (txt=0x1753494) at ../src/fmtlex/Scan.m3:77
#12 0x002603f5 in SchemeInputPort__NextToken (t=0x237ede4, wx=0x2430e44) at ../src/SchemeInputPort.m3:439
#13 0x00260756 in SchemeInputPort__ReadTail2 (t=0x237ede4, wx=0x2430e44) at ../src/SchemeInputPort.m3:240
#14 0x0025ebdd in SchemeInputPort__Read (t=0x237ede4) at ../src/SchemeInputPort.m3:179
#15 0x00260738 in SchemeInputPort__ReadTail2 (t=0x237ede4, wx=0x2430db0) at ../src/SchemeInputPort.m3:240
#16 0x0025ebdd in SchemeInputPort__Read (t=0x237ede4) at ../src/SchemeInputPort.m3:179
#17 0x00260738 in SchemeInputPort__ReadTail2 (t=0x237ede4, wx=0x2430cd8) at ../src/SchemeInputPort.m3:240
#18 0x0025ebdd in SchemeInputPort__Read (t=0x237ede4) at ../src/SchemeInputPort.m3:179
#19 0x0024db82 in Scheme__LoadPort (t=0x236e18c, in=0x237ede4) at ../src/Scheme.m3:317
#20 0x0024ae56 in Scheme__LoadRd (t=0x236e18c, rd=0x2377cfc, fn=0x0) at ../src/Scheme.m3:174
#21 0x0024e659 in Scheme__ReadInitialFiles (t=0x236e18c, files=0x4afc8c) at ../src/Scheme.m3:134
#22 0x0024ea46 in Scheme__Init2 (t=0x236e18c, input=0x15ac4ec, output=0x15ac514, files=0x4afc8c, env=0x0) at ../src/Scheme.m3:127
#23 0x0024a89a in Scheme__Init (t=0x236e18c, files=0x4afc8c, env=0x0) at ../src/Scheme.m3:102
#24 0x0002abbc in SchemeCalculator__Init (t=0x236e17c, code=0x225541c) at ../src/SchemeCalculator.m3:28
#25 0x0000e373 in EuroDriver__OApply (cl=0x2367ad8) at ../src/EuroDriver.m3:865
#26 0x003c0ea5 in ThreadPThread__RunThread (me=0x2008af0) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#27 0x003c0b9a in ThreadPThread__ThreadBase (param=0x2008af0) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#28 0x9713885d in _pthread_start ()
#29 0x971386e2 in thread_start ()

Thread 60 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x2006bc0) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003befff in ThreadPThread__LockMutex (m=0x22cc724) at ../src/thread/PTHREAD/ThreadPThread.m3:119
#4  0x003bed47 in Thread__Acquire (m=0x22cc724) at ../src/thread/PTHREAD/ThreadPThread.m3:81
#5  0x002d9fb9 in SX__Lock (arr=0x235a9a4) at ../src/SX.m3:318
#6  0x002d983f in SXSelect__WaitE (on=0xb1beccd8, except=0x0, touched=0x0) at ../src/SX.m3:241
#7  0x002d8e0d in SXSelect__Wait (on=0xb1beccd8, touched=0x0) at ../src/SX.m3:149
#8  0x00010c7d in EuroDriver__OApply__MainLoop.1174 () at ../src/EuroDriver.m3:832
#9  0x0000e830 in EuroDriver__OApply (cl=0x22aad10) at ../src/EuroDriver.m3:892
#10 0x003c0ea5 in ThreadPThread__RunThread (me=0x2007080) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#11 0x003c0b9a in ThreadPThread__ThreadBase (param=0x2007080) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#12 0x9713885d in _pthread_start ()
#13 0x971386e2 in thread_start ()

Thread 59 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613c20) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003beefd in ThreadPThread__InitMutex (m=0x2354a24, root=0x2354a20, Clean=0x3bed6f) at ../src/thread/PTHREAD/ThreadPThread.m3:101
#4  0x003befce in ThreadPThread__LockMutex (m=0x2354a20) at ../src/thread/PTHREAD/ThreadPThread.m3:117
#5  0x003bed47 in Thread__Acquire (m=0x2354a20) at ../src/thread/PTHREAD/ThreadPThread.m3:81
#6  0x00359e60 in RdClass__Lock (rd=0x2354a20) at ../src/rw/Rd.m3:521
#7  0x0025e4c4 in SchemeInputPort__Lock (t=0x2354b08) at ../src/SchemeInputPort.m3:72
#8  0x0025f3dc in SchemeInputPort__NextToken (t=0x2354b08, wx=0x2354b28) at ../src/SchemeInputPort.m3:299
#9  0x0025eb94 in SchemeInputPort__Read (t=0x2354b08) at ../src/SchemeInputPort.m3:177
#10 0x0024aea9 in Scheme__LoadEval (t=0x224418c, rd=0x2354a20) at ../src/Scheme.m3:291
#11 0x0024af22 in Scheme__LoadEvalText (t=0x224418c, txt=0x225541c) at ../src/Scheme.m3:306
#12 0x0002ad28 in SchemeCalculator__Calc (t=0x224417c, arr=0xb1b6ab04) at ../src/SchemeCalculator.m3:47
#13 0x0000f37c in EuroDriver__OApply__CalcOrderSize.1137 () at ../src/EuroDriver.m3:564
#14 0x0000ece1 in EuroDriver__OApply__SafetyViolation.1126 () at ../src/EuroDriver.m3:507
#15 0x00010a04 in EuroDriver__OApply__MainLoop.1174 () at ../src/EuroDriver.m3:795
#16 0x0000e830 in EuroDriver__OApply (cl=0x22aac78) at ../src/EuroDriver.m3:892
#17 0x003c0ea5 in ThreadPThread__RunThread (me=0x2007020) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#18 0x003c0b9a in ThreadPThread__ThreadBase (param=0x2007020) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#19 0x9713885d in _pthread_start ()
#20 0x971386e2 in thread_start ()

Thread 52 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613ca0) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003c3bb3 in RTOS__LockHeap () at ../src/thread/PTHREAD/ThreadPThread.m3:1336
#4  0x003a66fc in RTAllocator__AllocTraced (dataSize=24, dataAlignment=4, thread=0x161e144) at ../src/runtime/common/RTAllocator.m3:363
#5  0x003a5a99 in RTAllocator__GetTracedObj (def=0x616d54) at ../src/runtime/common/RTAllocator.m3:224
#6  0x003a550f in RTHooks__AllocateTracedObj (defn=0x616d54) at ../src/runtime/common/RTAllocator.m3:122
#7  0x003d3c0e in RTHooks__Concat (t=0x23669ec, u=0x56c460) at ../src/text/TextCat.m3:28
#8  0x002b0dc2 in TWSTester__MakeString (contract=0xb19e4dc0) at ../src/TWSTester.m3:92
#9  0x002b11ac in TWSTester__ReqMktData (t=0x2220800, tickerId=3, contract=0xb19e4dc0) at ../src/TWSTester.m3:136
#10 0x002bccaf in TWSTestBridge__MreqMktData (t=0x2220800, tickerId=3, contract=0xb19e4dc0) at ../src/TWSTestBridge.m3 => ../src/TWSBridgeG.mg:580
#11 0x002978ec in PMTWS__TSCApply (cl=0x2221044) at ../src/PMTWS.m3:210
#12 0x003c0ea5 in ThreadPThread__RunThread (me=0x161e100) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#13 0x003c0b9a in ThreadPThread__ThreadBase (param=0x161e100) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#14 0x9713885d in _pthread_start ()
#15 0x971386e2 in thread_start ()

Thread 51 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613ca0) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003c3bb3 in RTOS__LockHeap () at ../src/thread/PTHREAD/ThreadPThread.m3:1336
#4  0x003a66fc in RTAllocator__AllocTraced (dataSize=136, dataAlignment=8, thread=0x16265a4) at ../src/runtime/common/RTAllocator.m3:363
#5  0x003a581e in RTAllocator__GetTracedRef (def=0x5b1280) at ../src/runtime/common/RTAllocator.m3:202
#6  0x003a54da in RTHooks__AllocateTracedRef (defn=0x5b1280) at ../src/runtime/common/RTAllocator.m3:115
#7  0x000b03d0 in IntDBTable_gcoms_instrumentTbl__Put (tbl=0x235f464, key=0xb1962d7c, val=0xb1962d00) at ../I386_DARWIN/IntDBTable_gcoms_instrumentTbl.m3 => /usr/local/cm3/pkg/libm3/src/table/Table.mg:126
#8  0x000ad244 in DBTable_gcoms_instrumentMonitor__CopyC (cl=0x228c080, res=0x235f464) at ../I386_DARWIN/DBTable_gcoms_instrumentMonitor.m3 => /Users/mika/t/calarm/ratsql/src/TableMonitor.mg:362
#9  0x0008c6de in GCOMSPortfolio__Lock (t=0x2255cd8) at ../src/GCOMSPortfolio.m3:94
#10 0x00029faa in SXPortfolio__Scan (t=0x229a888) at ../src/SXPortfolio.m3:47
#11 0x0002a998 in SXPortfolio__Apply (cl=0x222057c) at ../src/SXPortfolio.m3:113
#12 0x003c0ea5 in ThreadPThread__RunThread (me=0x1626560) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#13 0x003c0b9a in ThreadPThread__ThreadBase (param=0x1626560) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#14 0x9713885d in _pthread_start ()
#15 0x971386e2 in thread_start ()

Thread 27 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x161e470) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003befff in ThreadPThread__LockMutex (m=0x22440a4) at ../src/thread/PTHREAD/ThreadPThread.m3:119
#4  0x002dbd63 in SXLongReal__Update (v=0x2244010, newValue=1293721230.087589, when=1293721230.087605) at ../I386_DARWIN/SXLongReal.m3 => ../src/SXType.mg:79
#5  0x002dc086 in SXLongReal__SetVar (v=0x2244010, newValue=1293721230.087589, when=1293721230.087605) at ../I386_DARWIN/SXLongReal.m3 => ../src/SXType.mg:98
#6  0x00300c16 in SXTimer__Loop (cl=0x173934c) at ../src/SXTimer.m3:64
#7  0x003c0ea5 in ThreadPThread__RunThread (me=0x2001ec0) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#8  0x003c0b9a in ThreadPThread__ThreadBase (param=0x2001ec0) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#9  0x9713885d in _pthread_start ()
#10 0x971386e2 in thread_start ()

Thread 2 (process 32362):
#0  0x9710b142 in semaphore_wait_signal_trap ()
#1  0x97110c46 in pthread_mutex_lock ()
#2  0x003c4a0a in ThreadPThread__pthread_mutex_lock (i=0x613c20) at ../src/thread/PTHREAD/ThreadPThreadC.c:452
#3  0x003beefd in ThreadPThread__InitMutex (m=0x23480ac, root=0x23480a8, Clean=0x3bed6f) at ../src/thread/PTHREAD/ThreadPThread.m3:101
#4  0x003befce in ThreadPThread__LockMutex (m=0x23480a8) at ../src/thread/PTHREAD/ThreadPThread.m3:117
#5  0x00355ff6 in Rd__GetChar (rd=0x23480a8) at ../src/rw/Rd.m3:33
#6  0x00364b5d in Lex__Skip (rd=0x23480a8, cs=0x587ff8) at ../src/fmtlex/Lex.m3:46
#7  0x00366e53 in Lex__ReadReal (rd=0x23480a8, sign=0xb0080ca3, digits=0xb0080cbc, digCnt=0xb0080ca8, exp=0xb0080cac) at ../src/fmtlex/Lex.m3:569
#8  0x003678f0 in Lex__LongReal (rd=0x23480a8) at ../src/fmtlex/Lex.m3:662
#9  0x0036834e in Scan__LongReal (txt=0x234807c) at ../src/fmtlex/Scan.m3:77
#10 0x003260a0 in XTime__ClApply (cl=0x15971b8) at ../src/XTime.m3:185
#11 0x003c0ea5 in ThreadPThread__RunThread (me=0x161a0b0) at ../src/thread/PTHREAD/ThreadPThread.m3:450
#12 0x003c0b9a in ThreadPThread__ThreadBase (param=0x161a0b0) at ../src/thread/PTHREAD/ThreadPThread.m3:422
#13 0x9713885d in _pthread_start ()
#14 0x971386e2 in thread_start ()

More information about the M3devel mailing list