[M3devel] deadlock in pthreads??

Sat Aug 9 21:16:04 CEST 2014

Hi m3devel (mainly Tony),

I am finally trying to join the rest of you in using pthreads since you
say it works and now I really need it for my next project.

I am on AMD64_LINUX, with two compilers installed, source code as of January for those:

1. user threads --- what I normally use with no problems

2. pthreads     --- built it but don't use it because it's from before the final fixes
                    that I am told make the thread tester work

(144)truffles:~/bsd/cm3-cvs-anon/cm3/m3-sys/cm3>uname -a
Linux truffles 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux

(147)truffles:~/bsd/cm3-cvs-anon/cm3/m3-sys/cm3>cm3 -version
Critical Mass Modula-3 version d5.9.0
  last updated: 2010-07-21
  compiled: 2014-01-03 04:15:06
  configuration: /home/mika/bsd/cm3-boot2/bin/cm3.cfg
  host: AMD64_LINUX
  target: AMD64_LINUX

What I did:

cvs update -d cm3

(cvs update everything)

Then I tried:

I. with existing user threads compiler

   ./upgrade.sh

   upgraded my compiler "successfully"---see below for results

II. with existing pthreads compiler

   1. ./upgrade.sh

   upgraded my compiler "successfully"---see below for results

   2. started over and did manual upgrade process per Tony's instructions a while back (attached)

   upgraded my compiler "successfully"---see below for results

RESULTS IN ALL CASES ABOVE:

when running finished compiler, it sometimes deadlocks.  "Sometimes" = frequently enough that every
build breaks.  "Sometimes not" seems to only be in gdb.

Here is a traceback:

(98)truffles:~/bsd/src>gdb cm3
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /big/home/mika/cm3-boot2.pthreads/bin/cm3...done.
(gdb) run
Starting program: /big/home/mika/cm3-boot2.pthreads/bin/cm3 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
--- building in ../AMD64_LINUX ---

[New Thread 0x2aaaab6f8700 (LWP 29517)]
^C
Program received signal SIGINT, Interrupt.
__lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
136     ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: No such file or directory.
(gdb) where
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
#1  0x00002aaaaaf59339 in _L_lock_926 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00002aaaaaf5915b in __pthread_mutex_lock (mutex=0x12d00e0) at pthread_mutex_lock.c:61
#3  0x0000000000720af8 in ThreadPThread__pthread_mutex_lock (mutex=0x12d00e0) at ../src/thread/PTHREAD/ThreadPThreadC.c:506
#4  0x000000000071ab5f in ThreadPThread__UnlockMutex (M3_AYIbX3_m=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:151
#5  0x000000000071b018 in ThreadPThread__XWait (M3_DMxDjQ_self=<error reading variable>, M3_AYIbX3_m=<error reading variable>, M3_Bl0jv4_c=<error reading variable>, M3_AicXUJ_alertable=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:204
#6  0x000000000071d313 in ThreadPThread__XJoin (M3_DMxDjQ_self=<error reading variable>, M3_BXP32l_t=<error reading variable>, M3_AicXUJ_alertable=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:557
#7  0x000000000071d462 in Thread__Join (M3_BXP32l_t=<error reading variable>) at ../src/thread/PTHREAD/ThreadPThread.m3:569
#8  0x000000000040da95 in Builder__ForceAllPromisesInParallel (M3_C58HwX_promises=<error reading variable>, M3_Cwb5VA_parallelism=<error reading variable>) at ../src/Builder.m3:1002
#9  0x000000000040e083 in Builder__CompileEverything (M3_DqhcFh_s=<error reading variable>, M3_AE6M8L_schedule=<error reading variable>) at ../src/Builder.m3:1038
#10 0x0000000000408ce3 in Builder__CompileUnits (M3_Bd56fi_main=<error reading variable>, M3_ClaoYw_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_A2QN6Z_info_kind=<error reading variable>, 
    M3_An02H2_mach=<error reading variable>) at ../src/Builder.m3:337
#11 0x000000000040726b in Builder__BuildPgm (M3_Bd56fi_prog=<error reading variable>, M3_ClaoYw_units=<error reading variable>, M3_C1FTrk_sys_libs=<error reading variable>, M3_AicXUJ_shared=<error reading variable>, M3_An02H2_m=<error reading variable>)
    at ../src/Builder.m3:28
#12 0x0000000000425d6f in M3Build__BuildProgram (M3_ABp1Zk_t=<error reading variable>, M3_DLS2Hj_nm=<error reading variable>) at ../src/M3Build.m3:1515
#13 0x0000000000425bb7 in M3Build__DoProgram (M3_An02H2_m=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>) at ../src/M3Build.m3:1491
#14 0x00000000004d0cf9 in QMachine__DoCall (M3_An02H2_t=<error reading variable>, M3_AcxOUs_n_args=<error reading variable>, M3_AicXUJ_isFunc=<error reading variable>, M3_AicXUJ_outer=<error reading variable>) at ../src/QMachine.m3:546
#15 0x00000000004cf98b in QMachine__Eval (M3_An02H2_t=<error reading variable>) at ../src/QMachine.m3:422
#16 0x00000000004cdcb3 in QMachine__Evaluate (M3_An02H2_t=<error reading variable>, M3_CYwAos_s=<error reading variable>) at ../src/QMachine.m3:165
#17 0x00000000004c4764 in Quake__Run (M3_An02H2_m=<error reading variable>, M3_Bd56fi_source_file=<error reading variable>) at ../src/Quake.m3:19
#18 0x000000000041f789 in M3Build__Run (M3_ABp1Zk_t=<error reading variable>, M3_Bd56fi_makefile=<error reading variable>) at ../src/M3Build.m3:242
#19 0x0000000000437b9f in Main__DoIt () at ../src/Main.m3:117
#20 0x0000000000437f23 in Main_M3 (M3_AcxOUs_mode=<error reading variable>) at ../src/Main.m3:231
#21 0x000000000070d749 in RTLinker__RunMainBody (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:408
#22 0x000000000070cad4 in RTLinker__AddUnitI (M3_DjPxE3_m=<error reading variable>) at ../src/runtime/common/RTLinker.m3:115
#23 0x000000000070cb68 in RTLinker__AddUnit (M3_DjPxE5_b=<error reading variable>) at ../src/runtime/common/RTLinker.m3:124
#24 0x00000000004061a8 in main (argc=1, argv=0x7fffffffe388, envp=0x7fffffffe398) at _m3main.c:22
(gdb) 

Below are the instructions I've been using in case II. 2. above.

Note that cm3cg doesn't get installed when following the instructions!  I'm not sure what does that, I have to copy it manually from the derived dir to the bin dir for the compiler.

Date:    Sun, 24 Jun 2007 10:38:34 EDT
To:      Mika Nystrom <mika at async.caltech.edu>
cc:      m3devel at elegosoft.de
From:    Tony Hosking <hosking at cs.purdue.edu>
Subject: Re: [M3devel] strange errors... 

Return-Path: hosking at cs.purdue.edu
Delivery-Date: Sun Jun 24 07:38:38 2007
In-Reply-To: <200706231838.l5NIcRef079333 at camembert.async.caltech.edu>
References: <200706231838.l5NIcRef079333 at camembert.async.caltech.edu>

Sounds like we really need some work done in this area.  I very  
rarely use the build scripts, since I bootstrap manually from old  
compilers to new compilers rather than use the scripts.  I suggest  
the following approach, which I hope you will try, and then report  
back on.

Install the bootstrap compiler as you did originally:

> rm -rf /usr/local/cm3/*
>
> cd ~/cm3-cvs
> mkdir boot
> cd boot
> tar xzvf ../cm3-min-POSIX-FreeBSD4-d5.3.1-2005-10-05.tgz
> ./cminstall

Now you will have some kind of cm3 installed, presumably in /usr/ 
local/cm3/bin/cm3.

Make sure you have a fresh CVS checkout in directory cm3 (let's  
assume this is in your home directory ~/cm3).  Also, make sure you  
have an up-to-date version of the CM3 backend compiler cm3cg  
installed by executing the following:

STEP 0:

export CM3=/usr/local/cm3/bin/cm3
cd ~/cm3/m3-sys/m3cc
$CM3
$CM3 -ship

You can skip this last step if you know your backend compiler is up  
to date.

Now, let's build the new compiler from scratch (this is the sequence  
I use regularly to test changes to the run-time system whenever I  
make them):

can replace compilation step with

rm -rf <der. dir> && $CM3 && $CM3 -ship 

STEP 1:

(do these LATER if bootstrapping from old compiler!)
cd ~/cm3/m3-libs/m3core
$CM3
$CM3 -ship
cd ~/cm3/m3-libs/libm3
$CM3
$CM3 -ship

Now build the compiler:

cd  ~/cm3/m3-libs/sysutils ; $CM3 etc.

cd ~/cm3/m3-sys/m3middle; $CM3 etc.

repeat for:

m3-sys/m3objfile 

m3-sys/m3back 

m3-sys/m3linker

m3-sys/m3front

m3-sys/m3quake

m3-sys/cm3

[ here we may need -DROOT=<root dir, e.g., /home/mika/cm3> ]

At this point you should have a bootstrapped version of cm3 installed  
in the directory /usr/local/cm3/pkg/cm3/TARGET/cm3 (where TARGET is  
the CM3 target platform you are building on -- e.g., LINUXLIBC6,  
PPC_DARWIN, ...).  Note that this did not overwrite your original  
installed compiler in /usr/local/cm3/bin/cm3.  We
are now going to test the new compiler, which was built using the old  
compiler, by bootstrapping it one more time.

(If m3core and libm3 were skipped earlier, do them now, and RE-DO all
of the above packages before proceeding.)

>From here on out, please replace TARGET with your target platform as  
appropriate.

STEP 2:

export CM3=/usr/local/cm3/pkg/cm3/TARGET/cm3
cd ~/cm3/scripts

./do-cm3-std.sh realclean
REPEAT STEP 1 to rebuild the libraries and the compiler using the  
STEP 1 bootstrap compiler.

Now you have a STEP 2 bootstrap compiler installed in /usr/local/cm3/ 
pkg/cm3/TARGET/cm3.  Let's assume the new compiler now works properly  
since it successfully bootstrapped itself, but to be sure we can now  
use it to rebuild the world:

cd ~/cm3/scripts
./do-cm3-std.sh realclean
./do-cm3-std.sh buildship

Assuming this succeeded then we can be pretty sure /usr/local/cm3/pkg/ 
cm3/TARGET/cm3 is good, so we can make it our default compiler by  
replacing the original /usr/local/cm3/bin/cm3:

cp $CM3 /usr/local/cm3/bin/cm3

On Jun 23, 2007, at 2:38 PM, Mika Nystrom wrote:

> Ok, I'm sorry if I seem a bit dimwitted in the morning like this,
> but how exactly does one get started?  I wish there were a file called
> "GETTING_STARTED" or something like that in scripts...
...