[M3devel] More on threading

Mika Nystrom mika at async.caltech.edu
Sun Feb 13 19:17:17 CET 2011


Jay K writes:
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>
>Yes=2C gcc is slow and using as is wasteful.
>Yes this should all be multi-threaded.
>However it seems to me that m3front has many globals and therefore
>really making things multi-threaded isn't going to be easy.

OK but as long as the back-end is taking up most of the cycles
the property that the output of the front-end is used only by cmcg1
(m3back/m3cgc/whatever you call it) and the assembler means that what I
did works for a pretty big speedup.  In fact you can see that it works
just fine (from the point of view of generating the correct output) since
you can re-order running the backend with respect to all the steps done
by the front-end.  What you lose is (perhaps) some error messages, and of
course, in general, any implicit dependencies between the error messages.

I don't understand why the program gets stuck in uxmxn though.  Maybe a 
problem with, sigh, user threads...

>I think you are mostly wasting your time on user threads though.

I hope so... I too would really like to use pthreads.

>
>
>Have you tried the NT/x86 system?
>It is very fast.
>
>
>In the gcc system=2C each .mc file is run through m3cgc1 to produce an .ms =
>file=2C
>which is run through as to produce an object file .mo.
>
>
>In the NT/x86 system=2C cm3 outputs .obj files directly.
>No .mc files. No m3cgc1 or as.
>Granted=2C I'd just as soon output .c files=2C and hope for a C compiler th=
>at doesn't run a separate assembler.

The NT compiler sounds faster but of course it's easier to parallelize the
unix one using the method I outlined :-)  Well unless you're using pthreads
of course, then you can just fork a new thread for the integrated NT backend
as long as it doesn't share too many variables with the front-end.

But no I don't use Windows.  I was asking on here how to get started with
Modula-3 on Windows once but my machine was/is a Windows 2000 system and
apparently CM3 no longer works on win2k..?

I used to use PM3/Klagenfurt a lot on Windows (Cygwin) but thankfully
my clients on that project let me switch the system to Unix so I don't
have to touch that anymore.

>
>
>My timing might loosen up a bit and I hope to spend some on the thread stre=
>ss tester using pthreads. Next weekend.
>Am I right though that Darwin works fine?
>Solaris? FreeBSD? NetBSD? OpenBSD? Linux? I know=2C not Linux.
>You generally don't need any of these systems.

That would be great.  I don't think pthreads works properly anywhere at
the moment.  My guess is that it never has, if you stress it hard enough.
I first ran into problems on Darwin and then switched to AMD64 simply
because I have better access to faster machines with AMD64.  At first
there were some obvious problems that Tony fixed quickly but then I
kept running into odd behavior with my application and I added various
other tests until I could reproduce everything I'd seen go wrong with my
application, and I think at that point I was seeing problems on every
platform (except user threads).  The uxmxn problem I see with user
threads on FreeBSD suggests that perhaps I need to add some more tests,
however.

     Mika

>All of the x86/AMD64 systems can run in a virtual machine.
>Solaris/sparc we have good access.
>
>
> - Jay
>
>
>> To: m3devel at elegosoft.com
>> Date: Sat=2C 12 Feb 2011 23:04:14 -0800
>> From: mika at async.caltech.edu
>> CC: jay.krell at cornell.edu
>> Subject: [M3devel] More on threading
>>=20
>>=20
>> Hi again m3devel=2C
>>=20
>> Well I thought I would relate another story having to do with threading
>> and CM3.  It's half-baked at the moment=2C but it only reflects a few hou=
>rs'
>> work on my part=2C sorry... I'm about to give up now because I don't know
>> if the problem is in my code=2C or in threading=2C or what... maybe it is
>> the atfork problem Jay talked about?
>>=20
>> Programmers spend a lot of time waiting for their compilers to run=2C so
>> it would be nice if the compilation process were faster.  After perusing
>> the source code for the CM3 I realized that there is no feedback at all
>> from the back-end and assembler steps to the rest of the compiler=2C and
>> I believe that with the GCC back-end the vast majority of the runtime
>> of the compiler is spent in these stages.
>>=20
>> So.... how hard could this be to parallelize?  I decided to try making
>> the two stages (cm3cg1 and as) into "promises" rather than running them
>> on the spot.  The QMachine adds a promise to call the back-end to a
>> RefSeq.T called "promises"=2C to which the Builder can also add promises.
>> Then=2C after making all the promises=2C collect them and run them ("forc=
>e
>> them" a true Schemer would say) in parallel.
>>=20
>> Of course something goes wrong somewhere.  With 10 threads I am able to
>> run 31 seconds of CPU time in 19 seconds of wallclock time (using an
>> average 160% of CPU) but then something goes wrong and my process
>> is stuck in state "uxmxn".  This is using user threads on AMD64_FREEBSD
>> on a machine with four CPUs.
>>=20
>>      Mika
>>=20
>> P.S. diff attached=2C it's a bit ugly=2C but maybe it works on some other
>> system....  the code works just fine if you run one thread at a time.
>> I.e.=2C if you change
>>=20
>>      IF threads.size() > 10 THEN EVAL Thread.Join(threads.remlo()) END=3B
>>=20
>> to
>>=20
>>      IF threads.size() > 0 THEN EVAL Thread.Join(threads.remlo()) END=3B
>>=20
>>=20
>>=20
>> Index: cm3/src/Builder.m3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/cm3/src/Builder.m3=2Cv
>> retrieving revision 1.36
>> diff -c -r1.36 Builder.m3
>> *** cm3/src/Builder.m3	24 Aug 2010 05:24:24 -0000	1.36
>> --- cm3/src/Builder.m3	13 Feb 2011 06:47:40 -0000
>> ***************
>> *** 15=2C20 ****
>> --- 15=2C22 ----
>>   IMPORT QIdent=3B
>>   FROM Target IMPORT M3BackendMode_t=2C BackendAssembly=2C BackendModeStr=
>ings=3B
>>   FROM M3Path IMPORT OSKind=2C OSKindStrings=3B
>> + IMPORT Pathname=3B
>> + IMPORT RefSeq=3B
>>  =20
>>   TYPE
>>     UK =3D M3Unit.Kind=3B
>> ***************
>> *** 131=2C136 ****
>> --- 133=2C139 ----
>>       link_coverage : TEXT=3B               (* coverage library *)
>>       m3_front_flags: Arg.List=3B           (* configuration options for =
>the front *)
>>       m3_options    : Arg.List=3B           (* misc. user options for the=
> frontend *)
>> +     delayBackend :=3D FALSE=3B
>>     END=3B
>>  =20
>>   TYPE
>> ***************
>> *** 932=2C937 ****
>> --- 935=2C956 ----
>>  =20
>>   (*------------------------------------------------------------ compilat=
>ion --*)
>>  =20
>> + TYPE MarkerPromise =3D QMachine.Promise BRANDED OBJECT OVERRIDES fulfil=
> :=3D FulfilNothing END=3B
>> +=20
>> + PROCEDURE FulfilNothing(<*UNUSED*>mp : MarkerPromise) =3D BEGIN END Ful=
>filNothing=3B
>> +=20
>> + TYPE SeqClosure =3D Thread.Closure OBJECT seq : RefSeq.T=3B OVERRIDES a=
>pply :=3D SeqApply END=3B
>> +=20
>> + PROCEDURE SeqApply(cl : SeqClosure) : REFANY =3D
>> +   BEGIN
>> +     FOR i :=3D 0 TO cl.seq.size()-1 DO
>> +       WITH p =3D NARROW(cl.seq.get(i)=2CQMachine.Promise) DO
>> +         p.fulfil()
>> +       END
>> +     END=3B
>> +     RETURN NIL
>> +   END SeqApply=3B
>> +=20
>>   PROCEDURE CompileEverything (s: State=3B  schedule: SourceList) =3D
>>     VAR u: M3Unit.T=3B
>>     BEGIN
>> ***************
>> *** 941=2C948 ****
>>  =20
>>       (* compile all the sources using the initial schedule *)
>>       FOR i :=3D 0 TO LAST (schedule^) DO
>> !       CompileOne (s=2C schedule[i])=3B
>>       END=3B
>>       FlushPending (s)=3B
>>  =20
>>       (* recompile any interfaces where we goofed on the exports *)
>> --- 960=2C1000 ----
>>  =20
>>       (* compile all the sources using the initial schedule *)
>>       FOR i :=3D 0 TO LAST (schedule^) DO
>> !       s.delayBackend :=3D TRUE=3B
>> !       TRY
>> !         CompileOne (s=2C schedule[i])=3B
>> !       FINALLY
>> !         s.delayBackend :=3D FALSE=3B
>> !       END=3B
>> !=20
>> !       s.machine.promises.addhi(NEW(MarkerPromise))=3B
>> !=20
>> !     END=3B
>> !=20
>> !     VAR
>> !       curSeq  :=3D NEW(RefSeq.T).init()=3B
>> !       threads :=3D NEW(RefSeq.T).init()=3B
>> !     BEGIN
>> !       FOR i :=3D 0 TO s.machine.promises.size()-1 DO
>> !         WITH p =3D s.machine.promises.get(i) DO
>> !           curSeq.addhi(p)=3B
>> !           IF i =3D s.machine.promises.size()-1 OR ISTYPE(p=2CMarkerProm=
>ise) THEN
>> !             WITH cl =3D NEW(SeqClosure=2C seq :=3D curSeq) DO
>> !               threads.addhi (Thread.Fork(cl))=3B
>> !=20
>> !               IF threads.size() > 10 THEN EVAL Thread.Join(threads.reml=
>o()) END=3B
>> !              =20
>> !               curSeq :=3D NEW(RefSeq.T).init()
>> !             END
>> !           END
>> !         END
>> !       END=3B
>> !       WHILE threads.size() > 0 DO EVAL Thread.Join(threads.remlo()) END=
>=3B
>>       END=3B
>> +            =20
>> +     EVAL s.machine.promises.init()=3B
>> +=20
>> +=20
>>       FlushPending (s)=3B
>>  =20
>>       (* recompile any interfaces where we goofed on the exports *)
>> ***************
>> *** 1151=2C1156 ****
>> --- 1203=2C1227 ----
>>       END=3B
>>     END CompileM3=3B
>>  =20
>> + TYPE
>> +   NotePromise =3D QMachine.Promise OBJECT
>> +     nam : Pathname.T=3B
>> +   OVERRIDES
>> +     fulfil :=3D FulfilNP=3B
>> +   END=3B
>> +=20
>> +   RemovePromise =3D QMachine.Promise OBJECT
>> +     nam : Pathname.T=3B
>> +   OVERRIDES
>> +     fulfil :=3D FulfilRP=3B
>> +   END=3B
>> +=20
>> + PROCEDURE FulfilNP(np : NotePromise) =3D=20
>> +   BEGIN Utils.NoteTempFile(np.nam) END FulfilNP=3B
>> +=20
>> + PROCEDURE FulfilRP(rp : RemovePromise) =3D=20
>> +   BEGIN Utils.Remove(rp.nam) END FulfilRP=3B
>> +=20
>>   PROCEDURE PushOneM3 (s: State=3B  u: M3Unit.T): BOOLEAN =3D
>>     VAR
>>       tmpC=2C tmpS: TEXT=3B
>> ***************
>> *** 1191=2C1208 ****
>>  =20
>>  =20
>>       | 3 =3D>  (* -bootstrap=2C +m3back=2C +asm *)
>>           tmpC :=3D TempCName (u)=3B
>>           tmpS :=3D TempSName (u)=3B
>>           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>>           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>>           IF RunM3 (s=2C u=2C tmpC) THEN
>> !           IF  RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> !           AND RunAsm (s=2C tmpS=2C u.object) THEN
>>             END=3B
>>             need_merge :=3D TRUE=3B
>>           END=3B
>>           IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>>           IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>>  =20
>>       | 6=2C    (* +bootstrap=2C +m3back=2C -asm *)
>>         7 =3D>  (* +bootstrap=2C +m3back=2C +asm *)
>> --- 1262=2C1318 ----
>>  =20
>>  =20
>>       | 3 =3D>  (* -bootstrap=2C +m3back=2C +asm *)
>> +       IF s.delayBackend THEN
>>           tmpC :=3D TempCName (u)=3B
>>           tmpS :=3D TempSName (u)=3B
>> + (*
>>           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>>           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>> + *)
>> +         IF (NOT s.keep_files) THEN=20
>> +           s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpC))=20
>> +         END=3B
>> +         IF (NOT s.keep_files) THEN=20
>> +           s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpS))=20
>> +         END=3B
>> +=20
>>           IF RunM3 (s=2C u=2C tmpC) THEN
>> !           s.machine.record(TRUE)=3B
>> !           TRY
>> !             IF  RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> !             AND RunAsm (s=2C tmpS=2C u.object) THEN
>> !             END=3B
>> !           FINALLY
>> !             s.machine.record(FALSE)
>>             END=3B
>> +=20
>>             need_merge :=3D TRUE=3B
>>           END=3B
>> + (*
>>           IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>>           IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>> + *)
>> +         IF (NOT s.keep_files) THEN=20
>> +           s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D tmpC))=
>=20
>> +         END=3B
>> +         IF (NOT s.keep_files) THEN=20
>> +           s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D tmpS))=
>=20
>> +         END=3B
>> +=20
>> +       ELSE
>> +           tmpC :=3D TempCName (u)=3B
>> +           tmpS :=3D TempSName (u)=3B
>> +           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>> +           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>> +           IF RunM3 (s=2C u=2C tmpC) THEN
>> +             IF  RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> +             AND RunAsm (s=2C tmpS=2C u.object) THEN
>> +             END=3B
>> +             need_merge :=3D TRUE=3B
>> +           END=3B
>> +           IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>> +           IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>> +       END
>>  =20
>>       | 6=2C    (* +bootstrap=2C +m3back=2C -asm *)
>>         7 =3D>  (* +bootstrap=2C +m3back=2C +asm *)
>> Index: cm3/src/m3makefile
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/cm3/src/m3makefile=2Cv
>> retrieving revision 1.24
>> diff -c -r1.24 m3makefile
>> *** cm3/src/m3makefile	8 Dec 2010 07:30:57 -0000	1.24
>> --- cm3/src/m3makefile	13 Feb 2011 06:47:42 -0000
>> ***************
>> *** 29=2C34 ****
>> --- 29=2C35 ----
>>   end
>>   module ("Arg")
>>   module ("Builder")
>>   module ("Dirs")
>>   module ("M3Build")
>>   module ("M3Loc")
>> Index: m3quake/src/QMachine.i3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.i3=2Cv
>> retrieving revision 1.6
>> diff -c -r1.6 QMachine.i3
>> *** m3quake/src/QMachine.i3	4 Sep 2009 10:24:07 -0000	1.6
>> --- m3quake/src/QMachine.i3	13 Feb 2011 06:47:43 -0000
>> ***************
>> *** 8=2C13 ****
>> --- 8=2C14 ----
>>  =20
>>   IMPORT Thread=2C Wr=2C QValue=2C QCode=3B
>>   FROM Quake IMPORT Machine=2C Error=2C ID=2C IDMap=3B
>> + IMPORT RefSeq=3B
>>  =20
>>   REVEAL
>>     T <: T_=3B
>> ***************
>> *** 15=2C20 ****
>> --- 16=2C22 ----
>>     T =3D Machine=3B
>>     T_ =3D OBJECT
>>       map: IDMap :=3D NIL=3B (* READONLY *)
>> +     promises : RefSeq.T=3B
>>     METHODS
>>       init      (map: IDMap): T=3B
>>       evaluate  (s: QCode.Stream)                     RAISES {Error=2C Th=
>read.Alerted}=3B
>> ***************
>> *** 37=2C42 ****
>> --- 39=2C46 ----
>>       set_wr    (wr: Wr.T)=3B
>>       exec_echo (b: BOOLEAN): BOOLEAN=3B
>>       trace     (b: BOOLEAN)=3B
>> +    =20
>> +     record(on : BOOLEAN)=3B     (* instead of performing certain acts=
>=2C promise *)
>>     END=3B
>>  =20
>>   PROCEDURE PushBool (t: T=3B  b: BOOLEAN)=3B
>> ***************
>> *** 51=2C54 ****
>> --- 55=2C63 ----
>>  =20
>>   PROCEDURE GetEnv (default=2C v0=2C v1=2C v2=2C v3=2C v4: TEXT :=3D NIL)=
>: TEXT=3B
>>  =20
>> + TYPE Promise =3D OBJECT METHODS fulfil() RAISES { Error }  END=3B
>> +=20
>>   END QMachine.
>> +=20
>> +=20
>> +=20
>> Index: m3quake/src/QMachine.m3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.m3=2Cv
>> retrieving revision 1.35
>> diff -c -r1.35 QMachine.m3
>> *** m3quake/src/QMachine.m3	3 Aug 2010 09:40:04 -0000	1.35
>> --- m3quake/src/QMachine.m3	13 Feb 2011 06:47:44 -0000
>> ***************
>> *** 16=2C21 ****
>> --- 16=2C22 ----
>>   IMPORT TextUtils=2C FSUtils=2C System=2C DirStack=3B (* sysutils *)
>>   IMPORT Compiler=3B
>>   IMPORT M3Path=3B
>> + IMPORT RefSeq=3B
>>  =20
>>   CONST
>>     OnUnix =3D (Compiler.ThisOS =3D Compiler.OS.POSIX)=3B
>> ***************
>> *** 44=2C49 ****
>> --- 45=2C52 ----
>>       shell     : TEXT         :=3D NIL=3B
>>       sh_option : TEXT         :=3D NIL=3B
>>       tmp_dir   : TEXT         :=3D NIL=3B
>> +=20
>> +     doRecord :=3D FALSE=3B
>>     OVERRIDES
>>       init      :=3D Init=3B
>>       evaluate  :=3D Evaluate=3B
>> ***************
>> *** 66=2C73 ****
>> --- 69=2C81 ----
>>       set_wr    :=3D SetWr=3B
>>       exec_echo :=3D ExecEcho=3B
>>       trace     :=3D Trace=3B
>> +=20
>> +     record :=3D Record=3B
>>     END=3B
>>  =20
>> + PROCEDURE Record(t : T=3B on : BOOLEAN) =3D=20
>> +   BEGIN t.doRecord :=3D on END Record=3B
>> +=20
>>   TYPE
>>     Registers =3D RECORD
>>       cp : QCode.Stream   :=3D NIL=3B (* code pointer *)
>> ***************
>> *** 139=2C144 ****
>> --- 147=2C154 ----
>>       t.globals    :=3D NEW (IntRefTbl.Default).init ()=3B
>>       t.default_wr :=3D Stdio.stdout=3B
>>  =20
>> +     t.promises :=3D NEW(RefSeq.T).init()=3B
>> +=20
>>       InitOSEnv (t)=3B
>>       InitBuiltins (t)=3B
>>  =20
>> ***************
>> *** 1555=2C1564 ****
>>             END=3B
>>           ELSE
>>             FlushIO ()=3B
>> !           Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=3B
>> !           handle :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=2C=
> n_shell_args)=2C
>> !                                     stdin :=3D stdin=2C stdout :=3D std=
>out=2C
>> !                                     stderr :=3D stderr)=3B
>>           END=3B
>>         EXCEPT
>>         | Thread.Alerted =3D>
>> --- 1565=2C1594 ----
>>             END=3B
>>           ELSE
>>             FlushIO ()=3B
>> !           IF t.doRecord THEN
>> !             handle :=3D NIL=3B
>> !             WITH a =3D NEW(REF ARRAY OF TEXT=2C n_shell_args) DO
>> !               a^ :=3D SUBARRAY(args=2C0=2Cn_shell_args)=3B
>> !               VAR wrx : Wr.T=3B BEGIN
>> !                 IF echo OR t.do_echo THEN
>> !                   wrx :=3D wr
>> !                 ELSE
>> !                   wrx :=3D NIL
>> !                 END=3B
>> !                 t.promises.addhi(NEW(ExecPromise=2C
>> !                                      cmd :=3D t.shell=2C
>> !                                      wr :=3D wrx=2C
>> !                                      args :=3D a=2C
>> !                                      t :=3D t=2C
>> !                                      ignore_errors :=3D ignore_errors))
>> !               END
>> !             END
>> !           ELSE
>> !             Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=
>=3B
>> !             handle :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=
>=2C n_shell_args)=2C
>> !                                       stdin :=3D stdin=2C stdout :=3D s=
>tdout=2C
>> !                                       stderr :=3D stderr)=3B
>> !           END=3B
>>           END=3B
>>         EXCEPT
>>         | Thread.Alerted =3D>
>> ***************
>> *** 1573=2C1579 ****
>>         END=3B
>>  =20
>>         (* wait for everything to shutdown... *)
>> !       exit_code :=3D Process.Wait (handle)=3B
>>       END=3B
>>  =20
>>       IF onlyTry THEN
>> --- 1603=2C1613 ----
>>         END=3B
>>  =20
>>         (* wait for everything to shutdown... *)
>> !       IF handle =3D NIL THEN
>> !         exit_code :=3D 0=20
>> !       ELSE
>> !         exit_code :=3D Process.Wait (handle)=3B
>> !       END=3B (* else we're only promising *)
>>       END=3B
>>  =20
>>       IF onlyTry THEN
>> ***************
>> *** 1589=2C1594 ****
>> --- 1623=2C1664 ----
>>  =20
>>     END ExecCommand=3B
>>  =20
>> + TYPE=20
>> +   ExecPromise =3D Promise OBJECT
>> +     cmd : TEXT=3B
>> +     args : REF ARRAY OF TEXT=3B
>> +     t : T=3B
>> +     wr : Wr.T=3B
>> +     ignore_errors : BOOLEAN=3B
>> +   OVERRIDES
>> +     fulfil :=3D FulfilExecPromise=3B
>> +   END=3B
>> +=20
>> + PROCEDURE FulfilExecPromise(ep : ExecPromise) RAISES { Error } =3D=20
>> +   VAR
>> +     stdin=2C stdout=2C stderr: File.T=3B
>> +   BEGIN
>> +     Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=3B
>> +     TRY
>> +       IF ep.wr # NIL THEN
>> +         Wr.PutText (ep.wr=2C ep.args[1])=3B
>> +         Wr.PutText (ep.wr=2C Wr.EOL)=3B
>> +         FlushIO ()=3B
>> +       END=3B
>> +       WITH handle =3D Process.Create (ep.cmd=2C ep.args^=2C
>> +                                     stdin :=3D stdin=2C stdout :=3D std=
>out=2C
>> +                                     stderr :=3D stderr)=2C
>> +            exit_code =3D Process.Wait(handle) DO
>> +         IF exit_code # 0 AND NOT ep.ignore_errors THEN
>> +           Err (ep.t=2C Fmt.F("exit %s: %s"=2C Fmt.Int(exit_code)=2C ep.=
>cmd))
>> +         END
>> +       END
>> +     EXCEPT
>> +       OSError.E (ec) =3D>
>> +           Err (ep.t=2C Fmt.F ("exec failed%s *** %s"=2C OSErr (ec)=2C e=
>p.cmd))=3B
>> +     END
>> +   END FulfilExecPromise=3B
>> +=20
>>   PROCEDURE KillProcess (handle: Process.T) =3D
>>     BEGIN
>>       IF (handle # NIL) THEN
>>=20
> 		 	   		  =
>
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 10pt=3B
>font-family:Tahoma
>}
>--></style>
></head>
><body class=3D'hmmessage'>
>Yes=2C gcc is slow and using as is wasteful.<br>Yes this should all be mult=
>i-threaded.<br>However it seems to me that m3front has many globals and the=
>refore<br>really making things multi-threaded isn't going to be easy.<br>I =
>think you are mostly wasting your time on user threads though.<br><br><br>H=
>ave you tried the NT/x86 system?<br>It is very fast.<br><br><br>In the gcc =
>system=2C each .mc file is run through m3cgc1 to produce an .ms file=2C<br>=
>which is run through as to produce an object file .mo.<br><br><br>In the NT=
>/x86 system=2C cm3 outputs .obj files directly.<br>No .mc files. No m3cgc1 =
>or as.<br>Granted=2C I'd just as soon output .c files=2C and hope for a C c=
>ompiler that doesn't run a separate assembler.<br><br><br>My timing might l=
>oosen up a bit and I hope to spend some on the thread stress tester using p=
>threads. Next weekend.<br>Am I right though that Darwin works fine?<br>Sola=
>ris? FreeBSD? NetBSD? OpenBSD? Linux? I know=2C not Linux.<br>You generally=
> don't need any of these systems.<br>All of the x86/AMD64 systems can run i=
>n a virtual machine.<br>Solaris/sparc we have good access.<br><br><br>&nbsp=
>=3B- Jay<br><br><br>&gt=3B To: m3devel at elegosoft.com<br>&gt=3B Date: Sat=2C=
> 12 Feb 2011 23:04:14 -0800<br>&gt=3B From: mika at async.caltech.edu<br>&gt=
>=3B CC: jay.krell at cornell.edu<br>&gt=3B Subject: [M3devel] More on threadin=
>g<br>&gt=3B <br>&gt=3B <br>&gt=3B Hi again m3devel=2C<br>&gt=3B <br>&gt=3B =
>Well I thought I would relate another story having to do with threading<br>=
>&gt=3B and CM3.  It's half-baked at the moment=2C but it only reflects a fe=
>w hours'<br>&gt=3B work on my part=2C sorry... I'm about to give up now bec=
>ause I don't know<br>&gt=3B if the problem is in my code=2C or in threading=
>=2C or what... maybe it is<br>&gt=3B the atfork problem Jay talked about?<b=
>r>&gt=3B <br>&gt=3B Programmers spend a lot of time waiting for their compi=
>lers to run=2C so<br>&gt=3B it would be nice if the compilation process wer=
>e faster.  After perusing<br>&gt=3B the source code for the CM3 I realized =
>that there is no feedback at all<br>&gt=3B from the back-end and assembler =
>steps to the rest of the compiler=2C and<br>&gt=3B I believe that with the =
>GCC back-end the vast majority of the runtime<br>&gt=3B of the compiler is =
>spent in these stages.<br>&gt=3B <br>&gt=3B So.... how hard could this be t=
>o parallelize?  I decided to try making<br>&gt=3B the two stages (cm3cg1 an=
>d as) into "promises" rather than running them<br>&gt=3B on the spot.  The =
>QMachine adds a promise to call the back-end to a<br>&gt=3B RefSeq.T called=
> "promises"=2C to which the Builder can also add promises.<br>&gt=3B Then=
>=2C after making all the promises=2C collect them and run them ("force<br>&=
>gt=3B them" a true Schemer would say) in parallel.<br>&gt=3B <br>&gt=3B Of =
>course something goes wrong somewhere.  With 10 threads I am able to<br>&gt=
>=3B run 31 seconds of CPU time in 19 seconds of wallclock time (using an<br=
>>&gt=3B average 160% of CPU) but then something goes wrong and my process<b=
>r>&gt=3B is stuck in state "uxmxn".  This is using user threads on AMD64_FR=
>EEBSD<br>&gt=3B on a machine with four CPUs.<br>&gt=3B <br>&gt=3B      Mika=
><br>&gt=3B <br>&gt=3B P.S. diff attached=2C it's a bit ugly=2C but maybe it=
> works on some other<br>&gt=3B system....  the code works just fine if you =
>run one thread at a time.<br>&gt=3B I.e.=2C if you change<br>&gt=3B <br>&gt=
>=3B      IF threads.size() &gt=3B 10 THEN EVAL Thread.Join(threads.remlo())=
> END=3B<br>&gt=3B <br>&gt=3B to<br>&gt=3B <br>&gt=3B      IF threads.size()=
> &gt=3B 0 THEN EVAL Thread.Join(threads.remlo()) END=3B<br>&gt=3B <br>&gt=
>=3B <br>&gt=3B <br>&gt=3B Index: cm3/src/Builder.m3<br>&gt=3B =3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>&gt=3B RCS file: /usr/cvs/cm3/m3=
>-sys/cm3/src/Builder.m3=2Cv<br>&gt=3B retrieving revision 1.36<br>&gt=3B di=
>ff -c -r1.36 Builder.m3<br>&gt=3B *** cm3/src/Builder.m3	24 Aug 2010 05:
>24:=
>24 -0000	1.36<br>&gt=3B --- cm3/src/Builder.m3	13 Feb 2011 06:47:40 -0
>000<b=
>r>&gt=3B ***************<br>&gt=3B *** 15=2C20 ****<br>&gt=3B --- 15=2C22 -=
>---<br>&gt=3B   IMPORT QIdent=3B<br>&gt=3B   FROM Target IMPORT M3BackendMo=
>de_t=2C BackendAssembly=2C BackendModeStrings=3B<br>&gt=3B   FROM M3Path IM=
>PORT OSKind=2C OSKindStrings=3B<br>&gt=3B + IMPORT Pathname=3B<br>&gt=3B + =
>IMPORT RefSeq=3B<br>&gt=3B   <br>&gt=3B   TYPE<br>&gt=3B     UK =3D M3Unit.=
>Kind=3B<br>&gt=3B ***************<br>&gt=3B *** 131=2C136 ****<br>&gt=3B --=
>- 133=2C139 ----<br>&gt=3B       link_coverage : TEXT=3B               (* c=
>overage library *)<br>&gt=3B       m3_front_flags: Arg.List=3B           (*=
> configuration options for the front *)<br>&gt=3B       m3_options    : Arg=
>.List=3B           (* misc. user options for the frontend *)<br>&gt=3B +   =
>  delayBackend :=3D FALSE=3B<br>&gt=3B     END=3B<br>&gt=3B   <br>&gt=3B   =
>TYPE<br>&gt=3B ***************<br>&gt=3B *** 932=2C937 ****<br>&gt=3B --- 9=
>35=2C956 ----<br>&gt=3B   <br>&gt=3B   (*----------------------------------=
>-------------------------- compilation --*)<br>&gt=3B   <br>&gt=3B + TYPE M=
>arkerPromise =3D QMachine.Promise BRANDED OBJECT OVERRIDES fulfil :=3D Fulf=
>ilNothing END=3B<br>&gt=3B + <br>&gt=3B + PROCEDURE FulfilNothing(&lt=3B*UN=
>USED*&gt=3Bmp : MarkerPromise) =3D BEGIN END FulfilNothing=3B<br>&gt=3B + <=
>br>&gt=3B + TYPE SeqClosure =3D Thread.Closure OBJECT seq : RefSeq.T=3B OVE=
>RRIDES apply :=3D SeqApply END=3B<br>&gt=3B + <br>&gt=3B + PROCEDURE SeqApp=
>ly(cl : SeqClosure) : REFANY =3D<br>&gt=3B +   BEGIN<br>&gt=3B +     FOR i =
>:=3D 0 TO cl.seq.size()-1 DO<br>&gt=3B +       WITH p =3D NARROW(cl.seq.get=
>(i)=2CQMachine.Promise) DO<br>&gt=3B +         p.fulfil()<br>&gt=3B +      =
> END<br>&gt=3B +     END=3B<br>&gt=3B +     RETURN NIL<br>&gt=3B +   END Se=
>qApply=3B<br>&gt=3B + <br>&gt=3B   PROCEDURE CompileEverything (s: State=3B=
>  schedule: SourceList) =3D<br>&gt=3B     VAR u: M3Unit.T=3B<br>&gt=3B     =
>BEGIN<br>&gt=3B ***************<br>&gt=3B *** 941=2C948 ****<br>&gt=3B   <b=
>r>&gt=3B       (* compile all the sources using the initial schedule *)<br>=
>&gt=3B       FOR i :=3D 0 TO LAST (schedule^) DO<br>&gt=3B !       CompileO=
>ne (s=2C schedule[i])=3B<br>&gt=3B       END=3B<br>&gt=3B       FlushPendin=
>g (s)=3B<br>&gt=3B   <br>&gt=3B       (* recompile any interfaces where we =
>goofed on the exports *)<br>&gt=3B --- 960=2C1000 ----<br>&gt=3B   <br>&gt=
>=3B       (* compile all the sources using the initial schedule *)<br>&gt=
>=3B       FOR i :=3D 0 TO LAST (schedule^) DO<br>&gt=3B !       s.delayBack=
>end :=3D TRUE=3B<br>&gt=3B !       TRY<br>&gt=3B !         CompileOne (s=2C=
> schedule[i])=3B<br>&gt=3B !       FINALLY<br>&gt=3B !         s.delayBacke=
>nd :=3D FALSE=3B<br>&gt=3B !       END=3B<br>&gt=3B ! <br>&gt=3B !       s.=
>machine.promises.addhi(NEW(MarkerPromise))=3B<br>&gt=3B ! <br>&gt=3B !     =
>END=3B<br>&gt=3B ! <br>&gt=3B !     VAR<br>&gt=3B !       curSeq  :=3D NEW(=
>RefSeq.T).init()=3B<br>&gt=3B !       threads :=3D NEW(RefSeq.T).init()=3B<=
>br>&gt=3B !     BEGIN<br>&gt=3B !       FOR i :=3D 0 TO s.machine.promises.=
>size()-1 DO<br>&gt=3B !         WITH p =3D s.machine.promises.get(i) DO<br>=
>&gt=3B !           curSeq.addhi(p)=3B<br>&gt=3B !           IF i =3D s.mach=
>ine.promises.size()-1 OR ISTYPE(p=2CMarkerPromise) THEN<br>&gt=3B !        =
>     WITH cl =3D NEW(SeqClosure=2C seq :=3D curSeq) DO<br>&gt=3B !         =
>      threads.addhi (Thread.Fork(cl))=3B<br>&gt=3B ! <br>&gt=3B !          =
>     IF threads.size() &gt=3B 10 THEN EVAL Thread.Join(threads.remlo()) END=
>=3B<br>&gt=3B !               <br>&gt=3B !               curSeq :=3D NEW(Re=
>fSeq.T).init()<br>&gt=3B !             END<br>&gt=3B !           END<br>&gt=
>=3B !         END<br>&gt=3B !       END=3B<br>&gt=3B !       WHILE threads.=
>size() &gt=3B 0 DO EVAL Thread.Join(threads.remlo()) END=3B<br>&gt=3B      =
> END=3B<br>&gt=3B +             <br>&gt=3B +     EVAL s.machine.promises.in=
>it()=3B<br>&gt=3B + <br>&gt=3B + <br>&gt=3B       FlushPending (s)=3B<br>&g=
>t=3B   <br>&gt=3B       (* recompile any interfaces where we goofed on the =
>exports *)<br>&gt=3B ***************<br>&gt=3B *** 1151=2C1156 ****<br>&gt=
>=3B --- 1203=2C1227 ----<br>&gt=3B       END=3B<br>&gt=3B     END CompileM3=
>=3B<br>&gt=3B   <br>&gt=3B + TYPE<br>&gt=3B +   NotePromise =3D QMachine.Pr=
>omise OBJECT<br>&gt=3B +     nam : Pathname.T=3B<br>&gt=3B +   OVERRIDES<br=
>>&gt=3B +     fulfil :=3D FulfilNP=3B<br>&gt=3B +   END=3B<br>&gt=3B + <br>=
>&gt=3B +   RemovePromise =3D QMachine.Promise OBJECT<br>&gt=3B +     nam : =
>Pathname.T=3B<br>&gt=3B +   OVERRIDES<br>&gt=3B +     fulfil :=3D FulfilRP=
>=3B<br>&gt=3B +   END=3B<br>&gt=3B + <br>&gt=3B + PROCEDURE FulfilNP(np : N=
>otePromise) =3D <br>&gt=3B +   BEGIN Utils.NoteTempFile(np.nam) END FulfilN=
>P=3B<br>&gt=3B + <br>&gt=3B + PROCEDURE FulfilRP(rp : RemovePromise) =3D <b=
>r>&gt=3B +   BEGIN Utils.Remove(rp.nam) END FulfilRP=3B<br>&gt=3B + <br>&gt=
>=3B   PROCEDURE PushOneM3 (s: State=3B  u: M3Unit.T): BOOLEAN =3D<br>&gt=3B=
>     VAR<br>&gt=3B       tmpC=2C tmpS: TEXT=3B<br>&gt=3B ***************<br=
>>&gt=3B *** 1191=2C1208 ****<br>&gt=3B   <br>&gt=3B   <br>&gt=3B       | 3 =
>=3D&gt=3B  (* -bootstrap=2C +m3back=2C +asm *)<br>&gt=3B           tmpC :=
>=3D TempCName (u)=3B<br>&gt=3B           tmpS :=3D TempSName (u)=3B<br>&gt=
>=3B           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B<b=
>r>&gt=3B           IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=
>=3B<br>&gt=3B           IF RunM3 (s=2C u=2C tmpC) THEN<br>&gt=3B !         =
>  IF  RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)<br>&gt=3B !   =
>        AND RunAsm (s=2C tmpS=2C u.object) THEN<br>&gt=3B             END=
>=3B<br>&gt=3B             need_merge :=3D TRUE=3B<br>&gt=3B           END=
>=3B<br>&gt=3B           IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=
>=3B<br>&gt=3B           IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=
>=3B<br>&gt=3B   <br>&gt=3B       | 6=2C    (* +bootstrap=2C +m3back=2C -asm=
> *)<br>&gt=3B         7 =3D&gt=3B  (* +bootstrap=2C +m3back=2C +asm *)<br>&=
>gt=3B --- 1262=2C1318 ----<br>&gt=3B   <br>&gt=3B   <br>&gt=3B       | 3 =
>=3D&gt=3B  (* -bootstrap=2C +m3back=2C +asm *)<br>&gt=3B +       IF s.delay=
>Backend THEN<br>&gt=3B           tmpC :=3D TempCName (u)=3B<br>&gt=3B      =
>     tmpS :=3D TempSName (u)=3B<br>&gt=3B + (*<br>&gt=3B           IF (NOT =
>s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B<br>&gt=3B           IF =
>(NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B<br>&gt=3B + *)<br>=
>&gt=3B +         IF (NOT s.keep_files) THEN <br>&gt=3B +           s.machin=
>e.promises.addhi(NEW(NotePromise=2C nam :=3D tmpC)) <br>&gt=3B +         EN=
>D=3B<br>&gt=3B +         IF (NOT s.keep_files) THEN <br>&gt=3B +           =
>s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpS)) <br>&gt=3B +   =
>      END=3B<br>&gt=3B + <br>&gt=3B           IF RunM3 (s=2C u=2C tmpC) THE=
>N<br>&gt=3B !           s.machine.record(TRUE)=3B<br>&gt=3B !           TRY=
><br>&gt=3B !             IF  RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.o=
>ptimize)<br>&gt=3B !             AND RunAsm (s=2C tmpS=2C u.object) THEN<br=
>>&gt=3B !             END=3B<br>&gt=3B !           FINALLY<br>&gt=3B !     =
>        s.machine.record(FALSE)<br>&gt=3B             END=3B<br>&gt=3B + <b=
>r>&gt=3B             need_merge :=3D TRUE=3B<br>&gt=3B           END=3B<br>=
>&gt=3B + (*<br>&gt=3B           IF (NOT s.keep_files) THEN Utils.Remove (tm=
>pC) END=3B<br>&gt=3B           IF (NOT s.keep_files) THEN Utils.Remove (tmp=
>S) END=3B<br>&gt=3B + *)<br>&gt=3B +         IF (NOT s.keep_files) THEN <br=
>>&gt=3B +           s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D =
>tmpC)) <br>&gt=3B +         END=3B<br>&gt=3B +         IF (NOT s.keep_files=
>) THEN <br>&gt=3B +           s.machine.promises.addhi(NEW(RemovePromise=2C=
> nam :=3D tmpS)) <br>&gt=3B +         END=3B<br>&gt=3B + <br>&gt=3B +      =
> ELSE<br>&gt=3B +           tmpC :=3D TempCName (u)=3B<br>&gt=3B +         =
>  tmpS :=3D TempSName (u)=3B<br>&gt=3B +           IF (NOT s.keep_files) TH=
>EN Utils.NoteTempFile (tmpC) END=3B<br>&gt=3B +           IF (NOT s.keep_fi=
>les) THEN Utils.NoteTempFile (tmpS) END=3B<br>&gt=3B +           IF RunM3 (=
>s=2C u=2C tmpC) THEN<br>&gt=3B +             IF  RunM3Back (s=2C tmpC=2C tm=
>pS=2C u.debug=2C u.optimize)<br>&gt=3B +             AND RunAsm (s=2C tmpS=
>=2C u.object) THEN<br>&gt=3B +             END=3B<br>&gt=3B +             n=
>eed_merge :=3D TRUE=3B<br>&gt=3B +           END=3B<br>&gt=3B +           I=
>F (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B<br>&gt=3B +           =
>IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B<br>&gt=3B +       END=
><br>&gt=3B   <br>&gt=3B       | 6=2C    (* +bootstrap=2C +m3back=2C -asm *)=
><br>&gt=3B         7 =3D&gt=3B  (* +bootstrap=2C +m3back=2C +asm *)<br>&gt=
>=3B Index: cm3/src/m3makefile<br>&gt=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D<br>&gt=3B RCS file: /usr/cvs/cm3/m3-sys/cm3/src/m3makefi=
>le=2Cv<br>&gt=3B retrieving revision 1.24<br>&gt=3B diff -c -r1.24 m3makefi=
>le<br>&gt=3B *** cm3/src/m3makefile	8 Dec 2010 07:30:57 -0000	1.24<br
>>&gt=
>=3B --- cm3/src/m3makefile	13 Feb 2011 06:47:42 -0000<br>&gt=3B **********
>*=
>****<br>&gt=3B *** 29=2C34 ****<br>&gt=3B --- 29=2C35 ----<br>&gt=3B   end<=
>br>&gt=3B   module ("Arg")<br>&gt=3B   module ("Builder")<br>&gt=3B   modul=
>e ("Dirs")<br>&gt=3B   module ("M3Build")<br>&gt=3B   module ("M3Loc")<br>&=
>gt=3B Index: m3quake/src/QMachine.i3<br>&gt=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D<br>&gt=3B RCS file: /usr/cvs/cm3/m3-sys/m3quake/sr=
>c/QMachine.i3=2Cv<br>&gt=3B retrieving revision 1.6<br>&gt=3B diff -c -r1.6=
> QMachine.i3<br>&gt=3B *** m3quake/src/QMachine.i3	4 Sep 2009 10:24:07 -00
>0=
>0	1.6<br>&gt=3B --- m3quake/src/QMachine.i3	13 Feb 2011 06:47:43 -0
>000<br>&=
>gt=3B ***************<br>&gt=3B *** 8=2C13 ****<br>&gt=3B --- 8=2C14 ----<b=
>r>&gt=3B   <br>&gt=3B   IMPORT Thread=2C Wr=2C QValue=2C QCode=3B<br>&gt=3B=
>   FROM Quake IMPORT Machine=2C Error=2C ID=2C IDMap=3B<br>&gt=3B + IMPORT =
>RefSeq=3B<br>&gt=3B   <br>&gt=3B   REVEAL<br>&gt=3B     T &lt=3B: T_=3B<br>=
>&gt=3B ***************<br>&gt=3B *** 15=2C20 ****<br>&gt=3B --- 16=2C22 ---=
>-<br>&gt=3B     T =3D Machine=3B<br>&gt=3B     T_ =3D OBJECT<br>&gt=3B     =
>  map: IDMap :=3D NIL=3B (* READONLY *)<br>&gt=3B +     promises : RefSeq.T=
>=3B<br>&gt=3B     METHODS<br>&gt=3B       init      (map: IDMap): T=3B<br>&=
>gt=3B       evaluate  (s: QCode.Stream)                     RAISES {Error=
>=2C Thread.Alerted}=3B<br>&gt=3B ***************<br>&gt=3B *** 37=2C42 ****=
><br>&gt=3B --- 39=2C46 ----<br>&gt=3B       set_wr    (wr: Wr.T)=3B<br>&gt=
>=3B       exec_echo (b: BOOLEAN): BOOLEAN=3B<br>&gt=3B       trace     (b: =
>BOOLEAN)=3B<br>&gt=3B +     <br>&gt=3B +     record(on : BOOLEAN)=3B     (*=
> instead of performing certain acts=2C promise *)<br>&gt=3B     END=3B<br>&=
>gt=3B   <br>&gt=3B   PROCEDURE PushBool (t: T=3B  b: BOOLEAN)=3B<br>&gt=3B =
>***************<br>&gt=3B *** 51=2C54 ****<br>&gt=3B --- 55=2C63 ----<br>&g=
>t=3B   <br>&gt=3B   PROCEDURE GetEnv (default=2C v0=2C v1=2C v2=2C v3=2C v4=
>: TEXT :=3D NIL): TEXT=3B<br>&gt=3B   <br>&gt=3B + TYPE Promise =3D OBJECT =
>METHODS fulfil() RAISES { Error }  END=3B<br>&gt=3B + <br>&gt=3B   END QMac=
>hine.<br>&gt=3B + <br>&gt=3B + <br>&gt=3B + <br>&gt=3B Index: m3quake/src/Q=
>Machine.m3<br>&gt=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br=
>>&gt=3B RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.m3=2Cv<br>&gt=3B=
> retrieving revision 1.35<br>&gt=3B diff -c -r1.35 QMachine.m3<br>&gt=3B **=
>* m3quake/src/QMachine.m3	3 Aug 2010 09:40:04 -0000	1.35<br>&gt=3B 
>--- m3qu=
>ake/src/QMachine.m3	13 Feb 2011 06:47:44 -0000<br>&gt=3B ***************<br
>=
>>&gt=3B *** 16=2C21 ****<br>&gt=3B --- 16=2C22 ----<br>&gt=3B   IMPORT Text=
>Utils=2C FSUtils=2C System=2C DirStack=3B (* sysutils *)<br>&gt=3B   IMPORT=
> Compiler=3B<br>&gt=3B   IMPORT M3Path=3B<br>&gt=3B + IMPORT RefSeq=3B<br>&=
>gt=3B   <br>&gt=3B   CONST<br>&gt=3B     OnUnix =3D (Compiler.ThisOS =3D Co=
>mpiler.OS.POSIX)=3B<br>&gt=3B ***************<br>&gt=3B *** 44=2C49 ****<br=
>>&gt=3B --- 45=2C52 ----<br>&gt=3B       shell     : TEXT         :=3D NIL=
>=3B<br>&gt=3B       sh_option : TEXT         :=3D NIL=3B<br>&gt=3B       tm=
>p_dir   : TEXT         :=3D NIL=3B<br>&gt=3B + <br>&gt=3B +     doRecord :=
>=3D FALSE=3B<br>&gt=3B     OVERRIDES<br>&gt=3B       init      :=3D Init=3B=
><br>&gt=3B       evaluate  :=3D Evaluate=3B<br>&gt=3B ***************<br>&g=
>t=3B *** 66=2C73 ****<br>&gt=3B --- 69=2C81 ----<br>&gt=3B       set_wr    =
>:=3D SetWr=3B<br>&gt=3B       exec_echo :=3D ExecEcho=3B<br>&gt=3B       tr=
>ace     :=3D Trace=3B<br>&gt=3B + <br>&gt=3B +     record :=3D Record=3B<br=
>>&gt=3B     END=3B<br>&gt=3B   <br>&gt=3B + PROCEDURE Record(t : T=3B on : =
>BOOLEAN) =3D <br>&gt=3B +   BEGIN t.doRecord :=3D on END Record=3B<br>&gt=
>=3B + <br>&gt=3B   TYPE<br>&gt=3B     Registers =3D RECORD<br>&gt=3B       =
>cp : QCode.Stream   :=3D NIL=3B (* code pointer *)<br>&gt=3B **************=
>*<br>&gt=3B *** 139=2C144 ****<br>&gt=3B --- 147=2C154 ----<br>&gt=3B      =
> t.globals    :=3D NEW (IntRefTbl.Default).init ()=3B<br>&gt=3B       t.def=
>ault_wr :=3D Stdio.stdout=3B<br>&gt=3B   <br>&gt=3B +     t.promises :=3D N=
>EW(RefSeq.T).init()=3B<br>&gt=3B + <br>&gt=3B       InitOSEnv (t)=3B<br>&gt=
>=3B       InitBuiltins (t)=3B<br>&gt=3B   <br>&gt=3B ***************<br>&gt=
>=3B *** 1555=2C1564 ****<br>&gt=3B             END=3B<br>&gt=3B           E=
>LSE<br>&gt=3B             FlushIO ()=3B<br>&gt=3B !           Process.GetSt=
>andardFileHandles (stdin=2C stdout=2C stderr)=3B<br>&gt=3B !           hand=
>le :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=2C n_shell_args)=2C<=
>br>&gt=3B !                                     stdin :=3D stdin=2C stdout =
>:=3D stdout=2C<br>&gt=3B !                                     stderr :=3D =
>stderr)=3B<br>&gt=3B           END=3B<br>&gt=3B         EXCEPT<br>&gt=3B   =
>      | Thread.Alerted =3D&gt=3B<br>&gt=3B --- 1565=2C1594 ----<br>&gt=3B  =
>           END=3B<br>&gt=3B           ELSE<br>&gt=3B             FlushIO ()=
>=3B<br>&gt=3B !           IF t.doRecord THEN<br>&gt=3B !             handle=
> :=3D NIL=3B<br>&gt=3B !             WITH a =3D NEW(REF ARRAY OF TEXT=2C n_=
>shell_args) DO<br>&gt=3B !               a^ :=3D SUBARRAY(args=2C0=2Cn_shel=
>l_args)=3B<br>&gt=3B !               VAR wrx : Wr.T=3B BEGIN<br>&gt=3B !   =
>              IF echo OR t.do_echo THEN<br>&gt=3B !                   wrx :=
>=3D wr<br>&gt=3B !                 ELSE<br>&gt=3B !                   wrx :=
>=3D NIL<br>&gt=3B !                 END=3B<br>&gt=3B !                 t.pr=
>omises.addhi(NEW(ExecPromise=2C<br>&gt=3B !                                =
>      cmd :=3D t.shell=2C<br>&gt=3B !                                      =
>wr :=3D wrx=2C<br>&gt=3B !                                      args :=3D a=
>=2C<br>&gt=3B !                                      t :=3D t=2C<br>&gt=3B =
>!                                      ignore_errors :=3D ignore_errors))<b=
>r>&gt=3B !               END<br>&gt=3B !             END<br>&gt=3B !       =
>    ELSE<br>&gt=3B !             Process.GetStandardFileHandles (stdin=2C s=
>tdout=2C stderr)=3B<br>&gt=3B !             handle :=3D Process.Create (t.s=
>hell=2C SUBARRAY (args=2C 0=2C n_shell_args)=2C<br>&gt=3B !                =
>                       stdin :=3D stdin=2C stdout :=3D stdout=2C<br>&gt=3B =
>!                                       stderr :=3D stderr)=3B<br>&gt=3B ! =
>          END=3B<br>&gt=3B           END=3B<br>&gt=3B         EXCEPT<br>&gt=
>=3B         | Thread.Alerted =3D&gt=3B<br>&gt=3B ***************<br>&gt=3B =
>*** 1573=2C1579 ****<br>&gt=3B         END=3B<br>&gt=3B   <br>&gt=3B       =
>  (* wait for everything to shutdown... *)<br>&gt=3B !       exit_code :=3D=
> Process.Wait (handle)=3B<br>&gt=3B       END=3B<br>&gt=3B   <br>&gt=3B    =
>   IF onlyTry THEN<br>&gt=3B --- 1603=2C1613 ----<br>&gt=3B         END=3B<=
>br>&gt=3B   <br>&gt=3B         (* wait for everything to shutdown... *)<br>=
>&gt=3B !       IF handle =3D NIL THEN<br>&gt=3B !         exit_code :=3D 0 =
><br>&gt=3B !       ELSE<br>&gt=3B !         exit_code :=3D Process.Wait (ha=
>ndle)=3B<br>&gt=3B !       END=3B (* else we're only promising *)<br>&gt=3B=
>       END=3B<br>&gt=3B   <br>&gt=3B       IF onlyTry THEN<br>&gt=3B ******=
>*********<br>&gt=3B *** 1589=2C1594 ****<br>&gt=3B --- 1623=2C1664 ----<br>=
>&gt=3B   <br>&gt=3B     END ExecCommand=3B<br>&gt=3B   <br>&gt=3B + TYPE <b=
>r>&gt=3B +   ExecPromise =3D Promise OBJECT<br>&gt=3B +     cmd : TEXT=3B<b=
>r>&gt=3B +     args : REF ARRAY OF TEXT=3B<br>&gt=3B +     t : T=3B<br>&gt=
>=3B +     wr : Wr.T=3B<br>&gt=3B +     ignore_errors : BOOLEAN=3B<br>&gt=3B=
> +   OVERRIDES<br>&gt=3B +     fulfil :=3D FulfilExecPromise=3B<br>&gt=3B +=
>   END=3B<br>&gt=3B + <br>&gt=3B + PROCEDURE FulfilExecPromise(ep : ExecPro=
>mise) RAISES { Error } =3D <br>&gt=3B +   VAR<br>&gt=3B +     stdin=2C stdo=
>ut=2C stderr: File.T=3B<br>&gt=3B +   BEGIN<br>&gt=3B +     Process.GetStan=
>dardFileHandles (stdin=2C stdout=2C stderr)=3B<br>&gt=3B +     TRY<br>&gt=
>=3B +       IF ep.wr # NIL THEN<br>&gt=3B +         Wr.PutText (ep.wr=2C ep=
>.args[1])=3B<br>&gt=3B +         Wr.PutText (ep.wr=2C Wr.EOL)=3B<br>&gt=3B =
>+         FlushIO ()=3B<br>&gt=3B +       END=3B<br>&gt=3B +       WITH han=
>dle =3D Process.Create (ep.cmd=2C ep.args^=2C<br>&gt=3B +                  =
>                   stdin :=3D stdin=2C stdout :=3D stdout=2C<br>&gt=3B +   =
>                                  stderr :=3D stderr)=2C<br>&gt=3B +       =
>     exit_code =3D Process.Wait(handle) DO<br>&gt=3B +         IF exit_code=
> # 0 AND NOT ep.ignore_errors THEN<br>&gt=3B +           Err (ep.t=2C Fmt.F=
>("exit %s: %s"=2C Fmt.Int(exit_code)=2C ep.cmd))<br>&gt=3B +         END<br=
>>&gt=3B +       END<br>&gt=3B +     EXCEPT<br>&gt=3B +       OSError.E (ec)=
> =3D&gt=3B<br>&gt=3B +           Err (ep.t=2C Fmt.F ("exec failed%s *** %s"=
>=2C OSErr (ec)=2C ep.cmd))=3B<br>&gt=3B +     END<br>&gt=3B +   END FulfilE=
>xecPromise=3B<br>&gt=3B + <br>&gt=3B   PROCEDURE KillProcess (handle: Proce=
>ss.T) =3D<br>&gt=3B     BEGIN<br>&gt=3B       IF (handle # NIL) THEN<br>&gt=
>=3B <br> 		 	   		  </body>
></html>=
>
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_--



More information about the M3devel mailing list