[M3devel] More on threading
Mika Nystrom
mika at async.caltech.edu
Sun Feb 13 19:17:17 CET 2011
Jay K writes:
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>
>Yes=2C gcc is slow and using as is wasteful.
>Yes this should all be multi-threaded.
>However it seems to me that m3front has many globals and therefore
>really making things multi-threaded isn't going to be easy.
OK but as long as the back-end is taking up most of the cycles
the property that the output of the front-end is used only by cmcg1
(m3back/m3cgc/whatever you call it) and the assembler means that what I
did works for a pretty big speedup. In fact you can see that it works
just fine (from the point of view of generating the correct output) since
you can re-order running the backend with respect to all the steps done
by the front-end. What you lose is (perhaps) some error messages, and of
course, in general, any implicit dependencies between the error messages.
I don't understand why the program gets stuck in uxmxn though. Maybe a
problem with, sigh, user threads...
>I think you are mostly wasting your time on user threads though.
I hope so... I too would really like to use pthreads.
>
>
>Have you tried the NT/x86 system?
>It is very fast.
>
>
>In the gcc system=2C each .mc file is run through m3cgc1 to produce an .ms =
>file=2C
>which is run through as to produce an object file .mo.
>
>
>In the NT/x86 system=2C cm3 outputs .obj files directly.
>No .mc files. No m3cgc1 or as.
>Granted=2C I'd just as soon output .c files=2C and hope for a C compiler th=
>at doesn't run a separate assembler.
The NT compiler sounds faster but of course it's easier to parallelize the
unix one using the method I outlined :-) Well unless you're using pthreads
of course, then you can just fork a new thread for the integrated NT backend
as long as it doesn't share too many variables with the front-end.
But no I don't use Windows. I was asking on here how to get started with
Modula-3 on Windows once but my machine was/is a Windows 2000 system and
apparently CM3 no longer works on win2k..?
I used to use PM3/Klagenfurt a lot on Windows (Cygwin) but thankfully
my clients on that project let me switch the system to Unix so I don't
have to touch that anymore.
>
>
>My timing might loosen up a bit and I hope to spend some on the thread stre=
>ss tester using pthreads. Next weekend.
>Am I right though that Darwin works fine?
>Solaris? FreeBSD? NetBSD? OpenBSD? Linux? I know=2C not Linux.
>You generally don't need any of these systems.
That would be great. I don't think pthreads works properly anywhere at
the moment. My guess is that it never has, if you stress it hard enough.
I first ran into problems on Darwin and then switched to AMD64 simply
because I have better access to faster machines with AMD64. At first
there were some obvious problems that Tony fixed quickly but then I
kept running into odd behavior with my application and I added various
other tests until I could reproduce everything I'd seen go wrong with my
application, and I think at that point I was seeing problems on every
platform (except user threads). The uxmxn problem I see with user
threads on FreeBSD suggests that perhaps I need to add some more tests,
however.
Mika
>All of the x86/AMD64 systems can run in a virtual machine.
>Solaris/sparc we have good access.
>
>
> - Jay
>
>
>> To: m3devel at elegosoft.com
>> Date: Sat=2C 12 Feb 2011 23:04:14 -0800
>> From: mika at async.caltech.edu
>> CC: jay.krell at cornell.edu
>> Subject: [M3devel] More on threading
>>=20
>>=20
>> Hi again m3devel=2C
>>=20
>> Well I thought I would relate another story having to do with threading
>> and CM3. It's half-baked at the moment=2C but it only reflects a few hou=
>rs'
>> work on my part=2C sorry... I'm about to give up now because I don't know
>> if the problem is in my code=2C or in threading=2C or what... maybe it is
>> the atfork problem Jay talked about?
>>=20
>> Programmers spend a lot of time waiting for their compilers to run=2C so
>> it would be nice if the compilation process were faster. After perusing
>> the source code for the CM3 I realized that there is no feedback at all
>> from the back-end and assembler steps to the rest of the compiler=2C and
>> I believe that with the GCC back-end the vast majority of the runtime
>> of the compiler is spent in these stages.
>>=20
>> So.... how hard could this be to parallelize? I decided to try making
>> the two stages (cm3cg1 and as) into "promises" rather than running them
>> on the spot. The QMachine adds a promise to call the back-end to a
>> RefSeq.T called "promises"=2C to which the Builder can also add promises.
>> Then=2C after making all the promises=2C collect them and run them ("forc=
>e
>> them" a true Schemer would say) in parallel.
>>=20
>> Of course something goes wrong somewhere. With 10 threads I am able to
>> run 31 seconds of CPU time in 19 seconds of wallclock time (using an
>> average 160% of CPU) but then something goes wrong and my process
>> is stuck in state "uxmxn". This is using user threads on AMD64_FREEBSD
>> on a machine with four CPUs.
>>=20
>> Mika
>>=20
>> P.S. diff attached=2C it's a bit ugly=2C but maybe it works on some other
>> system.... the code works just fine if you run one thread at a time.
>> I.e.=2C if you change
>>=20
>> IF threads.size() > 10 THEN EVAL Thread.Join(threads.remlo()) END=3B
>>=20
>> to
>>=20
>> IF threads.size() > 0 THEN EVAL Thread.Join(threads.remlo()) END=3B
>>=20
>>=20
>>=20
>> Index: cm3/src/Builder.m3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/cm3/src/Builder.m3=2Cv
>> retrieving revision 1.36
>> diff -c -r1.36 Builder.m3
>> *** cm3/src/Builder.m3 24 Aug 2010 05:24:24 -0000 1.36
>> --- cm3/src/Builder.m3 13 Feb 2011 06:47:40 -0000
>> ***************
>> *** 15=2C20 ****
>> --- 15=2C22 ----
>> IMPORT QIdent=3B
>> FROM Target IMPORT M3BackendMode_t=2C BackendAssembly=2C BackendModeStr=
>ings=3B
>> FROM M3Path IMPORT OSKind=2C OSKindStrings=3B
>> + IMPORT Pathname=3B
>> + IMPORT RefSeq=3B
>> =20
>> TYPE
>> UK =3D M3Unit.Kind=3B
>> ***************
>> *** 131=2C136 ****
>> --- 133=2C139 ----
>> link_coverage : TEXT=3B (* coverage library *)
>> m3_front_flags: Arg.List=3B (* configuration options for =
>the front *)
>> m3_options : Arg.List=3B (* misc. user options for the=
> frontend *)
>> + delayBackend :=3D FALSE=3B
>> END=3B
>> =20
>> TYPE
>> ***************
>> *** 932=2C937 ****
>> --- 935=2C956 ----
>> =20
>> (*------------------------------------------------------------ compilat=
>ion --*)
>> =20
>> + TYPE MarkerPromise =3D QMachine.Promise BRANDED OBJECT OVERRIDES fulfil=
> :=3D FulfilNothing END=3B
>> +=20
>> + PROCEDURE FulfilNothing(<*UNUSED*>mp : MarkerPromise) =3D BEGIN END Ful=
>filNothing=3B
>> +=20
>> + TYPE SeqClosure =3D Thread.Closure OBJECT seq : RefSeq.T=3B OVERRIDES a=
>pply :=3D SeqApply END=3B
>> +=20
>> + PROCEDURE SeqApply(cl : SeqClosure) : REFANY =3D
>> + BEGIN
>> + FOR i :=3D 0 TO cl.seq.size()-1 DO
>> + WITH p =3D NARROW(cl.seq.get(i)=2CQMachine.Promise) DO
>> + p.fulfil()
>> + END
>> + END=3B
>> + RETURN NIL
>> + END SeqApply=3B
>> +=20
>> PROCEDURE CompileEverything (s: State=3B schedule: SourceList) =3D
>> VAR u: M3Unit.T=3B
>> BEGIN
>> ***************
>> *** 941=2C948 ****
>> =20
>> (* compile all the sources using the initial schedule *)
>> FOR i :=3D 0 TO LAST (schedule^) DO
>> ! CompileOne (s=2C schedule[i])=3B
>> END=3B
>> FlushPending (s)=3B
>> =20
>> (* recompile any interfaces where we goofed on the exports *)
>> --- 960=2C1000 ----
>> =20
>> (* compile all the sources using the initial schedule *)
>> FOR i :=3D 0 TO LAST (schedule^) DO
>> ! s.delayBackend :=3D TRUE=3B
>> ! TRY
>> ! CompileOne (s=2C schedule[i])=3B
>> ! FINALLY
>> ! s.delayBackend :=3D FALSE=3B
>> ! END=3B
>> !=20
>> ! s.machine.promises.addhi(NEW(MarkerPromise))=3B
>> !=20
>> ! END=3B
>> !=20
>> ! VAR
>> ! curSeq :=3D NEW(RefSeq.T).init()=3B
>> ! threads :=3D NEW(RefSeq.T).init()=3B
>> ! BEGIN
>> ! FOR i :=3D 0 TO s.machine.promises.size()-1 DO
>> ! WITH p =3D s.machine.promises.get(i) DO
>> ! curSeq.addhi(p)=3B
>> ! IF i =3D s.machine.promises.size()-1 OR ISTYPE(p=2CMarkerProm=
>ise) THEN
>> ! WITH cl =3D NEW(SeqClosure=2C seq :=3D curSeq) DO
>> ! threads.addhi (Thread.Fork(cl))=3B
>> !=20
>> ! IF threads.size() > 10 THEN EVAL Thread.Join(threads.reml=
>o()) END=3B
>> ! =20
>> ! curSeq :=3D NEW(RefSeq.T).init()
>> ! END
>> ! END
>> ! END
>> ! END=3B
>> ! WHILE threads.size() > 0 DO EVAL Thread.Join(threads.remlo()) END=
>=3B
>> END=3B
>> + =20
>> + EVAL s.machine.promises.init()=3B
>> +=20
>> +=20
>> FlushPending (s)=3B
>> =20
>> (* recompile any interfaces where we goofed on the exports *)
>> ***************
>> *** 1151=2C1156 ****
>> --- 1203=2C1227 ----
>> END=3B
>> END CompileM3=3B
>> =20
>> + TYPE
>> + NotePromise =3D QMachine.Promise OBJECT
>> + nam : Pathname.T=3B
>> + OVERRIDES
>> + fulfil :=3D FulfilNP=3B
>> + END=3B
>> +=20
>> + RemovePromise =3D QMachine.Promise OBJECT
>> + nam : Pathname.T=3B
>> + OVERRIDES
>> + fulfil :=3D FulfilRP=3B
>> + END=3B
>> +=20
>> + PROCEDURE FulfilNP(np : NotePromise) =3D=20
>> + BEGIN Utils.NoteTempFile(np.nam) END FulfilNP=3B
>> +=20
>> + PROCEDURE FulfilRP(rp : RemovePromise) =3D=20
>> + BEGIN Utils.Remove(rp.nam) END FulfilRP=3B
>> +=20
>> PROCEDURE PushOneM3 (s: State=3B u: M3Unit.T): BOOLEAN =3D
>> VAR
>> tmpC=2C tmpS: TEXT=3B
>> ***************
>> *** 1191=2C1208 ****
>> =20
>> =20
>> | 3 =3D> (* -bootstrap=2C +m3back=2C +asm *)
>> tmpC :=3D TempCName (u)=3B
>> tmpS :=3D TempSName (u)=3B
>> IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>> IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>> IF RunM3 (s=2C u=2C tmpC) THEN
>> ! IF RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> ! AND RunAsm (s=2C tmpS=2C u.object) THEN
>> END=3B
>> need_merge :=3D TRUE=3B
>> END=3B
>> IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>> IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>> =20
>> | 6=2C (* +bootstrap=2C +m3back=2C -asm *)
>> 7 =3D> (* +bootstrap=2C +m3back=2C +asm *)
>> --- 1262=2C1318 ----
>> =20
>> =20
>> | 3 =3D> (* -bootstrap=2C +m3back=2C +asm *)
>> + IF s.delayBackend THEN
>> tmpC :=3D TempCName (u)=3B
>> tmpS :=3D TempSName (u)=3B
>> + (*
>> IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>> IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>> + *)
>> + IF (NOT s.keep_files) THEN=20
>> + s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpC))=20
>> + END=3B
>> + IF (NOT s.keep_files) THEN=20
>> + s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpS))=20
>> + END=3B
>> +=20
>> IF RunM3 (s=2C u=2C tmpC) THEN
>> ! s.machine.record(TRUE)=3B
>> ! TRY
>> ! IF RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> ! AND RunAsm (s=2C tmpS=2C u.object) THEN
>> ! END=3B
>> ! FINALLY
>> ! s.machine.record(FALSE)
>> END=3B
>> +=20
>> need_merge :=3D TRUE=3B
>> END=3B
>> + (*
>> IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>> IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>> + *)
>> + IF (NOT s.keep_files) THEN=20
>> + s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D tmpC))=
>=20
>> + END=3B
>> + IF (NOT s.keep_files) THEN=20
>> + s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D tmpS))=
>=20
>> + END=3B
>> +=20
>> + ELSE
>> + tmpC :=3D TempCName (u)=3B
>> + tmpS :=3D TempSName (u)=3B
>> + IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B
>> + IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B
>> + IF RunM3 (s=2C u=2C tmpC) THEN
>> + IF RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)
>> + AND RunAsm (s=2C tmpS=2C u.object) THEN
>> + END=3B
>> + need_merge :=3D TRUE=3B
>> + END=3B
>> + IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B
>> + IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B
>> + END
>> =20
>> | 6=2C (* +bootstrap=2C +m3back=2C -asm *)
>> 7 =3D> (* +bootstrap=2C +m3back=2C +asm *)
>> Index: cm3/src/m3makefile
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/cm3/src/m3makefile=2Cv
>> retrieving revision 1.24
>> diff -c -r1.24 m3makefile
>> *** cm3/src/m3makefile 8 Dec 2010 07:30:57 -0000 1.24
>> --- cm3/src/m3makefile 13 Feb 2011 06:47:42 -0000
>> ***************
>> *** 29=2C34 ****
>> --- 29=2C35 ----
>> end
>> module ("Arg")
>> module ("Builder")
>> module ("Dirs")
>> module ("M3Build")
>> module ("M3Loc")
>> Index: m3quake/src/QMachine.i3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.i3=2Cv
>> retrieving revision 1.6
>> diff -c -r1.6 QMachine.i3
>> *** m3quake/src/QMachine.i3 4 Sep 2009 10:24:07 -0000 1.6
>> --- m3quake/src/QMachine.i3 13 Feb 2011 06:47:43 -0000
>> ***************
>> *** 8=2C13 ****
>> --- 8=2C14 ----
>> =20
>> IMPORT Thread=2C Wr=2C QValue=2C QCode=3B
>> FROM Quake IMPORT Machine=2C Error=2C ID=2C IDMap=3B
>> + IMPORT RefSeq=3B
>> =20
>> REVEAL
>> T <: T_=3B
>> ***************
>> *** 15=2C20 ****
>> --- 16=2C22 ----
>> T =3D Machine=3B
>> T_ =3D OBJECT
>> map: IDMap :=3D NIL=3B (* READONLY *)
>> + promises : RefSeq.T=3B
>> METHODS
>> init (map: IDMap): T=3B
>> evaluate (s: QCode.Stream) RAISES {Error=2C Th=
>read.Alerted}=3B
>> ***************
>> *** 37=2C42 ****
>> --- 39=2C46 ----
>> set_wr (wr: Wr.T)=3B
>> exec_echo (b: BOOLEAN): BOOLEAN=3B
>> trace (b: BOOLEAN)=3B
>> + =20
>> + record(on : BOOLEAN)=3B (* instead of performing certain acts=
>=2C promise *)
>> END=3B
>> =20
>> PROCEDURE PushBool (t: T=3B b: BOOLEAN)=3B
>> ***************
>> *** 51=2C54 ****
>> --- 55=2C63 ----
>> =20
>> PROCEDURE GetEnv (default=2C v0=2C v1=2C v2=2C v3=2C v4: TEXT :=3D NIL)=
>: TEXT=3B
>> =20
>> + TYPE Promise =3D OBJECT METHODS fulfil() RAISES { Error } END=3B
>> +=20
>> END QMachine.
>> +=20
>> +=20
>> +=20
>> Index: m3quake/src/QMachine.m3
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.m3=2Cv
>> retrieving revision 1.35
>> diff -c -r1.35 QMachine.m3
>> *** m3quake/src/QMachine.m3 3 Aug 2010 09:40:04 -0000 1.35
>> --- m3quake/src/QMachine.m3 13 Feb 2011 06:47:44 -0000
>> ***************
>> *** 16=2C21 ****
>> --- 16=2C22 ----
>> IMPORT TextUtils=2C FSUtils=2C System=2C DirStack=3B (* sysutils *)
>> IMPORT Compiler=3B
>> IMPORT M3Path=3B
>> + IMPORT RefSeq=3B
>> =20
>> CONST
>> OnUnix =3D (Compiler.ThisOS =3D Compiler.OS.POSIX)=3B
>> ***************
>> *** 44=2C49 ****
>> --- 45=2C52 ----
>> shell : TEXT :=3D NIL=3B
>> sh_option : TEXT :=3D NIL=3B
>> tmp_dir : TEXT :=3D NIL=3B
>> +=20
>> + doRecord :=3D FALSE=3B
>> OVERRIDES
>> init :=3D Init=3B
>> evaluate :=3D Evaluate=3B
>> ***************
>> *** 66=2C73 ****
>> --- 69=2C81 ----
>> set_wr :=3D SetWr=3B
>> exec_echo :=3D ExecEcho=3B
>> trace :=3D Trace=3B
>> +=20
>> + record :=3D Record=3B
>> END=3B
>> =20
>> + PROCEDURE Record(t : T=3B on : BOOLEAN) =3D=20
>> + BEGIN t.doRecord :=3D on END Record=3B
>> +=20
>> TYPE
>> Registers =3D RECORD
>> cp : QCode.Stream :=3D NIL=3B (* code pointer *)
>> ***************
>> *** 139=2C144 ****
>> --- 147=2C154 ----
>> t.globals :=3D NEW (IntRefTbl.Default).init ()=3B
>> t.default_wr :=3D Stdio.stdout=3B
>> =20
>> + t.promises :=3D NEW(RefSeq.T).init()=3B
>> +=20
>> InitOSEnv (t)=3B
>> InitBuiltins (t)=3B
>> =20
>> ***************
>> *** 1555=2C1564 ****
>> END=3B
>> ELSE
>> FlushIO ()=3B
>> ! Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=3B
>> ! handle :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=2C=
> n_shell_args)=2C
>> ! stdin :=3D stdin=2C stdout :=3D std=
>out=2C
>> ! stderr :=3D stderr)=3B
>> END=3B
>> EXCEPT
>> | Thread.Alerted =3D>
>> --- 1565=2C1594 ----
>> END=3B
>> ELSE
>> FlushIO ()=3B
>> ! IF t.doRecord THEN
>> ! handle :=3D NIL=3B
>> ! WITH a =3D NEW(REF ARRAY OF TEXT=2C n_shell_args) DO
>> ! a^ :=3D SUBARRAY(args=2C0=2Cn_shell_args)=3B
>> ! VAR wrx : Wr.T=3B BEGIN
>> ! IF echo OR t.do_echo THEN
>> ! wrx :=3D wr
>> ! ELSE
>> ! wrx :=3D NIL
>> ! END=3B
>> ! t.promises.addhi(NEW(ExecPromise=2C
>> ! cmd :=3D t.shell=2C
>> ! wr :=3D wrx=2C
>> ! args :=3D a=2C
>> ! t :=3D t=2C
>> ! ignore_errors :=3D ignore_errors))
>> ! END
>> ! END
>> ! ELSE
>> ! Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=
>=3B
>> ! handle :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=
>=2C n_shell_args)=2C
>> ! stdin :=3D stdin=2C stdout :=3D s=
>tdout=2C
>> ! stderr :=3D stderr)=3B
>> ! END=3B
>> END=3B
>> EXCEPT
>> | Thread.Alerted =3D>
>> ***************
>> *** 1573=2C1579 ****
>> END=3B
>> =20
>> (* wait for everything to shutdown... *)
>> ! exit_code :=3D Process.Wait (handle)=3B
>> END=3B
>> =20
>> IF onlyTry THEN
>> --- 1603=2C1613 ----
>> END=3B
>> =20
>> (* wait for everything to shutdown... *)
>> ! IF handle =3D NIL THEN
>> ! exit_code :=3D 0=20
>> ! ELSE
>> ! exit_code :=3D Process.Wait (handle)=3B
>> ! END=3B (* else we're only promising *)
>> END=3B
>> =20
>> IF onlyTry THEN
>> ***************
>> *** 1589=2C1594 ****
>> --- 1623=2C1664 ----
>> =20
>> END ExecCommand=3B
>> =20
>> + TYPE=20
>> + ExecPromise =3D Promise OBJECT
>> + cmd : TEXT=3B
>> + args : REF ARRAY OF TEXT=3B
>> + t : T=3B
>> + wr : Wr.T=3B
>> + ignore_errors : BOOLEAN=3B
>> + OVERRIDES
>> + fulfil :=3D FulfilExecPromise=3B
>> + END=3B
>> +=20
>> + PROCEDURE FulfilExecPromise(ep : ExecPromise) RAISES { Error } =3D=20
>> + VAR
>> + stdin=2C stdout=2C stderr: File.T=3B
>> + BEGIN
>> + Process.GetStandardFileHandles (stdin=2C stdout=2C stderr)=3B
>> + TRY
>> + IF ep.wr # NIL THEN
>> + Wr.PutText (ep.wr=2C ep.args[1])=3B
>> + Wr.PutText (ep.wr=2C Wr.EOL)=3B
>> + FlushIO ()=3B
>> + END=3B
>> + WITH handle =3D Process.Create (ep.cmd=2C ep.args^=2C
>> + stdin :=3D stdin=2C stdout :=3D std=
>out=2C
>> + stderr :=3D stderr)=2C
>> + exit_code =3D Process.Wait(handle) DO
>> + IF exit_code # 0 AND NOT ep.ignore_errors THEN
>> + Err (ep.t=2C Fmt.F("exit %s: %s"=2C Fmt.Int(exit_code)=2C ep.=
>cmd))
>> + END
>> + END
>> + EXCEPT
>> + OSError.E (ec) =3D>
>> + Err (ep.t=2C Fmt.F ("exec failed%s *** %s"=2C OSErr (ec)=2C e=
>p.cmd))=3B
>> + END
>> + END FulfilExecPromise=3B
>> +=20
>> PROCEDURE KillProcess (handle: Process.T) =3D
>> BEGIN
>> IF (handle # NIL) THEN
>>=20
> =
>
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 10pt=3B
>font-family:Tahoma
>}
>--></style>
></head>
><body class=3D'hmmessage'>
>Yes=2C gcc is slow and using as is wasteful.<br>Yes this should all be mult=
>i-threaded.<br>However it seems to me that m3front has many globals and the=
>refore<br>really making things multi-threaded isn't going to be easy.<br>I =
>think you are mostly wasting your time on user threads though.<br><br><br>H=
>ave you tried the NT/x86 system?<br>It is very fast.<br><br><br>In the gcc =
>system=2C each .mc file is run through m3cgc1 to produce an .ms file=2C<br>=
>which is run through as to produce an object file .mo.<br><br><br>In the NT=
>/x86 system=2C cm3 outputs .obj files directly.<br>No .mc files. No m3cgc1 =
>or as.<br>Granted=2C I'd just as soon output .c files=2C and hope for a C c=
>ompiler that doesn't run a separate assembler.<br><br><br>My timing might l=
>oosen up a bit and I hope to spend some on the thread stress tester using p=
>threads. Next weekend.<br>Am I right though that Darwin works fine?<br>Sola=
>ris? FreeBSD? NetBSD? OpenBSD? Linux? I know=2C not Linux.<br>You generally=
> don't need any of these systems.<br>All of the x86/AMD64 systems can run i=
>n a virtual machine.<br>Solaris/sparc we have good access.<br><br><br> =
>=3B- Jay<br><br><br>>=3B To: m3devel at elegosoft.com<br>>=3B Date: Sat=2C=
> 12 Feb 2011 23:04:14 -0800<br>>=3B From: mika at async.caltech.edu<br>>=
>=3B CC: jay.krell at cornell.edu<br>>=3B Subject: [M3devel] More on threadin=
>g<br>>=3B <br>>=3B <br>>=3B Hi again m3devel=2C<br>>=3B <br>>=3B =
>Well I thought I would relate another story having to do with threading<br>=
>>=3B and CM3. It's half-baked at the moment=2C but it only reflects a fe=
>w hours'<br>>=3B work on my part=2C sorry... I'm about to give up now bec=
>ause I don't know<br>>=3B if the problem is in my code=2C or in threading=
>=2C or what... maybe it is<br>>=3B the atfork problem Jay talked about?<b=
>r>>=3B <br>>=3B Programmers spend a lot of time waiting for their compi=
>lers to run=2C so<br>>=3B it would be nice if the compilation process wer=
>e faster. After perusing<br>>=3B the source code for the CM3 I realized =
>that there is no feedback at all<br>>=3B from the back-end and assembler =
>steps to the rest of the compiler=2C and<br>>=3B I believe that with the =
>GCC back-end the vast majority of the runtime<br>>=3B of the compiler is =
>spent in these stages.<br>>=3B <br>>=3B So.... how hard could this be t=
>o parallelize? I decided to try making<br>>=3B the two stages (cm3cg1 an=
>d as) into "promises" rather than running them<br>>=3B on the spot. The =
>QMachine adds a promise to call the back-end to a<br>>=3B RefSeq.T called=
> "promises"=2C to which the Builder can also add promises.<br>>=3B Then=
>=2C after making all the promises=2C collect them and run them ("force<br>&=
>gt=3B them" a true Schemer would say) in parallel.<br>>=3B <br>>=3B Of =
>course something goes wrong somewhere. With 10 threads I am able to<br>>=
>=3B run 31 seconds of CPU time in 19 seconds of wallclock time (using an<br=
>>>=3B average 160% of CPU) but then something goes wrong and my process<b=
>r>>=3B is stuck in state "uxmxn". This is using user threads on AMD64_FR=
>EEBSD<br>>=3B on a machine with four CPUs.<br>>=3B <br>>=3B Mika=
><br>>=3B <br>>=3B P.S. diff attached=2C it's a bit ugly=2C but maybe it=
> works on some other<br>>=3B system.... the code works just fine if you =
>run one thread at a time.<br>>=3B I.e.=2C if you change<br>>=3B <br>>=
>=3B IF threads.size() >=3B 10 THEN EVAL Thread.Join(threads.remlo())=
> END=3B<br>>=3B <br>>=3B to<br>>=3B <br>>=3B IF threads.size()=
> >=3B 0 THEN EVAL Thread.Join(threads.remlo()) END=3B<br>>=3B <br>>=
>=3B <br>>=3B <br>>=3B Index: cm3/src/Builder.m3<br>>=3B =3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br>>=3B RCS file: /usr/cvs/cm3/m3=
>-sys/cm3/src/Builder.m3=2Cv<br>>=3B retrieving revision 1.36<br>>=3B di=
>ff -c -r1.36 Builder.m3<br>>=3B *** cm3/src/Builder.m3 24 Aug 2010 05:
>24:=
>24 -0000 1.36<br>>=3B --- cm3/src/Builder.m3 13 Feb 2011 06:47:40 -0
>000<b=
>r>>=3B ***************<br>>=3B *** 15=2C20 ****<br>>=3B --- 15=2C22 -=
>---<br>>=3B IMPORT QIdent=3B<br>>=3B FROM Target IMPORT M3BackendMo=
>de_t=2C BackendAssembly=2C BackendModeStrings=3B<br>>=3B FROM M3Path IM=
>PORT OSKind=2C OSKindStrings=3B<br>>=3B + IMPORT Pathname=3B<br>>=3B + =
>IMPORT RefSeq=3B<br>>=3B <br>>=3B TYPE<br>>=3B UK =3D M3Unit.=
>Kind=3B<br>>=3B ***************<br>>=3B *** 131=2C136 ****<br>>=3B --=
>- 133=2C139 ----<br>>=3B link_coverage : TEXT=3B (* c=
>overage library *)<br>>=3B m3_front_flags: Arg.List=3B (*=
> configuration options for the front *)<br>>=3B m3_options : Arg=
>.List=3B (* misc. user options for the frontend *)<br>>=3B + =
> delayBackend :=3D FALSE=3B<br>>=3B END=3B<br>>=3B <br>>=3B =
>TYPE<br>>=3B ***************<br>>=3B *** 932=2C937 ****<br>>=3B --- 9=
>35=2C956 ----<br>>=3B <br>>=3B (*----------------------------------=
>-------------------------- compilation --*)<br>>=3B <br>>=3B + TYPE M=
>arkerPromise =3D QMachine.Promise BRANDED OBJECT OVERRIDES fulfil :=3D Fulf=
>ilNothing END=3B<br>>=3B + <br>>=3B + PROCEDURE FulfilNothing(<=3B*UN=
>USED*>=3Bmp : MarkerPromise) =3D BEGIN END FulfilNothing=3B<br>>=3B + <=
>br>>=3B + TYPE SeqClosure =3D Thread.Closure OBJECT seq : RefSeq.T=3B OVE=
>RRIDES apply :=3D SeqApply END=3B<br>>=3B + <br>>=3B + PROCEDURE SeqApp=
>ly(cl : SeqClosure) : REFANY =3D<br>>=3B + BEGIN<br>>=3B + FOR i =
>:=3D 0 TO cl.seq.size()-1 DO<br>>=3B + WITH p =3D NARROW(cl.seq.get=
>(i)=2CQMachine.Promise) DO<br>>=3B + p.fulfil()<br>>=3B + =
> END<br>>=3B + END=3B<br>>=3B + RETURN NIL<br>>=3B + END Se=
>qApply=3B<br>>=3B + <br>>=3B PROCEDURE CompileEverything (s: State=3B=
> schedule: SourceList) =3D<br>>=3B VAR u: M3Unit.T=3B<br>>=3B =
>BEGIN<br>>=3B ***************<br>>=3B *** 941=2C948 ****<br>>=3B <b=
>r>>=3B (* compile all the sources using the initial schedule *)<br>=
>>=3B FOR i :=3D 0 TO LAST (schedule^) DO<br>>=3B ! CompileO=
>ne (s=2C schedule[i])=3B<br>>=3B END=3B<br>>=3B FlushPendin=
>g (s)=3B<br>>=3B <br>>=3B (* recompile any interfaces where we =
>goofed on the exports *)<br>>=3B --- 960=2C1000 ----<br>>=3B <br>>=
>=3B (* compile all the sources using the initial schedule *)<br>>=
>=3B FOR i :=3D 0 TO LAST (schedule^) DO<br>>=3B ! s.delayBack=
>end :=3D TRUE=3B<br>>=3B ! TRY<br>>=3B ! CompileOne (s=2C=
> schedule[i])=3B<br>>=3B ! FINALLY<br>>=3B ! s.delayBacke=
>nd :=3D FALSE=3B<br>>=3B ! END=3B<br>>=3B ! <br>>=3B ! s.=
>machine.promises.addhi(NEW(MarkerPromise))=3B<br>>=3B ! <br>>=3B ! =
>END=3B<br>>=3B ! <br>>=3B ! VAR<br>>=3B ! curSeq :=3D NEW(=
>RefSeq.T).init()=3B<br>>=3B ! threads :=3D NEW(RefSeq.T).init()=3B<=
>br>>=3B ! BEGIN<br>>=3B ! FOR i :=3D 0 TO s.machine.promises.=
>size()-1 DO<br>>=3B ! WITH p =3D s.machine.promises.get(i) DO<br>=
>>=3B ! curSeq.addhi(p)=3B<br>>=3B ! IF i =3D s.mach=
>ine.promises.size()-1 OR ISTYPE(p=2CMarkerPromise) THEN<br>>=3B ! =
> WITH cl =3D NEW(SeqClosure=2C seq :=3D curSeq) DO<br>>=3B ! =
> threads.addhi (Thread.Fork(cl))=3B<br>>=3B ! <br>>=3B ! =
> IF threads.size() >=3B 10 THEN EVAL Thread.Join(threads.remlo()) END=
>=3B<br>>=3B ! <br>>=3B ! curSeq :=3D NEW(Re=
>fSeq.T).init()<br>>=3B ! END<br>>=3B ! END<br>>=
>=3B ! END<br>>=3B ! END=3B<br>>=3B ! WHILE threads.=
>size() >=3B 0 DO EVAL Thread.Join(threads.remlo()) END=3B<br>>=3B =
> END=3B<br>>=3B + <br>>=3B + EVAL s.machine.promises.in=
>it()=3B<br>>=3B + <br>>=3B + <br>>=3B FlushPending (s)=3B<br>&g=
>t=3B <br>>=3B (* recompile any interfaces where we goofed on the =
>exports *)<br>>=3B ***************<br>>=3B *** 1151=2C1156 ****<br>>=
>=3B --- 1203=2C1227 ----<br>>=3B END=3B<br>>=3B END CompileM3=
>=3B<br>>=3B <br>>=3B + TYPE<br>>=3B + NotePromise =3D QMachine.Pr=
>omise OBJECT<br>>=3B + nam : Pathname.T=3B<br>>=3B + OVERRIDES<br=
>>>=3B + fulfil :=3D FulfilNP=3B<br>>=3B + END=3B<br>>=3B + <br>=
>>=3B + RemovePromise =3D QMachine.Promise OBJECT<br>>=3B + nam : =
>Pathname.T=3B<br>>=3B + OVERRIDES<br>>=3B + fulfil :=3D FulfilRP=
>=3B<br>>=3B + END=3B<br>>=3B + <br>>=3B + PROCEDURE FulfilNP(np : N=
>otePromise) =3D <br>>=3B + BEGIN Utils.NoteTempFile(np.nam) END FulfilN=
>P=3B<br>>=3B + <br>>=3B + PROCEDURE FulfilRP(rp : RemovePromise) =3D <b=
>r>>=3B + BEGIN Utils.Remove(rp.nam) END FulfilRP=3B<br>>=3B + <br>>=
>=3B PROCEDURE PushOneM3 (s: State=3B u: M3Unit.T): BOOLEAN =3D<br>>=3B=
> VAR<br>>=3B tmpC=2C tmpS: TEXT=3B<br>>=3B ***************<br=
>>>=3B *** 1191=2C1208 ****<br>>=3B <br>>=3B <br>>=3B | 3 =
>=3D>=3B (* -bootstrap=2C +m3back=2C +asm *)<br>>=3B tmpC :=
>=3D TempCName (u)=3B<br>>=3B tmpS :=3D TempSName (u)=3B<br>>=
>=3B IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B<b=
>r>>=3B IF (NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=
>=3B<br>>=3B IF RunM3 (s=2C u=2C tmpC) THEN<br>>=3B ! =
> IF RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.optimize)<br>>=3B ! =
> AND RunAsm (s=2C tmpS=2C u.object) THEN<br>>=3B END=
>=3B<br>>=3B need_merge :=3D TRUE=3B<br>>=3B END=
>=3B<br>>=3B IF (NOT s.keep_files) THEN Utils.Remove (tmpC) END=
>=3B<br>>=3B IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=
>=3B<br>>=3B <br>>=3B | 6=2C (* +bootstrap=2C +m3back=2C -asm=
> *)<br>>=3B 7 =3D>=3B (* +bootstrap=2C +m3back=2C +asm *)<br>&=
>gt=3B --- 1262=2C1318 ----<br>>=3B <br>>=3B <br>>=3B | 3 =
>=3D>=3B (* -bootstrap=2C +m3back=2C +asm *)<br>>=3B + IF s.delay=
>Backend THEN<br>>=3B tmpC :=3D TempCName (u)=3B<br>>=3B =
> tmpS :=3D TempSName (u)=3B<br>>=3B + (*<br>>=3B IF (NOT =
>s.keep_files) THEN Utils.NoteTempFile (tmpC) END=3B<br>>=3B IF =
>(NOT s.keep_files) THEN Utils.NoteTempFile (tmpS) END=3B<br>>=3B + *)<br>=
>>=3B + IF (NOT s.keep_files) THEN <br>>=3B + s.machin=
>e.promises.addhi(NEW(NotePromise=2C nam :=3D tmpC)) <br>>=3B + EN=
>D=3B<br>>=3B + IF (NOT s.keep_files) THEN <br>>=3B + =
>s.machine.promises.addhi(NEW(NotePromise=2C nam :=3D tmpS)) <br>>=3B + =
> END=3B<br>>=3B + <br>>=3B IF RunM3 (s=2C u=2C tmpC) THE=
>N<br>>=3B ! s.machine.record(TRUE)=3B<br>>=3B ! TRY=
><br>>=3B ! IF RunM3Back (s=2C tmpC=2C tmpS=2C u.debug=2C u.o=
>ptimize)<br>>=3B ! AND RunAsm (s=2C tmpS=2C u.object) THEN<br=
>>>=3B ! END=3B<br>>=3B ! FINALLY<br>>=3B ! =
> s.machine.record(FALSE)<br>>=3B END=3B<br>>=3B + <b=
>r>>=3B need_merge :=3D TRUE=3B<br>>=3B END=3B<br>=
>>=3B + (*<br>>=3B IF (NOT s.keep_files) THEN Utils.Remove (tm=
>pC) END=3B<br>>=3B IF (NOT s.keep_files) THEN Utils.Remove (tmp=
>S) END=3B<br>>=3B + *)<br>>=3B + IF (NOT s.keep_files) THEN <br=
>>>=3B + s.machine.promises.addhi(NEW(RemovePromise=2C nam :=3D =
>tmpC)) <br>>=3B + END=3B<br>>=3B + IF (NOT s.keep_files=
>) THEN <br>>=3B + s.machine.promises.addhi(NEW(RemovePromise=2C=
> nam :=3D tmpS)) <br>>=3B + END=3B<br>>=3B + <br>>=3B + =
> ELSE<br>>=3B + tmpC :=3D TempCName (u)=3B<br>>=3B + =
> tmpS :=3D TempSName (u)=3B<br>>=3B + IF (NOT s.keep_files) TH=
>EN Utils.NoteTempFile (tmpC) END=3B<br>>=3B + IF (NOT s.keep_fi=
>les) THEN Utils.NoteTempFile (tmpS) END=3B<br>>=3B + IF RunM3 (=
>s=2C u=2C tmpC) THEN<br>>=3B + IF RunM3Back (s=2C tmpC=2C tm=
>pS=2C u.debug=2C u.optimize)<br>>=3B + AND RunAsm (s=2C tmpS=
>=2C u.object) THEN<br>>=3B + END=3B<br>>=3B + n=
>eed_merge :=3D TRUE=3B<br>>=3B + END=3B<br>>=3B + I=
>F (NOT s.keep_files) THEN Utils.Remove (tmpC) END=3B<br>>=3B + =
>IF (NOT s.keep_files) THEN Utils.Remove (tmpS) END=3B<br>>=3B + END=
><br>>=3B <br>>=3B | 6=2C (* +bootstrap=2C +m3back=2C -asm *)=
><br>>=3B 7 =3D>=3B (* +bootstrap=2C +m3back=2C +asm *)<br>>=
>=3B Index: cm3/src/m3makefile<br>>=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D<br>>=3B RCS file: /usr/cvs/cm3/m3-sys/cm3/src/m3makefi=
>le=2Cv<br>>=3B retrieving revision 1.24<br>>=3B diff -c -r1.24 m3makefi=
>le<br>>=3B *** cm3/src/m3makefile 8 Dec 2010 07:30:57 -0000 1.24<br
>>>=
>=3B --- cm3/src/m3makefile 13 Feb 2011 06:47:42 -0000<br>>=3B **********
>*=
>****<br>>=3B *** 29=2C34 ****<br>>=3B --- 29=2C35 ----<br>>=3B end<=
>br>>=3B module ("Arg")<br>>=3B module ("Builder")<br>>=3B modul=
>e ("Dirs")<br>>=3B module ("M3Build")<br>>=3B module ("M3Loc")<br>&=
>gt=3B Index: m3quake/src/QMachine.i3<br>>=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D<br>>=3B RCS file: /usr/cvs/cm3/m3-sys/m3quake/sr=
>c/QMachine.i3=2Cv<br>>=3B retrieving revision 1.6<br>>=3B diff -c -r1.6=
> QMachine.i3<br>>=3B *** m3quake/src/QMachine.i3 4 Sep 2009 10:24:07 -00
>0=
>0 1.6<br>>=3B --- m3quake/src/QMachine.i3 13 Feb 2011 06:47:43 -0
>000<br>&=
>gt=3B ***************<br>>=3B *** 8=2C13 ****<br>>=3B --- 8=2C14 ----<b=
>r>>=3B <br>>=3B IMPORT Thread=2C Wr=2C QValue=2C QCode=3B<br>>=3B=
> FROM Quake IMPORT Machine=2C Error=2C ID=2C IDMap=3B<br>>=3B + IMPORT =
>RefSeq=3B<br>>=3B <br>>=3B REVEAL<br>>=3B T <=3B: T_=3B<br>=
>>=3B ***************<br>>=3B *** 15=2C20 ****<br>>=3B --- 16=2C22 ---=
>-<br>>=3B T =3D Machine=3B<br>>=3B T_ =3D OBJECT<br>>=3B =
> map: IDMap :=3D NIL=3B (* READONLY *)<br>>=3B + promises : RefSeq.T=
>=3B<br>>=3B METHODS<br>>=3B init (map: IDMap): T=3B<br>&=
>gt=3B evaluate (s: QCode.Stream) RAISES {Error=
>=2C Thread.Alerted}=3B<br>>=3B ***************<br>>=3B *** 37=2C42 ****=
><br>>=3B --- 39=2C46 ----<br>>=3B set_wr (wr: Wr.T)=3B<br>>=
>=3B exec_echo (b: BOOLEAN): BOOLEAN=3B<br>>=3B trace (b: =
>BOOLEAN)=3B<br>>=3B + <br>>=3B + record(on : BOOLEAN)=3B (*=
> instead of performing certain acts=2C promise *)<br>>=3B END=3B<br>&=
>gt=3B <br>>=3B PROCEDURE PushBool (t: T=3B b: BOOLEAN)=3B<br>>=3B =
>***************<br>>=3B *** 51=2C54 ****<br>>=3B --- 55=2C63 ----<br>&g=
>t=3B <br>>=3B PROCEDURE GetEnv (default=2C v0=2C v1=2C v2=2C v3=2C v4=
>: TEXT :=3D NIL): TEXT=3B<br>>=3B <br>>=3B + TYPE Promise =3D OBJECT =
>METHODS fulfil() RAISES { Error } END=3B<br>>=3B + <br>>=3B END QMac=
>hine.<br>>=3B + <br>>=3B + <br>>=3B + <br>>=3B Index: m3quake/src/Q=
>Machine.m3<br>>=3B =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br=
>>>=3B RCS file: /usr/cvs/cm3/m3-sys/m3quake/src/QMachine.m3=2Cv<br>>=3B=
> retrieving revision 1.35<br>>=3B diff -c -r1.35 QMachine.m3<br>>=3B **=
>* m3quake/src/QMachine.m3 3 Aug 2010 09:40:04 -0000 1.35<br>>=3B
>--- m3qu=
>ake/src/QMachine.m3 13 Feb 2011 06:47:44 -0000<br>>=3B ***************<br
>=
>>>=3B *** 16=2C21 ****<br>>=3B --- 16=2C22 ----<br>>=3B IMPORT Text=
>Utils=2C FSUtils=2C System=2C DirStack=3B (* sysutils *)<br>>=3B IMPORT=
> Compiler=3B<br>>=3B IMPORT M3Path=3B<br>>=3B + IMPORT RefSeq=3B<br>&=
>gt=3B <br>>=3B CONST<br>>=3B OnUnix =3D (Compiler.ThisOS =3D Co=
>mpiler.OS.POSIX)=3B<br>>=3B ***************<br>>=3B *** 44=2C49 ****<br=
>>>=3B --- 45=2C52 ----<br>>=3B shell : TEXT :=3D NIL=
>=3B<br>>=3B sh_option : TEXT :=3D NIL=3B<br>>=3B tm=
>p_dir : TEXT :=3D NIL=3B<br>>=3B + <br>>=3B + doRecord :=
>=3D FALSE=3B<br>>=3B OVERRIDES<br>>=3B init :=3D Init=3B=
><br>>=3B evaluate :=3D Evaluate=3B<br>>=3B ***************<br>&g=
>t=3B *** 66=2C73 ****<br>>=3B --- 69=2C81 ----<br>>=3B set_wr =
>:=3D SetWr=3B<br>>=3B exec_echo :=3D ExecEcho=3B<br>>=3B tr=
>ace :=3D Trace=3B<br>>=3B + <br>>=3B + record :=3D Record=3B<br=
>>>=3B END=3B<br>>=3B <br>>=3B + PROCEDURE Record(t : T=3B on : =
>BOOLEAN) =3D <br>>=3B + BEGIN t.doRecord :=3D on END Record=3B<br>>=
>=3B + <br>>=3B TYPE<br>>=3B Registers =3D RECORD<br>>=3B =
>cp : QCode.Stream :=3D NIL=3B (* code pointer *)<br>>=3B **************=
>*<br>>=3B *** 139=2C144 ****<br>>=3B --- 147=2C154 ----<br>>=3B =
> t.globals :=3D NEW (IntRefTbl.Default).init ()=3B<br>>=3B t.def=
>ault_wr :=3D Stdio.stdout=3B<br>>=3B <br>>=3B + t.promises :=3D N=
>EW(RefSeq.T).init()=3B<br>>=3B + <br>>=3B InitOSEnv (t)=3B<br>>=
>=3B InitBuiltins (t)=3B<br>>=3B <br>>=3B ***************<br>>=
>=3B *** 1555=2C1564 ****<br>>=3B END=3B<br>>=3B E=
>LSE<br>>=3B FlushIO ()=3B<br>>=3B ! Process.GetSt=
>andardFileHandles (stdin=2C stdout=2C stderr)=3B<br>>=3B ! hand=
>le :=3D Process.Create (t.shell=2C SUBARRAY (args=2C 0=2C n_shell_args)=2C<=
>br>>=3B ! stdin :=3D stdin=2C stdout =
>:=3D stdout=2C<br>>=3B ! stderr :=3D =
>stderr)=3B<br>>=3B END=3B<br>>=3B EXCEPT<br>>=3B =
> | Thread.Alerted =3D>=3B<br>>=3B --- 1565=2C1594 ----<br>>=3B =
> END=3B<br>>=3B ELSE<br>>=3B FlushIO ()=
>=3B<br>>=3B ! IF t.doRecord THEN<br>>=3B ! handle=
> :=3D NIL=3B<br>>=3B ! WITH a =3D NEW(REF ARRAY OF TEXT=2C n_=
>shell_args) DO<br>>=3B ! a^ :=3D SUBARRAY(args=2C0=2Cn_shel=
>l_args)=3B<br>>=3B ! VAR wrx : Wr.T=3B BEGIN<br>>=3B ! =
> IF echo OR t.do_echo THEN<br>>=3B ! wrx :=
>=3D wr<br>>=3B ! ELSE<br>>=3B ! wrx :=
>=3D NIL<br>>=3B ! END=3B<br>>=3B ! t.pr=
>omises.addhi(NEW(ExecPromise=2C<br>>=3B ! =
> cmd :=3D t.shell=2C<br>>=3B ! =
>wr :=3D wrx=2C<br>>=3B ! args :=3D a=
>=2C<br>>=3B ! t :=3D t=2C<br>>=3B =
>! ignore_errors :=3D ignore_errors))<b=
>r>>=3B ! END<br>>=3B ! END<br>>=3B ! =
> ELSE<br>>=3B ! Process.GetStandardFileHandles (stdin=2C s=
>tdout=2C stderr)=3B<br>>=3B ! handle :=3D Process.Create (t.s=
>hell=2C SUBARRAY (args=2C 0=2C n_shell_args)=2C<br>>=3B ! =
> stdin :=3D stdin=2C stdout :=3D stdout=2C<br>>=3B =
>! stderr :=3D stderr)=3B<br>>=3B ! =
> END=3B<br>>=3B END=3B<br>>=3B EXCEPT<br>>=
>=3B | Thread.Alerted =3D>=3B<br>>=3B ***************<br>>=3B =
>*** 1573=2C1579 ****<br>>=3B END=3B<br>>=3B <br>>=3B =
> (* wait for everything to shutdown... *)<br>>=3B ! exit_code :=3D=
> Process.Wait (handle)=3B<br>>=3B END=3B<br>>=3B <br>>=3B =
> IF onlyTry THEN<br>>=3B --- 1603=2C1613 ----<br>>=3B END=3B<=
>br>>=3B <br>>=3B (* wait for everything to shutdown... *)<br>=
>>=3B ! IF handle =3D NIL THEN<br>>=3B ! exit_code :=3D 0 =
><br>>=3B ! ELSE<br>>=3B ! exit_code :=3D Process.Wait (ha=
>ndle)=3B<br>>=3B ! END=3B (* else we're only promising *)<br>>=3B=
> END=3B<br>>=3B <br>>=3B IF onlyTry THEN<br>>=3B ******=
>*********<br>>=3B *** 1589=2C1594 ****<br>>=3B --- 1623=2C1664 ----<br>=
>>=3B <br>>=3B END ExecCommand=3B<br>>=3B <br>>=3B + TYPE <b=
>r>>=3B + ExecPromise =3D Promise OBJECT<br>>=3B + cmd : TEXT=3B<b=
>r>>=3B + args : REF ARRAY OF TEXT=3B<br>>=3B + t : T=3B<br>>=
>=3B + wr : Wr.T=3B<br>>=3B + ignore_errors : BOOLEAN=3B<br>>=3B=
> + OVERRIDES<br>>=3B + fulfil :=3D FulfilExecPromise=3B<br>>=3B +=
> END=3B<br>>=3B + <br>>=3B + PROCEDURE FulfilExecPromise(ep : ExecPro=
>mise) RAISES { Error } =3D <br>>=3B + VAR<br>>=3B + stdin=2C stdo=
>ut=2C stderr: File.T=3B<br>>=3B + BEGIN<br>>=3B + Process.GetStan=
>dardFileHandles (stdin=2C stdout=2C stderr)=3B<br>>=3B + TRY<br>>=
>=3B + IF ep.wr # NIL THEN<br>>=3B + Wr.PutText (ep.wr=2C ep=
>.args[1])=3B<br>>=3B + Wr.PutText (ep.wr=2C Wr.EOL)=3B<br>>=3B =
>+ FlushIO ()=3B<br>>=3B + END=3B<br>>=3B + WITH han=
>dle =3D Process.Create (ep.cmd=2C ep.args^=2C<br>>=3B + =
> stdin :=3D stdin=2C stdout :=3D stdout=2C<br>>=3B + =
> stderr :=3D stderr)=2C<br>>=3B + =
> exit_code =3D Process.Wait(handle) DO<br>>=3B + IF exit_code=
> # 0 AND NOT ep.ignore_errors THEN<br>>=3B + Err (ep.t=2C Fmt.F=
>("exit %s: %s"=2C Fmt.Int(exit_code)=2C ep.cmd))<br>>=3B + END<br=
>>>=3B + END<br>>=3B + EXCEPT<br>>=3B + OSError.E (ec)=
> =3D>=3B<br>>=3B + Err (ep.t=2C Fmt.F ("exec failed%s *** %s"=
>=2C OSErr (ec)=2C ep.cmd))=3B<br>>=3B + END<br>>=3B + END FulfilE=
>xecPromise=3B<br>>=3B + <br>>=3B PROCEDURE KillProcess (handle: Proce=
>ss.T) =3D<br>>=3B BEGIN<br>>=3B IF (handle # NIL) THEN<br>>=
>=3B <br> </body>
></html>=
>
>--_b09106d1-2a84-4ffe-876c-0b2e4cdfd0b0_--
More information about the M3devel
mailing list