<html>
<head>
<style>
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
FONT-SIZE: 10pt;
FONT-FAMILY:Tahoma
}
</style>
</head>
<body class='hmmessage'>I believe cm3 is affected by this.<BR>I don't have numbers yet.<BR>
<BR>
I propose some fairly obvious/small/simple/safe changes in order to likely achieve a large speed up in NT386GNU.<BR>
I am skeptical that existing functions can be changed in a compatible enough way.<BR>
<BR>
So I propose, roughly:<BR>
<BR>
Add this to Process.i3:<BR>
<BR>
PROCEDURE Spawn(cmd: Pathname.T; READONLY params: ARRAY OF TEXT) : T RAISES {OSError.E};<BR>(* A restricted form of Create that is much faster on Cygwin. *)<BR>
<BR>
The name is very iffy.<BR>
It could be in fact not be in the public interface, but merely notice if wd = stdin = stdout = stderr = nil.<BR>
It could probably be in be less limited than shown.<BR>
Probably all of the parameters are settable, by altering the parent's globals, within a critical section.<BR>
Environment certainly is settable.<BR>
It is tempting to leave it limited like this though, such as to be implementable perhaps with system.<BR>
(It turns out Cygwin system is slower than spawnve; surprising since system is the most limited of the exec/spawn variants -- I think related to it having an implied sh wrapper but the others do not.)<BR><BR>The intent is simple and obvious -- some path to spawnve or spawnvpe.<BR>
p has path search.<BR>
<BR>
On all but Cygwin, this limited Create/Spawn will just call the normal Create. (Even on Win32).<BR>
On Cygwin it will call spawnvpe (or spawnve if people really want, but "p" seems "better").<BR>
<BR>
Now, in Quake, all the existing exec variants wrap the command line in either sh or cmd or command.com.<BR>
Changing that is probably very dangerous, even with an attempt to discern if the wrapper buys anything, on a command line by command line basis.<BR>
For example, if all of the characters * ? | < > % $ & ; ( ) are absent from the command, the shell wrapper probably doesn't buy anything and could be removed from existing paths. However that's not true -- for example system("echo foo") depends on a shell wrapper to run the builtin "echo" (at least on Windows, there is no echo.exe).<BR>
<BR>
I think there's no choice but to add a new Quake function, spawn, or limited_exec, or fast_exec, or process_runfast, exec_noshell, or something.<BR>
Again I'm not sure what to call it, but it'd simply call Process.Spawn, or Process.Create but with right the right parameters to get into the Cygwin fast path.<BR><BR>For now I'm going with Process.Spawn and fast_exec.<BR>
I hope to have numbers "soon" as to the perf change.<BR>
<BR>
Another good option here, that I tried in the past but failed, and is partly not difficult to solve, but also partly, is to implemet Quake exec using Win32 CreateProcess instead of Cygwin spawn/exec. There are at least two sets of problems here. One is that the existing code returns a File.T, and for that there is the Posix and Win32 types, Cygwin uses Posix. You'd have to warp the code somehow here. I gave up on that point without much trying. It's not that much code though. Cygwin is using CreateProcess of course.<BR>
The other problem is on the input, the interpretation of the command line. Again this is the value that presently Cygwin provides (albeit sometimes with great cost).<BR>
<BR>
Of course another angle is to work on Cygwin to make vfork efficient. It is presently implemented by calling fork.<BR>
There is #ifdef'ed code for another path but it appears not enabled.<BR>
<BR>
I know polluting the system just for the sake of Cygwin isn't great, however:<BR>
- I expect the win is quite large <BR>
- "spawn*" is a pretty old thing, nothing new/controversial here, long known as an often viable replacement for fork+exec at least on Windows.<BR>
It's in msvc*.dll for example.<BR>
<BR>
There may even be wins to be had on other Posix systems by avoiding the sh wrapper?<BR>
<BR>
"batching" where cm3cg is run once per directory seems like a very good idea and worth trying; the problem is, that still leaves the assembler.<BR>
Perhaps the assembler could be linked in statically to cm3cg? Probably, but not particularly easily and probably unpopular upstream...<BR>
Unless maybe some nice gcc perf gains would be demonstrated?<BR>
<BR>
- Jay<BR>
<BLOCKQUOTE>
<HR id=EC_stopSpelling>
From: jayk123@hotmail.com<BR>To: m3devel@elegosoft.com<BR>Subject: possible cygwin createprocess/fork cost measurements..<BR>Date: Mon, 17 Mar 2008 10:14:50 +0000<BR><BR>
<META content="Microsoft SafeHTML" name=Generator>
<STYLE>
.ExternalClass .EC_hmmessage P
{padding:0px;}
.ExternalClass EC_body.hmmessage
{font-size:10pt;font-family:Tahoma;}
</STYLE>
I ran some mostly scientific measures of Cygwin. <BR> On one machine, no reboots, one OS, one set of files.<BR> x86, single proc, Windows 2000 (I'll go back to XP soon). <BR> It shows that..well, at least that wrapping Cygwin processes with sh is VERY expensive. <BR> Like, the data isn't yet complete, but this could cut building Cygwin libm3 <BR> from around 100 seconds to around 20 seconds. Not counting the Modula-3 front end time. <BR> Just cm3cg+as.<BR><BR> cd libm3\NT386GNU <BR> having already built successfully, all the *.ic *.mc files are present <BR> <BR><BR> cm3cg not wrapped with sh (F1) <BR> Repeated runs. <BR> 28 seconds (other stuff running on machine) <BR> 16 seconds <BR> 13 seconds (13.?) <BR> 13.8 seconds <BR> 14.01 seconds <BR> 13.3 seconds <BR> now add the -o flag <BR> 13.64 seconds <BR> 14.07 seconds <BR> now without echoing <BR> 13.22 seconds <BR> 13.18 seconds <BR> <BR><BR> cm3cg wrapped with sh (F2) <BR> 51 seconds <BR> 51.35 seconds <BR> 51.19 seconds <BR> 50.88 seconds <BR> now add the -o flag <BR> 51.76 seconds <BR> now without echoing <BR> 51.05 seconds <BR> These runs did NOT have -o flags, but subsequent runs with -o were about the same. <BR> I added -o so I could run the as variations. <BR> now the same with .s <BR> note that due to the way the above worked, I just have *.s files, and <BR> not the usual *.is and *.ms <BR> <BR><BR> as not wrapped with sh (F3) <BR> 5.6 seconds <BR> 5.28 seconds <BR> now remove echo <BR> 5.08 seconds <BR> 5.08 seconds <BR> 5.04 seconds <BR> forgot -o flag, oh well, enough data <BR> as wrapped with sh (F4) <BR> 43 seconds <BR> 43.56 seconds <BR> forgot -o flag, oh well, enough data <BR> What is not yet confirmed is: <BR> 1) Does cm3 wrap everything with sh? <BR> 2) Does calling m3cg/as from cm3 have these costs? <BR> Very clear: <BR> Wrapping stuff with sh on Cygwin is expensive! <BR> <BR> <BR> Actions: <BR> Confirm this cost is being paid by cm3. <BR> Either: <BR> 1) implement some "batch modes" in cm3 and/or cm3cg <BR> 2) or maybe, um, just make sure that cm3 does not wrap with sh, and <BR> if cm3 itself causes this slowdown, because of how Cygwin works, try <BR> interposing a small Win32 helper app. I think Cygwin handles runnig <BR> Win32 apps and being run from Win32 apps differently than Cygwin running <BR> Cygwin -- i.e. not slowly. I'll see. Could be that creating twice the number <BR> of processes, in order to avoid Cygwin running Cygwin, could be faster. Not yet known. <BR> <BR> Maybe use system() instead vfork() + exec()? Odd, though, vfork instead of fork is supposed to help.<BR><BR> Here is the test code, you edit it to run one case or another: <BR> <BR> <BR>@if not "%1" == "" goto :%1<BR>@rem \cygwin\bin\time cmd /c %~f0 F1<BR>@rem \cygwin\bin\time cmd /c %~f0 F2<BR>@rem \cygwin\bin\time cmd /c %~f0 F3<BR>@\cygwin\bin\time cmd /c %~f0 F4<BR>@goto :eof<BR> <BR> <BR>:F1<BR>@echo off<BR>@del *s *o<BR>for %%a in (*.ic *.mc) do cm3cg -quiet %%a -o %%a.s"<BR>goto :eof<BR> <BR> <BR>:F2<BR>@echo off<BR>@del *s *o<BR>for %%a in (*.ic *.mc) do sh -c "cm3cg -quiet %%a -o %%a.s"<BR>goto :eof<BR> <BR> <BR>:F3<BR>@del *o<BR>@echo off<BR>for %%a in (*.s) do as %%a<BR>goto :eof<BR> <BR> <BR>:F4<BR>@del *o<BR>for %%a in (*.s) do sh -c "as %%a"<BR>goto :eof<BR> <BR><BR> - Jay<BR><BR><BR><BR>
<HR>
Helping your favorite cause is as easy as instant messaging. You IM, we give. <A href="http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join" target=_blank>Learn more.</A> </BLOCKQUOTE><br /><hr />Connect and share in new ways with Windows Live. <a href='http://www.windowslive.com/share.html?ocid=TXT_TAGHM_Wave2_sharelife_012008' target='_new'>Get it now!</a></body>
</html>