<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Syscalls are "doubly wrapped".<br>Exports from ntdll.dll that start "Nt" or "Zw" are syscalls and nothing else. If you look at them, they all look special. But they expose a standard calling convention.<br>Lesser known, there are also some syscalls in gdi32.dll and user32.dll.<br>None of them are public interfaces.<br>They are all reached through public interfaces typically in kernel32.dll, gdi32.dll, user32.dll.<br>For example, kernel32!CreateFileW might eventually call ntdll!NtCreateFile, but not directly, and this can change.<br>Unix is similar and different -- sometimes functions are just a syscall, sometimes they are wrappers that do other work.<br><br>There are counter examples -- functions that just return global variables like GetTickCount/GetTickCount64/GetCommandLine/GetEnvironmentVariable without a syscall.<br><br>Bottom line is though, one isn't really supposed to know where the syscalls are.<br><br> - Jay<br><br><div><hr id="stopSpelling">From: jay.krell@cornell.edu<br>To: hosking@cs.purdue.edu; rcolebur@scires.com<br>CC: m3devel@elegosoft.com<br>Subject: RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?<br>Date: Thu, 23 Jan 2014 19:57:27 +0000<br><br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.<br>Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.<br><br> - Jay<br><br><div><hr id="ecxstopSpelling">From: hosking@cs.purdue.edu<br>Date: Thu, 23 Jan 2014 11:51:02 -0500<br>To: rcolebur@SCIRES.COM<br>CC: m3devel@elegosoft.com; jay.krell@cornell.edu<br>Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?<br><br>Yes, agreed.<div>We need to address the native thread problem on all platforms (both pthread and windows).</div><div><br></div><div>However, this particular problem is pernicious if we need to save M3 stack state at all external calls.</div><div><br></div><div>Are there any linker tricks for Windows one can use to wrap only system calls?</div><div><br></div><div>I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of.</div><div><br></div><div>It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.</div><div><br><div>
<span class="ecxApple-style-span" style="border-collapse:separate;border-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;orphans:2;text-indent:0px;text-transform:none;white-space:normal;widows:2;word-spacing:0px;"><div style="word-wrap:break-word;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><div style="word-wrap:break-word;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><div><font class="ecxApple-style-span" color="#0000FF"><font class="ecxApple-style-span" face="Gill Sans"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';">Antony Hosking</span></span></font></font><font class="ecxApple-style-span" face="Gill Sans"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-converted-space"> </span>|<span class="ecxApple-converted-space"> </span></span></span><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';">Associate Professor</span></span><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"> | Computer Science | Purdue University</span></span></font></div><div><font class="ecxApple-style-span" face="GillSans-Light"><span class="ecxApple-style-span" style="font-family:GillSans-Light;">305 N. University Street | West Lafayette | IN 47907 | USA</span></font></div><div><font class="ecxApple-style-span" color="#0000FF" face="Gill Sans"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';">Mobile</span></span></font><font class="ecxApple-style-span" face="GillSans-Light"><span class="ecxApple-style-span" style="font-family:GillSans-Light;"><span class="ecxApple-style-span" style="font-family:GillSans-Light;"><span class="ecxApple-converted-space"> </span>+1 765 427 5484</span></span></font></div><div><font class="ecxApple-style-span" face="GillSans-Light"><br class="ecxkhtml-block-placeholder"></font></div></span></span></span></span></span></span></span><br class="ecxApple-interchange-newline"></span></div></span></div></span><br class="ecxApple-interchange-newline"></span><br class="ecxApple-interchange-newline">
</div>
<br><div><div>On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <<a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a>> wrote:</div><br class="ecxApple-interchange-newline"><blockquote><div dir="auto" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;"><div>Jay,</div><div><br></div><div>If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. </div><div><br></div><div>Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. </div><div><br></div><div>Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. </div><div><br></div><div>Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?</div><div><br></div><div>--Randy<br><br>Sent from my iPhone</div><div><br>On Jan 23, 2014, at 2:58 AM, "Jay K" <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:<br><br></div><blockquote><div dir="ltr"> There is a behavior/bug in wow64. <span class="ecxApple-converted-space"> </span><br> Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.<br><br><br><br> SuspendThread / GetThreadContext work like this: <span class="ecxApple-converted-space"> </span><br><br><br> 32bit processes consist almost entirely of 32bit code. <span class="ecxApple-converted-space"> </span><br> There is a small amount of 64bit code. <span class="ecxApple-converted-space"> </span><br><br><br> If you suspend while running 32bit code, GetThreadContext works. <span class="ecxApple-converted-space"> </span><br> if you suspend while running 64bit code, GetThreadContext usually but not always works. <span class="ecxApple-converted-space"> </span><br><br><br> 64bit code is run en route to syscalls. <span class="ecxApple-converted-space"> </span><br> For example you call: <span class="ecxApple-converted-space"> </span><br> 1 kernel32!Sleep <span class="ecxApple-converted-space"> </span><br> 2 it calls 32bit NtDelayExecution <span class="ecxApple-converted-space"> </span><br> 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call)<span class="ecxApple-converted-space"> </span><br> 4 which calls native NtDelayExecuation <span class="ecxApple-converted-space"> </span><br><br> <br> In between 2 and 3, within 64bit code, the 32bit context is saved. <span class="ecxApple-converted-space"> </span><br> You can step through it very easily in a debugger. Really.<span class="ecxApple-converted-space"> </span><br> Where GetThreadContext knows where to get it. <span class="ecxApple-converted-space"> </span><br> The problem is that saving context is not atomic. <span class="ecxApple-converted-space"> </span><br> You can suspend while saving context. <span class="ecxApple-converted-space"> </span><br><br><br> What to do?<span class="ecxApple-converted-space"> </span><br><br><br> scratch/wow64stack contains a program that detects the bug.<br> I believe it is the basis of a workaround for the bug. <span class="ecxApple-converted-space"> </span><br><br><br> Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,<span class="ecxApple-converted-space"> </span><br> not only functions that use exception handling, but also functions that call "extern" functions <span class="ecxApple-converted-space"> </span><br> call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection<br> use that, if it isn't zero.<span class="ecxApple-converted-space"> </span><br> Normally it will be zero. <span class="ecxApple-converted-space"> </span><br> Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. <span class="ecxApple-converted-space"> </span><br> That is why it isn't a set/set-zero pattern (like in the test program). <span class="ecxApple-converted-space"> </span><br><br><br> Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.<span class="ecxApple-converted-space"> </span><br><br><br> Rejected counter proposal:<span class="ecxApple-converted-space"> </span><br> Don't suspend/gc when running a syscall. <span class="ecxApple-converted-space"> </span><br> No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. <span class="ecxApple-converted-space"> </span><br><br><br> Possible augmentation: if running native, short circuit most of this<span class="ecxApple-converted-space"> </span><br><br><br> Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate<span class="ecxApple-converted-space"> </span><br> compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.<span class="ecxApple-converted-space"> </span><br><br><br> AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,<span class="ecxApple-converted-space"> </span><br> until/unless there is another 32bit platform runable on some similar 64bit platforms.<span class="ecxApple-converted-space"> </span><br><br><br> Performance impact: hypothetically large but probably not noticable.<span class="ecxApple-converted-space"> </span><br><br><br> Furthe refinements: It isn't extern/native code per se, it is syscalls.<br> We could further augment pragmas to discern them.<span class="ecxApple-converted-space"> </span><br><br><br> We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls<span class="ecxApple-converted-space"> </span><br> to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.<span class="ecxApple-converted-space"> </span><br> It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...<span class="ecxApple-converted-space"> </span><br><br><br><br> Agreed? I'll make the compiler change?<span class="ecxApple-converted-space"> </span><br><br><br>Oh, also, not just stack pointer, but other registers, at least non-volatiles?<br><br><br>Eventually cooperative suspend will cause this to fall away as a problem..<br><br><br> - Jay</div></blockquote></div></blockquote></div><br></div></div> </div></div> </div></body>
</html>