<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>I might yet have another solution here..there might be a way to detect the race and retry...later..<br><br> - Jay<br><br><div><hr id="stopSpelling">From: hosking@cs.purdue.edu<br>Date: Fri, 24 Jan 2014 13:24:35 -0500<br>To: jay.krell@cornell.edu<br>CC: m3devel@elegosoft.com<br>Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?<br><br><div>As a way forward I propose that we indeed treat EXTERNAL calls as needing to capture a GC context. At least that should be the default.</div><div>Some calls could be annotated as not needing to capture the GC context with a different EXTERNAL pragma. Knowing that such a call can safely ignore saving the context would require inspection of the external function, or understanding of its behavior.</div><br><div>
<span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;orphans:2;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;orphans:2;text-indent:0px;text-transform:none;white-space:normal;widows:2;word-spacing:0px;"><div style="word-wrap:break-word;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><div style="word-wrap:break-word;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><span class="ecxApple-style-span" style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;orphans:2;white-space:normal;widows:2;word-spacing:0px;"><div><font class="ecxApple-style-span" color="#0000FF"><font class="ecxApple-style-span" face="Gill Sans"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';">Antony Hosking</span></span></font></font><font class="ecxApple-style-span" face="Gill Sans"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-converted-space"> </span>|<span class="ecxApple-converted-space"> </span></span></span><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';">Associate Professor</span></span><span class="ecxApple-style-span" style="font-family:'Gill Sans';"><span class="ecxApple-style-span" style="font-family:'Gill Sans';"> | Computer Science | Purdue University</span></span></font></div><div><font class="ecxApple-style-span" face="GillSans-Light"><span class="ecxApple-style-span" style="font-family:GillSans-Light;">305 N. University Street | West Lafayette | IN 47907 | USA</span></font></div><div><font class="ecxApple-style-span" color="#0000FF" face="Gill Sans"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';"><span class="ecxApple-style-span" style="color:rgb(0, 0, 255);font-family:'Gill Sans';">Mobile</span></span></font><font class="ecxApple-style-span" face="GillSans-Light"><span class="ecxApple-style-span" style="font-family:GillSans-Light;"><span class="ecxApple-style-span" style="font-family:GillSans-Light;"><span class="ecxApple-converted-space"> </span>+1 765 427 5484</span></span></font></div><div><font class="ecxApple-style-span" face="GillSans-Light"><br class="ecxkhtml-block-placeholder"></font></div></span></span></span></span></span></span></span><br class="ecxApple-interchange-newline"></span></div></span></div></span><br class="ecxApple-interchange-newline"></span><br class="ecxApple-interchange-newline">
</div>
<br><div><div>On Jan 23, 2014, at 6:09 PM, Jay K <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:</div><br class="ecxApple-interchange-newline"><blockquote><div class="ecxhmmessage" style="font-size:12pt;font-family:Calibri;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;"><div dir="ltr">Yes. Agreed.<br>They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.<br>I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.<br> <br> <br> - Jay<br><br> <br><div><hr id="ecxstopSpelling">From: <a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a><br>To: <a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a><br>Date: Thu, 23 Jan 2014 22:23:43 +0000<br>Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?<br><br><div class="ecxWordSection1"><div style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;">Jay:</span></div><p class="ecxMsoNormal" style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;"> </span></p><div style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;">When you say native 32/64-bit “should be ok”, you really mean ok with respect to the SYSWOW64 problem you’ve identified. </span></div><p class="ecxMsoNormal" style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;"> </span></p><div style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;">Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct?</span></div><p class="ecxMsoNormal" style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;"> </span></p><div style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;">--Randy</span></div><p class="ecxMsoNormal" style="padding:0px;"><span style="color:rgb(31, 73, 125);font-family:Calibri, sans-serif;font-size:11pt;"> </span></p><div><div style="border-width:1pt medium medium;border-style:solid none none;border-color:rgb(181, 196, 223) currentcolor currentcolor;padding:3pt 0in 0in;"><div style="padding:0px;"><b><span style="font-family:Tahoma, sans-serif;font-size:10pt;">From:</span></b><span style="font-family:Tahoma, sans-serif;font-size:10pt;"><span class="ecxApple-converted-space"> </span><a href="mailto:jayk123@hotmail.com">jayk123@hotmail.com</a> [<a href="mailto:jayk123@hotmail.com">mailto:jayk123@hotmail.com</a>]<span class="ecxApple-converted-space"> </span><b>On Behalf Of<span class="ecxApple-converted-space"> </span></b>Jay K<br><b>Sent:</b><span class="ecxApple-converted-space"> </span>Thursday, January 23, 2014 2:57 PM<br><b>To:</b><span class="ecxApple-converted-space"> </span>Tony; Coleburn, Randy<br><b>Cc:</b><span class="ecxApple-converted-space"> </span>m3devel<br><b>Subject:</b><span class="ecxApple-converted-space"> </span>EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?</span></div></div></div><p class="ecxMsoNormal" style="padding:0px;"> </p><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.<br>Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.<br><br> - Jay</span></div><div><div class="ecxMsoNormal" style="text-align:center;" align="center"><span style="font-family:Calibri, sans-serif;"><hr id="ecxstopSpelling" align="center" size="3" width="100%"></span></div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">From:<span class="ecxApple-converted-space"> </span><a href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</a><br>Date: Thu, 23 Jan 2014 11:51:02 -0500<br>To:<span class="ecxApple-converted-space"> </span><a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a><br>CC:<span class="ecxApple-converted-space"> </span><a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a>;<span class="ecxApple-converted-space"> </span><a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?<br><br>Yes, agreed.</span></div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">We need to address the native thread problem on all platforms (both pthread and windows).</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">However, this particular problem is pernicious if we need to save M3 stack state at all external calls.</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">Are there any linker tricks for Windows one can use to wrap only system calls?</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped, This approach is used for all modern Java implementations that I am aware of.</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread state.</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p><div><div><div><div style="padding:0px;"><span class="ecxapple-style-span"><span style="color:blue;font-family:'Gill Sans', serif;font-size:9pt;">Antony Hosking</span></span><span class="ecxapple-converted-space"><span style="font-family:'Gill Sans', serif;font-size:9pt;"> </span></span><span class="ecxapple-style-span"><span style="font-family:'Gill Sans', serif;font-size:9pt;">|</span></span><span class="ecxapple-converted-space"><span style="font-family:'Gill Sans', serif;font-size:9pt;"> </span></span><span class="ecxapple-style-span"><span style="font-family:'Gill Sans', serif;font-size:9pt;">Associate Professor | Computer Science | Purdue University</span></span><span style="font-family:Helvetica, sans-serif;font-size:9pt;"></span></div></div><div><div style="padding:0px;"><span class="ecxapple-style-span"><span style="font-family:GillSans-Light, serif;font-size:9pt;">305 N. University Street | West Lafayette | IN 47907 | USA</span></span><span style="font-family:Helvetica, sans-serif;font-size:9pt;"></span></div></div><div><div style="padding:0px;"><span class="ecxapple-style-span"><span style="color:blue;font-family:'Gill Sans', serif;font-size:9pt;">Mobile</span></span><span class="ecxapple-converted-space"><span style="font-family:GillSans-Light, serif;font-size:9pt;"> </span></span><span class="ecxapple-style-span"><span style="font-family:GillSans-Light, serif;font-size:9pt;">+1 765 427 5484</span></span><span style="font-family:Helvetica, sans-serif;font-size:9pt;"></span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"><br><br></span></div></div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;"><br><br></span></div></div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p><div><div><div style="padding:0px;"><span style="font-family:Calibri, sans-serif;">On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <<a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a>> wrote:</span></div></div><blockquote><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Calibri, sans-serif;"> </span></p><div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">Jay,</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. </span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both 32 & 64 bit platforms. </span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. </span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?</span></div></div><div><p class="ecxMsoNormal" style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> </span></p></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;">--Randy<br><br>Sent from my iPhone</span></div></div><div><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"><br>On Jan 23, 2014, at 2:58 AM, "Jay K" <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:</span></div></div><blockquote><div style="padding:0px;"><span style="font-family:Helvetica, sans-serif;font-size:9pt;"> There is a behavior/bug in wow64. <span class="ecxapple-converted-space"> </span><br> Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.<br><br><br><br> SuspendThread / GetThreadContext work like this: <span class="ecxapple-converted-space"> </span><br><br><br> 32bit processes consist almost entirely of 32bit code. <span class="ecxapple-converted-space"> </span><br> There is a small amount of 64bit code. <span class="ecxapple-converted-space"> </span><br><br><br> If you suspend while running 32bit code, GetThreadContext works. <span class="ecxapple-converted-space"> </span><br> if you suspend while running 64bit code, GetThreadContext usually but not always works. <span class="ecxapple-converted-space"> </span><br><br><br> 64bit code is run en route to syscalls. <span class="ecxapple-converted-space"> </span><br> For example you call: <span class="ecxapple-converted-space"> </span><br> 1 kernel32!Sleep <span class="ecxapple-converted-space"> </span><br> 2 it calls 32bit NtDelayExecution <span class="ecxapple-converted-space"> </span><br> 3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call)<span class="ecxapple-converted-space"> </span><br> 4 which calls native NtDelayExecuation <span class="ecxapple-converted-space"> </span><br><br> <br> In between 2 and 3, within 64bit code, the 32bit context is saved. <span class="ecxapple-converted-space"> </span><br> You can step through it very easily in a debugger. Really.<span class="ecxapple-converted-space"> </span><br> Where GetThreadContext knows where to get it. <span class="ecxapple-converted-space"> </span><br> The problem is that saving context is not atomic. <span class="ecxapple-converted-space"> </span><br> You can suspend while saving context. <span class="ecxapple-converted-space"> </span><br><br><br> What to do?<span class="ecxapple-converted-space"> </span><br><br><br> scratch/wow64stack contains a program that detects the bug.<br> I believe it is the basis of a workaround for the bug. <span class="ecxapple-converted-space"> </span><br><br><br> Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,<span class="ecxapple-converted-space"> </span><br> not only functions that use exception handling, but also functions that call "extern" functions <span class="ecxapple-converted-space"> </span><br> call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection<br> use that, if it isn't zero.<span class="ecxapple-converted-space"> </span><br> Normally it will be zero. <span class="ecxapple-converted-space"> </span><br> Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. <span class="ecxapple-converted-space"> </span><br> That is why it isn't a set/set-zero pattern (like in the test program). <span class="ecxapple-converted-space"> </span><br><br><br> Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.<span class="ecxapple-converted-space"> </span><br><br><br> Rejected counter proposal:<span class="ecxapple-converted-space"> </span><br> Don't suspend/gc when running a syscall. <span class="ecxapple-converted-space"> </span><br> No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. <span class="ecxapple-converted-space"> </span><br><br><br> Possible augmentation: if running native, short circuit most of this<span class="ecxapple-converted-space"> </span><br><br><br> Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate<span class="ecxapple-converted-space"> </span><br> compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.<span class="ecxapple-converted-space"> </span><br><br><br> AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,<span class="ecxapple-converted-space"> </span><br> until/unless there is another 32bit platform runable on some similar 64bit platforms.<span class="ecxapple-converted-space"> </span><br><br><br> Performance impact: hypothetically large but probably not noticable.<span class="ecxapple-converted-space"> </span><br><br><br> Furthe refinements: It isn't extern/native code per se, it is syscalls.<br> We could further augment pragmas to discern them.<span class="ecxapple-converted-space"> </span><br><br><br> We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls<span class="ecxapple-converted-space"> </span><br> to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.<span class="ecxapple-converted-space"> </span><br> It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...<span class="ecxapple-converted-space"> </span><br><br><br><br> Agreed? I'll make the compiler change?<span class="ecxapple-converted-space"> </span><br><br><br>Oh, also, not just stack pointer, but other registers, at least non-volatiles?<br><br><br>Eventually cooperative suspend will cause this to fall away as a problem..<br><br><br> - Jay</span></div></blockquote></div></blockquote></div></div></div></div></div></div></div></div></blockquote></div><br></div> </div></body>
</html>