<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Yes. Agreed.<BR>They should be ok with respect to SuspendThread + GetThreadContext. The small C++ test program in scratch.<BR>I don't know about the "other" problem that affects even pthreads and I agree we should focus on that first, on pthreads, native I386_NT, and possibly AMD64_NT, but for now, at least briefly, ignore I386_NT on 64bit Windows.<BR> <BR> <BR> - Jay<br><br> <BR><div><hr id="stopSpelling">From: rcolebur@SCIRES.COM<br>To: m3devel@elegosoft.com<br>Date: Thu, 23 Jan 2014 22:23:43 +0000<br>Subject: Re: [M3devel] proposal for getting wow64 context/stack reliably?<br><br>
<style><!--
.ExternalClass .ecxshape {
}
--></style><style><!--
.ExternalClass p.ecxMsoNormal, .ExternalClass li.ecxMsoNormal, .ExternalClass div.ecxMsoNormal {
font-size:12.0pt;
font-family:"Times New Roman","serif";
}
.ExternalClass a:link, .ExternalClass span.ecxMsoHyperlink {
color:blue;
text-decoration:underline;
}
.ExternalClass span.ecxMsoHyperlinkFollowed {
color:purple;
text-decoration:underline;
}
.ExternalClass p {
font-size:12.0pt;
font-family:"Times New Roman","serif";
}
.ExternalClass p.ecxMsoAcetate, .ExternalClass li.ecxMsoAcetate, .ExternalClass div.ecxMsoAcetate {
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
}
.ExternalClass span.ecxapple-style-span {
}
.ExternalClass span.ecxapple-converted-space {
}
.ExternalClass span.ecxBalloonTextChar {
font-family:"Tahoma","sans-serif";
}
.ExternalClass span.ecxEmailStyle22 {
font-family:"Calibri","sans-serif";
color:#1F497D;
}
.ExternalClass .ecxMsoChpDefault {
font-size:10.0pt;
}
.ExternalClass div.ecxWordSection1 {
}
--></style>
<div class="ecxWordSection1">
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'>Jay:</span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'> </span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'>When you say native 32/64-bit “should be ok”, you really mean ok with respect to the SYSWOW64 problem you’ve identified.
</span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'> </span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'>Both of these native implementations still have threading misbehaviors as identified by the thread test program. Correct?</span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'> </span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'>--Randy</span></p>
<p class="ecxMsoNormal"><span style='color: rgb(31, 73, 125); font-family: "Calibri","sans-serif"; font-size: 11pt;'> </span></p>
<div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentColor currentColor; padding: 3pt 0in 0in;">
<p class="ecxMsoNormal"><b><span style='font-family: "Tahoma","sans-serif"; font-size: 10pt;'>From:</span></b><span style='font-family: "Tahoma","sans-serif"; font-size: 10pt;'> jayk123@hotmail.com [mailto:jayk123@hotmail.com]
<b>On Behalf Of </b>Jay K<br>
<b>Sent:</b> Thursday, January 23, 2014 2:57 PM<br>
<b>To:</b> Tony; Coleburn, Randy<br>
<b>Cc:</b> m3devel<br>
<b>Subject:</b> EXT:RE: [M3devel] EXT: proposal for getting wow64 context/stack reliably?</span></p>
</div>
</div>
<p class="ecxMsoNormal"> </p>
<div>
<p class="ecxMsoNormal">
<span style='font-family: "Calibri","sans-serif";'>There are no linker tricks. There are no syscalls, they aren't a special case just about anywhere, there are just functions, all treated the same.<br>
Correct it is only 32bit-on-64bit. Native 32bit should be ok. Native 64bit should be ok.<br>
<br>
- Jay</span></p>
<div>
<div align="center" class="ecxMsoNormal" style="text-align: center;">
<span style='font-family: "Calibri","sans-serif";'>
<hr width="100%" size="3" align="center" id="ecxstopSpelling">
</span></div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>From:
<a href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</a><br>
Date: Thu, 23 Jan 2014 11:51:02 -0500<br>
To: <a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a><br>
CC: <a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a>; <a href="mailto:jay.krell@cornell.edu">
jay.krell@cornell.edu</a><br>
Subject: Re: [M3devel] EXT: proposal for getting wow64 context/stack reliably?<br>
<br>
Yes, agreed.</span></p>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>We need to address the native thread problem on all platforms (both pthread and windows).</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>However, this particular problem is pernicious if we need to save M3 stack state at all external calls.</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>Are there any linker tricks for Windows one can use to wrap only system calls?</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>I note that cooperative suspend will also need to do this at system calls so we can treat threads running code running outside of Modula-3 as essentially stopped,
This approach is used for all modern Java implementations that I am aware of.</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>It really begins to sound like we must bite the bullet and move to cooperative threading so that we can guarantee GC safepoints where we can safely get the thread
state.</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
<div>
<div>
<div>
<div>
<p class="ecxMsoNormal"><span class="ecxapple-style-span"><span style='color: blue; font-family: "Gill Sans","serif"; font-size: 9pt;'>Antony Hosking</span></span><span class="ecxapple-converted-space"><span style='color: black; font-family: "Gill Sans","serif"; font-size: 9pt;'> </span></span><span class="ecxapple-style-span"><span style='color: black; font-family: "Gill Sans","serif"; font-size: 9pt;'>|</span></span><span class="ecxapple-converted-space"><span style='color: black; font-family: "Gill Sans","serif"; font-size: 9pt;'> </span></span><span class="ecxapple-style-span"><span style='color: black; font-family: "Gill Sans","serif"; font-size: 9pt;'>Associate
Professor | Computer Science | Purdue University</span></span><span style='color: black; font-family: "Helvetica","sans-serif"; font-size: 9pt;'></span></p>
</div>
<div>
<p class="ecxMsoNormal"><span class="ecxapple-style-span"><span style='color: black; font-family: "GillSans-Light","serif"; font-size: 9pt;'>305 N. University Street | West Lafayette | IN 47907 | USA</span></span><span style='color: black; font-family: "Helvetica","sans-serif"; font-size: 9pt;'></span></p>
</div>
<div>
<p class="ecxMsoNormal"><span class="ecxapple-style-span"><span style='color: blue; font-family: "Gill Sans","serif"; font-size: 9pt;'>Mobile</span></span><span class="ecxapple-converted-space"><span style='color: black; font-family: "GillSans-Light","serif"; font-size: 9pt;'> </span></span><span class="ecxapple-style-span"><span style='color: black; font-family: "GillSans-Light","serif"; font-size: 9pt;'>+1
765 427 5484</span></span><span style='color: black; font-family: "Helvetica","sans-serif"; font-size: 9pt;'></span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='color: black; font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<p class="ecxMsoNormal"><span style='color: black; font-family: "Helvetica","sans-serif"; font-size: 9pt;'><br>
<br>
</span></p>
</div>
</div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'><br>
<br>
</span></p>
</div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
<div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'>On Jan 23, 2014, at 6:59 AM, Coleburn, Randy <<a href="mailto:rcolebur@SCIRES.COM">rcolebur@SCIRES.COM</a>> wrote:</span></p>
</div>
<blockquote>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
<div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>Jay,</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>If I'm following you correctly, you are saying that this bug happens when running on 64bit Windows, but not on 32bit, since it applies only to SysWOW64. </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>Since the thread test program also misbehaves on 32bit Windows, there must yet be some other bug in the underlying implementation affecting both
32 & 64 bit platforms. </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>Thus, the SysWOW64 issue is a 2nd, additional problem for 64bit. </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>Has anyone made any progress in solving the 1st, underlying problem affecting both 32/64 bit Windows?</span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> </span></p>
</div>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'>--Randy<br>
<br>
Sent from my iPhone</span></p>
</div>
<div>
<p class="ecxMsoNormal">
<span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'><br>
On Jan 23, 2014, at 2:58 AM, "Jay K" <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:</span></p>
</div>
<blockquote>
<div>
<p class="ecxMsoNormal"><span style='font-family: "Helvetica","sans-serif"; font-size: 9pt;'> There is a behavior/bug in wow64. <span class="ecxapple-converted-space"> </span><br>
Bing for "wow64 GetThreadContext" "wow64 stack pointer", etc.<br>
<br>
<br>
<br>
SuspendThread / GetThreadContext work like this: <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
32bit processes consist almost entirely of 32bit code. <span class="ecxapple-converted-space"> </span><br>
There is a small amount of 64bit code. <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
If you suspend while running 32bit code, GetThreadContext works. <span class="ecxapple-converted-space"> </span><br>
if you suspend while running 64bit code, GetThreadContext usually but not always works. <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
64bit code is run en route to syscalls. <span class="ecxapple-converted-space"> </span><br>
For example you call: <span class="ecxapple-converted-space"> </span><br>
1 kernel32!Sleep <span class="ecxapple-converted-space"> </span><br>
2 it calls 32bit NtDelayExecution <span class="ecxapple-converted-space"> </span><br>
3 that calls wow64NtDelayExecutation (via a cross segment "far" jmp or call)<span class="ecxapple-converted-space"> </span><br>
4 which calls native NtDelayExecuation <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
In between 2 and 3, within 64bit code, the 32bit context is saved. <span class="ecxapple-converted-space"> </span><br>
You can step through it very easily in a debugger. Really.<span class="ecxapple-converted-space"> </span><br>
Where GetThreadContext knows where to get it. <span class="ecxapple-converted-space"> </span><br>
The problem is that saving context is not atomic. <span class="ecxapple-converted-space"> </span><br>
You can suspend while saving context. <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
What to do?<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
scratch/wow64stack contains a program that detects the bug.<br>
I believe it is the basis of a workaround for the bug. <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Proposal is that in the compiler, for I386_NT/NT386/I386_MINGWIN/I386_CYGWIN/I386_INTERIX platforms,<span class="ecxapple-converted-space"> </span><br>
not only functions that use exception handling, but also functions that call "extern" functions <span class="ecxapple-converted-space"> </span><br>
call GetActivation / SetActivation and therein save/set/restore the stack pointer. And garbage collection<br>
use that, if it isn't zero.<span class="ecxapple-converted-space"> </span><br>
Normally it will be zero. <span class="ecxapple-converted-space"> </span><br>
Syscalls do nest -- I can call SendMessage and in my window proc call CreateFile. <span class="ecxapple-converted-space"> </span><br>
That is why it isn't a set/set-zero pattern (like in the test program). <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Downside: We would like to get rid of GetActivation / SetActivation, i.e. and reuse efficient C++ exception handling.<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Rejected counter proposal:<span class="ecxapple-converted-space"> </span><br>
Don't suspend/gc when running a syscall. <span class="ecxapple-converted-space"> </span><br>
No -- you can Sleep() a while. You can/should-be-able-to suspend and GC during that. <span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Possible augmentation: if running native, short circuit most of this<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Rejected counter proposal: Have I386_NT_NATIVE that doesn't do this stuff. Relegate<span class="ecxapple-converted-space"> </span><br>
compatibility to I386_NT_COMPATIBLE. I don't like having more platforms/targets.<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
AMD64_NT wouldn't have this stuff. Nor would other hypothetical platforms like ARM32_NT,<span class="ecxapple-converted-space"> </span><br>
until/unless there is another 32bit platform runable on some similar 64bit platforms.<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Performance impact: hypothetically large but probably not noticable.<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Furthe refinements: It isn't extern/native code per se, it is syscalls.<br>
We could further augment pragmas to discern them.<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
We could leave it to writing "syscall wrappers" and informally (no enforcement) ban direct calls<span class="ecxapple-converted-space"> </span><br>
to any functions that make syscalls. This is likely too hard to maintain and too unfriendly.<span class="ecxapple-converted-space"> </span><br>
It pretty viable in m3core, but then also libm3 and Trestle and Qt wrappers, etc...<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
<br>
Agreed? I'll make the compiler change?<span class="ecxapple-converted-space"> </span><br>
<br>
<br>
Oh, also, not just stack pointer, but other registers, at least non-volatiles?<br>
<br>
<br>
Eventually cooperative suspend will cause this to fall away as a problem..<br>
<br>
<br>
- Jay</span></p>
</div>
</blockquote>
</div>
</blockquote>
</div>
<p class="ecxMsoNormal"><span style='font-family: "Calibri","sans-serif";'> </span></p>
</div>
</div>
</div>
</div></div> </div></body>
</html>