[M3devel] results of threadtest program on Windows7

Sun Mar 13 21:34:27 CET 2011

Hi all:
I was wondering how it might to produce an environment such of gcc level compatibility, without breaking the runtime in two languages, that is sort of the idea I guess never fully released GNU Modula-3. Perhaps that was the idea of RMS, to make compatible interfaces so anything misbehaved in any platform could be replaced with something more usable for those purposes (and so he was just interested in code generator that accompanied m3tk) see:
http://opensource.apple.com/source/gcc/gcc-5479/boehm-gc/doc/README.macros

So in that case we could use some of the RT requirements more easily and give each platform free compatibility of our RT so more freedom for further developments, say GPL interface against implementations, etc and stabilize more the RT in terms of more up to date requirements, with openjdk, or any other RT, even OS deprecated ones (say we would want that applied in Android OS or any other lightweight system e.g Java I/O subsystem borrow):

http://www.semiramis.org.uk/rrw/software/misc/ksynch.html

Indeed a contracted developer by MIT developed such automating of information recovery from C interfaces (well an OO C dialect) to link with Modula-3 runtime without too much replication and much more smoothly  results  in terms of stability of  Modula-3 RT such e.g apply it too Atoms, etc with full compatiblity if so:
http://www.doc.ic.ac.uk/~pcc03/transfer.pdf
Perhaps this way can we reuse some more time of current developers and historical ones too, which I believe are very important too (especially ESC in Java for any kind of developer even from smartphones and innovative systems like  see:
http://whiley.org/about/overview/
).

Thanks in advance

--- El dom, 13/3/11, Tony Hosking <hosking at cs.purdue.edu> escribió:

De: Tony Hosking <hosking at cs.purdue.edu>
Asunto: Re: [M3devel] results of threadtest program on Windows7
Para: "Jay K" <jay.krell at cornell.edu>
CC: "m3devel" <m3devel at elegosoft.com>
Fecha: domingo, 13 de marzo, 2011 13:08

I'd like to debug the current problem because it does not seem to imply any problem with suspend/resume.  More likely is losing a reference in the stacks somehow.

Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAOffice +1 765 494 6001 | Mobile +1 765 427 5484

On Mar 12, 2011, at 7:40 PM, Jay K wrote:
I thought maybe they'd show a way for us to get away with "preemptive suspend",
a way to use SuspendThread/GetThreadContext with confidence.

So, you agree we want cooperative suspend then?
And we should maybe get going on implementing it?
Maybe do that before debugging Mika's problem?
  Well, to repeat myself, I'm ok with looking into 32bit-only Windows for Mika's problem.
  Though, clearly, as i've said, I have A LOT less time for Modula-3 now and probably ever.

 - Jay

Subject: Re: [M3devel] results of threadtest program on Windows7
From: hosking at cs.purdue.edu
Date: Sat, 12 Mar 2011 11:23:04 -0500
CC: m3devel at elegosoft.com
To: jay.krell at cornell.edu

I know they use cooperative suspend.We do in Jikes RVM too.That's another code base to look at.  Probably easier as it is all in Java.
On Mar 12, 2011, at 4:57 AM, Jay K wrote:
There is a lack of evidence that Java does things in a way at all resembling what Modula-3 does, on Win32.
 There is precious little in the way of calling the Win32 functions SuspendThread and GetThreadContext.
There is evidence that they use cooperative suspend.
It isn't conclusive, to me..just because there is so much there, it is not immediately clear.
That is, pretty clear there is code implementing cooperative suspend, but is that their entire story? I don't know.

 - Jay

From: jay.krell at cornell.edu
To: hosking at cs.purdue.edu
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] results of threadtest program on Windows7
Date: Thu, 10 Mar 2011 21:03:58 +0000

I often forget this. Thank you.

 - Jay

From: hosking at cs.purdue.edu
Date: Thu, 10 Mar 2011 11:50:40 -0500
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com
Subject: Re: [M3devel] results of threadtest program on Windows7

Even better perhaps is to look at OpenJDK!

On Mar 10, 2011, at 6:05 AM, Jay K wrote:
Tony, more and more I think we want "cooperative suspend".
That would address this, avoid scanning the entire stack only the live stack on FreeBSD and OpenBSD, and
be much more portable, and bigger and slower.
I'll see what Mono and Rotor ("sscli") do though.
http://en.wikipedia.org/wiki/WoW64
Apparently the Boehm GC has problems too.

We could probably do something like, note the ranges of all Modula-3 .dlls, and check that EIP is in one of them,
else let the thread run longer and try suspend again. Maybe..

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Thu, 10 Mar 2011 10:40:58 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

oh, but to repeat, use of SuspendThread / GetThreadContext makes me nervous

http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html

some of the talk is only about the wrong esp, but I wonder if it is all wrong sometimes.
I think we can range check esp and reject if it is out of range, similar to the hack
that was there for Win9x.

Maybe stick to 32bit OS while investigating threadtest?

Or maybe ignore Win32 for now and find when pthreads worked?

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Thu, 10 Mar 2011 10:09:02 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

I spoke too soon, alas.
5.2.6 Win32:

..........laziest thread is 0/0/0 (tests: read 0/0/0 fork 0/0/0 alloc 0/0/0 lock
 0/0/0)
.

***
*** runtime error:
***    <*ASSERT*> failed.
***    file "..\src\runtime\common\RTCollector.m3", line 1073
***

Stack trace:
   FP         PC      Procedure
---------  ---------  -------------------------------
0xbf8f5f0  0x7531f4b3  CleanBetween + 0x48 in ..\src\runtime\common\RTCollector.m3
0xbf8f628  0x7531f326  CleanPage + 0x83 in ..\src\runtime\common\RTCollector.m3
0xbf8f658  0x7531ebca  FinishThreadPages + 0xa2 in ..\src\runtime\common\RTCollector.m3
0xbf8f6a0  0x7531eae6  CollectSomeInStateZero + 0x5a3 in ..\src\runtime\common\RTCollector.m3
0xbf8f6b4  0x7531e507  CollectSome + 0x6e in ..\src\runtime\common\RTCollector.m3
0xbf8f6cc  0x7531e084  CollectEnough + 0x9c in ..\src\runtime\common\RTCollector.m3
0xbf8f714  0x7532015c  LongAlloc + 0x74 in ..\src\runtime\common\RTCollector.m3
0xbf8f74c  0x75320086  AllocTraced + 0x8a in ..\src\runtime\common\RTCollector.m3
0xbf8f790  0x75316fa4  GetOpenArray + 0x8e in ..\src\runtime\common\RTAllocator.m3
0xbf8f7c0  0x7531688a  AllocateOpenArray + 0x2d in ..\src\runtime\common\RTAllocator.m3
.........  .........  ... more frames ...

I should double check these results.
And then I'm afraid I'm tempted to try 3.6 (DEC release) and 4.1 (commercial release)...

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Subject: RE: [M3devel] results of threadtest program on Windows7
Date: Wed, 9 Mar 2011 20:10:37 +0000

5.2.6 seems to have no problem with the thread test, on Win32.
Now to either:
  search for when the problems started.
  and/or try replacing current ThreadWin32.m3 with 5.2.6 or vice versa

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Subject: RE: [M3devel] results of threadtest program on Windows7
Date: Mon, 7 Mar 2011 06:58:45 +0000

Hm.. I think the first change may not be advisable. There are multiple such address range walkers and the interface probably doesn't constrain the order they go on.
So, instead, another change would be to touch every stack page at thread start, so that later walks going "out of order" won't trigger stack overflow exceptions.
This has the side benefit of not needing the backend to learn about chkstk.

That is, on Win32, stack pages must be touched in order from highest to lowest.
That is, the first time a stack page is touched, it should be only one lower than the lowest stack page previously touched.
Once some stack pages have been touched, they can later be touched in any order.
Functions with more than a page of locals are supposed to touch all of their frame's pages.
Otherwise function call's pushing a return address suffices.
The Modula-3 backend doesn't do this.
I've never seen this clearly documented, but it is clearly the intent of the C compiler's generated code.
  The C compiler generates a call to chkstk for functions with more than a page of locals, and chkstk touches a byte on each page.
  And chkstk and alloca are the same function.
  If this wasn't the case, the compiler could just inline subtraction.

And any GC code that walks the stack could possible violate this, depending on what it uses for stack bounds.
Currently it uses "accurate" bounds on Win32, and so is ok.

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Mon, 7 Mar 2011 06:53:06 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

I'm going to make one or two changes because of this.

 - Where we process memory in a range, I'm going to go in stack growth order instead from lowest address to highest. If we don't already.
    Or possibly recieve a direction parameter and pass it as 1 everywhere except for -1 when processing Win32 stacks.

 - On Win32 I'm going to process the "entire" stack, instead of just from the current stack pointer to the "initial" stack pointer (highest address).
   This is wasteful, will avoid reclaiming some garbage, will scan generally far more than needed.
   However it is also what we do on FreeBSD and OpenBSD (see ThreadOpenBSD.c's use of pthread_stackseg_np and ThreadFreeBSD.c's use of pthread_stackseg_np)

I think the main other alternative is the more portable "cooperative" scheme Tony has proposed where each thread has a boolean
it checks occasionally as to if suspension is requested. It is a nice scheme, since it would remove a bunch of platform-specific code
and posix-specific code (i.e. all signal stuff that isn't for user threads).

On the matter of garbage lingering in stack frames that have been popped...should we consider NIL'ing local pointers before turn?
Or it is too expensive and doesn't gain much?
And they'll tend to get overwritten fairly soon anyway??
Seems very gray.

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Mon, 7 Mar 2011 05:51:32 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

I find this worrisome, and want to understand it, and workaround it, somehow.

http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Mon, 7 Mar 2011 05:39:53 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

Ok. I see it fail often on Windows now too. I didn't change anything.
I enabled pageheap to try to narrow it down, only to find an operating system bug, which I have worked around.
"pageheap" causes heap allocations to be followed immediately by an unused page, to catch heap overruns right away.
 Modulo alignment restrictions -- i.e. malloc(1) will still have a few bytes you can silently corrupt before the unused page.

Here is one instance.

0:000> g
Writing file...done
Creating read threads...done
Creating fork threads...done
Creating alloc threads...done
Creating lock threads...done
running...printing oldest/median age/newest
.ModLoad: 73b20000 73b6b000   C:\Windows\SysWOW64\apphelp.dll
(1028.ca0): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=01155e18 ebx=00200000 ecx=00000005 edx=0111caee esi=001ffffc edi=00a80018
eip=0111cb3e esp=06d9f5c4 ebp=06d9f5fc iopl=0         nv up ei pl nz ac pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010216
*** WARNING: Unable to verify checksum for threadtest.exe
threadtest!RTCollector__Move+0x50:
0111cb3e 8b1e            mov     ebx,dword ptr [esi]  ds:002b:001ffffc=????????
0:009> r esi
esi=001ffffc
0:009> db @esi
001ffffc  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020000c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020001c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020002c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020003c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020004c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020005c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0020006c  ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ??  ????????????????
0:009> ~*k
   0  Id: 1028.d60 Suspend: 2 Teb: 7efdd000 Unfrozen
ChildEBP RetAddr

...
0045f674 0111a448 KERNELBASE!Sleep+0xf
0045f6b4 0111a218 threadtest!ThreadWin32__XPause+0x1cd
0045f6d8 010f356e threadtest!Thread__Pause+0x37
0045f7cc 01115268 threadtest!Main_M3+0xe67
0045f810 01114818 threadtest!RTLinker__RunMainBody+0x257
0045f828 011148bf threadtest!RTLinker__AddUnitI+0xf7
0045f848 010f1038 threadtest!RTLinker__AddUnit+0x9f
0045f864 011515d3 threadtest!main+0x38
0045f8a8 75e63677 threadtest!__tmainCRTStartup+0x122
0045f8b4 771c9f02 kernel32!BaseThreadInitThunk+0x12
...

   1  Id: 1028.ab8 Suspend: 2 Teb: 7efda000 Unfrozen
ChildEBP RetAddr
00f6f680 771c8cc8 ntdll!NtWaitForSingleObject+0x15
00f6f6a8 0111b262 ntdll!RtlIntegerToUnicodeString+0x20b
00f6f6bc 01117b86 threadtest!RTOS__LockHeap+0x2c
00f6f6fc 011171cc threadtest!RTAllocator__AllocTraced+0xcd
00f6f730 01116ce6 threadtest!RTAllocator__GetTracedObj+0x89
00f6f74c 010f6829 threadtest!RTHooks__AllocateTracedObj+0x15
00f6f774 010f1288 threadtest!FileRd__Open+0x19
00f6f8ac 01119ca4 threadtest!Main__RApply+0x78
00f6f8e8 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

   2  Id: 1028.d50 Suspend: 2 Teb: 7efd7000 Unfrozen
ChildEBP RetAddr
...
0106f9b0 011188ef threadtest!ThreadWin32__InitMutex+0x56
0106f9c8 010f7c08 threadtest!ThreadWin32__LockMutex+0x42
0106f9f0 010f12e3 threadtest!Rd__GetChar+0x28
0106fb28 01119ca4 threadtest!Main__RApply+0xd3
0106fb64 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

   3  Id: 1028.ef0 Suspend: 2 Teb: 7efaf000 Unfrozen
ChildEBP RetAddr
...
062ff510 01117b86 threadtest!RTOS__LockHeap+0x2c
062ff550 011171cc threadtest!RTAllocator__AllocTraced+0xcd
062ff584 01116ce6 threadtest!RTAllocator__GetTracedObj+0x89
062ff5a0 011076b2 threadtest!RTHooks__AllocateTracedObj+0x15
062ff5d8 01106302 threadtest!FileWin32__New+0x52
062ff600 010f6839 threadtest!FS__OpenFileReadonly+0x8a
062ff628 010f1288 threadtest!FileRd__Open+0x29
062ff760 01119ca4 threadtest!Main__RApply+0x78
062ff79c 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

   4  Id: 1028.10c8 Suspend: 2 Teb: 7efa9000 Unfrozen
ChildEBP RetAddr
...
0674fc44 010f17b9 threadtest!Process__Wait+0x8c
0674fd00 01119ca4 threadtest!Main__FApply+0xa2
0674fd3c 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

   5  Id: 1028.c54 Suspend: 2 Teb: 7efac000 Unfrozen
ChildEBP RetAddr
...
0646fcc8 010f17b9 threadtest!Process__Wait+0x8c
0646fd84 01119ca4 threadtest!Main__FApply+0xa2
0646fdc0 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...
   6  Id: 1028.101c Suspend: 2 Teb: 7efa3000 Unfrozen
ChildEBP RetAddr
...
06a1fdac 01117b86 threadtest!RTOS__LockHeap+0x2c
06a1fdec 0111772a threadtest!RTAllocator__AllocTraced+0xcd
06a1fe28 01116d9b threadtest!RTAllocator__GetOpenArray+0x94
06a1fe4c 010f1ada threadtest!RTHooks__AllocateOpenArray+0x19
06a1fea8 01119ca4 threadtest!Main__AApply+0x46
06a1fee4 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
06a1fef0 771c9f02 kernel32!BaseThreadInitThunk+0x12
...
   7  Id: 1028.d5c Suspend: 2 Teb: 7efa6000 Unfrozen
ChildEBP RetAddr
...
0684f6d0 01118eb1 threadtest!ThreadWin32__InitCondition+0xb0
0684f6f0 0111b328 threadtest!Thread__Broadcast+0x3f
0684f70c 01117c31 threadtest!RTOS__UnlockHeap+0xb3
0684f728 01117bbe threadtest!RTAllocator_M3_LINE_368+0x15
0684f768 011171cc threadtest!RTAllocator__AllocTraced+0x105
0684f79c 01116ce6 threadtest!RTAllocator__GetTracedObj+0x89
0684f7b8 011410c9 threadtest!RTHooks__AllocateTracedObj+0x15
0684f7d4 0112ce59 threadtest!Text8CString__New+0x19
0684f7ec 0112b63a threadtest!M3toC__StoT+0x16
0684f818 010fb22d threadtest!RTArgs__GetArg+0x70
0684f840 010f1776 threadtest!Params__Get+0x5d
0684f8fc 01119ca4 threadtest!Main__FApply+0x5f
0684f938 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...
   8  Id: 1028.d14 Suspend: 2 Teb: 7efa0000 Unfrozen
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
...
06c6f6c8 01117b86 threadtest!RTOS__LockHeap+0x2c
06c6f708 0111772a threadtest!RTAllocator__AllocTraced+0xcd
06c6f744 01116d9b threadtest!RTAllocator__GetOpenArray+0x94
06c6f768 010f1ada threadtest!RTHooks__AllocateOpenArray+0x19
06c6f7c4 01119ca4 threadtest!Main__AApply+0x46
06c6f800 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

#  9  Id: 1028.ca0 Suspend: 1 Teb: 7ef9d000 Unfrozen
ChildEBP RetAddr
06d9f5fc 0113e6bd threadtest!RTCollector__Move+0x50
06d9f640 0113dfa8 threadtest!RTHeapMap__Walk+0x45a
06d9f664 0113df83 threadtest!RTHeapMap__DoWalkRef+0x62
06d9f688 0113df83 threadtest!RTHeapMap__DoWalkRef+0x3d
06d9f6ac 0113df3e threadtest!RTHeapMap__DoWalkRef+0x3d
06d9f6d8 0111e65f threadtest!RTHeapMap__WalkRef+0xfe
06d9f6fc 0111e4ad threadtest!RTCollector__CleanBetween+0xeb
06d9f724 0111de48 threadtest!RTCollector__CleanPage+0x54
06d9f778 0111d8ab threadtest!RTCollector__CollectSomeInStateZero+0x562
06d9f788 0111d549 threadtest!RTCollector__CollectSome+0x6e
06d9f7cc 01117b8c threadtest!RTHeapRep__CollectEnough+0x9b
06d9f80c 0111772a threadtest!RTAllocator__AllocTraced+0xd3
06d9f848 01116d9b threadtest!RTAllocator__GetOpenArray+0x94
06d9f86c 010f1ada threadtest!RTHooks__AllocateOpenArray+0x19
06d9f8c8 01119ca4 threadtest!Main__AApply+0x46
06d9f904 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...
  10  Id: 1028.4cc Suspend: 2 Teb: 7ef97000 Unfrozen
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be wrong.
0664fdec 771c8cc8 ntdll!NtWaitForSingleObject+0x15
0664fe14 011188ff ntdll!RtlIntegerToUnicodeString+0x20b
0664fe2c 010f1d9b threadtest!ThreadWin32__LockMutex+0x52
0664fe74 01119ca4 threadtest!Main__LApply+0x7d
0664feb0 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

  11  Id: 1028.b48 Suspend: 2 Teb: 7ef9a000 Unfrozen
ChildEBP RetAddr
...
06f1fe38 010f1d9b threadtest!ThreadWin32__LockMutex+0x52
06f1fe80 01119ca4 threadtest!Main__LApply+0x7d
06f1febc 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

  12  Id: 1028.a0c Suspend: 2 Teb: 7ef94000 Unfrozen
ChildEBP RetAddr
...
06b1fa98 6c2a016a kernel32!HeapFree+0x14
06b1faac 0113328d MSVCR100!free+0x1c
06b1fab8 01116e2e threadtest!Cstdlib__free+0xd
06b1facc 011184fa threadtest!RTHooks__DisposeUntracedRef+0x2c
06b1fae0 011185c0 threadtest!ThreadWin32__DelCriticalSection+0x2d
06b1fb14 011188ef threadtest!ThreadWin32__InitMutex+0x6e
06b1fb2c 010f1d9b threadtest!ThreadWin32__LockMutex+0x42
06b1fb74 01119ca4 threadtest!Main__LApply+0x7d
06b1fbb0 75e63677 threadtest!ThreadWin32__ThreadBase+0x25a
...

Suggestions?

That last thread is a bit different, but I guess merely suggesting heavy stress.
  Init goes to Del only when another thread calls Init at about the same time on the same mutex.

Maybe try user threads but with a smaller quanta?
Maybe try a much older Win32 thread implementation?
Maybe go back to uniproc? Just for anecdotal evidence?
I'll try with noinc/nogen/paranoid on Win32 also.

 - Jay

From: jay.krell at cornell.edu
To: rcolebur at scires.com; m3devel at elegosoft.com
Date: Mon, 7 Mar 2011 01:25:59 +0000
Subject: Re: [M3devel] results of threadtest program on Windows7

Visual Studio should work.
You just open the .exe you want to debug, and it should decide you have opened it because you want to debug it.
Does it let you set breakpoints by typing in function names?
e.g. main or cm3!main or threadtest!main.

windbg is also very good and a free download.
e.g.
\bin\x86\windbg threadtest
bp threadtest!main
bp threadtest!RTError__MsgS
g

 - Jay

From: rcolebur at SCIRES.COM
To: m3devel at elegosoft.com
Date: Sat, 5 Mar 2011 20:35:41 -0500
Subject: Re: [M3devel] results of threadtest program on Windows7

Jay: I have Visual Studio Express installed.  Can I use its debugger?  If so, how do I make it work with cm3?  Or, do you suggest something else? Regards,Randy From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Saturday, March 05, 2011 12:38 AM
To: Coleburn, Randy; m3devel at elegosoft.com
Subject: RE: [M3devel] results of threadtest program on Windows7 Microsoft debuggers are freely downloadable. 

Jay/phone

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20110313/c6361136/attachment-0002.html>