<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>The summary though: my "recent" (years ago) implementation of condition variables et. al. was unconvincing, or more likely wrong. "Condition variables et. al." -- tied into threads and mutexes. I replaced it again. The replacement is based closely on the pthreads version. I noticed though that the pthread use of pthread condition variables does not require much, so win32 pre-vista mapped easily enough. </div><div><br></div><div><br></div><div>And now with the strong resemblance, more code sharing should be possible, but hasn't been done yet. A thin abstraction layer will be needed.</div><div><br> - Jay</div><div><br>On Aug 29, 2016, at 1:03 PM, Jay <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:<br><br></div><blockquote type="cite"><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div>There was a lull in mails, but I haven't made this jump yet anyway. </div><div><br> - Jay</div><div><br>On Aug 29, 2016, at 6:28 AM, "Hosking, Antony L" <<a href="mailto:hosking@purdue.edu">hosking@purdue.edu</a>> wrote:<br><br></div><blockquote type="cite"><div>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<div>Jay, I've not seen these commits or accompanying emails. (Not sure why!)</div>
<div id="AppleMailSignature">Can you summarize what is going on here?</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Thanks,</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Tony<br>
<br>
Sent from my iPad</div>
<div><br>
On Aug 29, 2016, at 10:49 PM, Jay K <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div><style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style>
<div dir="ltr">I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code.
<div><br>
</div>
<div>Win32 will then, for now, be one of the "Posix" platforms with "direct suspend".</div>
<div><br>
</div>
<div><br>
<div>Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected).</div>
<div><br>
</div>
<div>There are still a few steps to get there.</div>
<div><br>
</div>
<div>One of them is naming.</div>
<div>So far I'm thinking of names like:</div>
<div> ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion</div>
<div> ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event</div>
<div><br>
</div>
<div>Another thin abstraction is needed around computing times/timeouts.</div>
<div><br>
</div>
<div> - Jay<br>
<br>
<br>
<div>
<hr id="stopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sun, 31 Jul 2016 20:45:48 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress.
<div>It is attached. </div>
<div>I also removed the indirection on critical sections -- exposing the size to Modula3. </div>
<div> </div>
<div>I went with just an event as I indicated. </div>
<div>So the ThreadPThread.m3 code, but with slight changes: </div>
<div> - waits specify time to wait until time to wait until </div>
<div> - event instead of pthread_cond </div>
<div> - same maintenance of list of waiters </div>
<div><br>
</div>
<div>Reasonable?</div>
<div><br>
</div>
<div> - Jay<br>
<br>
<br>
<div>
<hr id="ecxstopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sun, 31 Jul 2016 10:37:30 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.
<div>An even per thread should work.</div>
<div><br>
</div>
<div> - Jay<br>
<br>
<br>
<div>
<hr id="ecxstopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sun, 31 Jul 2016 04:24:14 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">Something like attached?
<div>Still testing..might be crashing.</div>
<div>A lot of the code and patterns come from ThreadPThread.m3.</div>
<div><br>
</div>
<div> - Jay</div>
<div><br>
<br>
<div>
<hr id="ecxstopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sun, 31 Jul 2016 02:52:02 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">While I agree this is confusing, it is holding together for me and I have mostly coded it up.
<div><br>
</div>
<div>Repeating myself:</div>
<div> In this scheme there are two types of condition variable.</div>
<div> There are Modula-3 condition variables and posix-like/schmidt-like condition variables.</div>
<div><br>
</div>
<div>The posix-like/smidt-like condition variables:</div>
<div> Can be implemented directly via posix condition variables (if Windows pre-Vista had them).</div>
<div> They are implemented as a slight modification of the Smidt counter method.</div>
<div> The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible</div>
<div> to hold the same lock as he holds when calling cond_wait.</div>
<div><br>
</div>
<div> Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.</div>
<div> I still have to work through your examples.</div>
<div><br>
</div>
<div>Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.</div>
<div><br>
</div>
<div>This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.</div>
<div>XPause changes to call XWait, essentially.</div>
<div><br>
</div>
<div>I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.</div>
<div><br>
</div>
<div>If this works, we should be able to share more code between the pthread and win32 implementations.</div>
<div>We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).</div>
<div><br>
</div>
<div> - Jay<br>
<br>
<br>
<div>
<hr id="ecxstopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sun, 31 Jul 2016 00:47:24 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">ThreadPThread.m3 had the same narrow startup race condition.
<div>I fixed it.</div>
<div>Again I'm not seeing commit mails.</div>
<div><br>
</div>
<div>On the larger matter:</div>
<div><br>
</div>
<div>
<div>I think there is a solution, in that, if you look at</div>
<div>ThreadPThread.m3 and how it uses Posix condition variables,</div>
<div>in that it does a fair amount of the work itself,</div>
<div>you can term this "condition within condition".</div>
<div><br>
</div>
<div><br>
</div>
<div>So I think maybe we can use the counting solution</div>
<div>and 1) merge the locks and/or 2) take the lock in Signal/Broadcast.</div>
<div><br>
</div>
<div><br>
</div>
<div>But I have to look at it more.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Specifically, in ThreadPThread.m3:</div>
<div><br>
</div>
<div>Broadcast:</div>
<div> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;</div>
<div> next := t.nextWaiter;</div>
<div> t.nextWaiter := NIL;</div>
<div> t.waitingOn := NIL;</div>
<div> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;</div>
<div><br>
</div>
<div>Signal:</div>
<div> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;</div>
<div> t.nextWaiter := NIL;</div>
<div> t.waitingOn := NIL;</div>
<div> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;</div>
<div><br>
</div>
<div><br>
</div>
<div>Becaise they signal while locked, we can implement this limited</div>
<div>form of condition variable, and then use the Modula-3 pthread logic</div>
<div>using that.</div>
<div><br>
</div>
<div><br>
</div>
<div>Make some sense?</div>
<div><br>
</div>
<div><br>
</div>
<div>Essentially we will have almost-posix condition variables,</div>
<div>and exactly-posix mutexes, and then we can use just the pthread code</div>
<div>on top of that.</div>
<div><br>
</div>
<div><br>
</div>
<div>Almost-posix condition variable is a posix condition variable, except</div>
<div>it is guaranteed you will call pthread_cond_signal with the same</div>
<div>lock held as when you call pthread_cond_wait.</div>
<div><br>
</div>
<div><br>
</div>
<div>On a pthread system, the almost-posix condition variables and exactly-posix</div>
<div>mutexes are just pass through to the underlying pthreads.</div>
<div><br>
</div>
<div><br>
</div>
<div>On a win32 system, an almost-posix condition variable is similar</div>
<div>to what we have now, except, because we know this lock is held,</div>
<div>"XWait" can remove some of the leave/enter cycles and remove race conditions.</div>
<div><br>
</div>
<div><br>
</div>
<div>Essentially we only need a "monitor", which is one lock and one condition variable.</div>
<div><br>
</div>
<div><br>
</div>
<div>OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.</div>
<div><br>
</div>
<div> - Jay</div>
<div><br>
<br>
<div>
<hr id="ecxstopSpelling">
From: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>
To: <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a>; <a href="mailto:m3devel@elegosoft.com">
m3devel@elegosoft.com</a><br>
Subject: RE: A win32 thread test case<br>
Date: Sat, 30 Jul 2016 21:18:08 +0000<br>
<br>
<style><!--
.ExternalClass .ecxhmmessage P {
padding:0px;
}
.ExternalClass body.ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">
<div>I fixed the new_slots problem.</div>
<div><br>
</div>
<div><br>
</div>
<div>I agree there are problems here.</div>
<div> I think Java got away with a similar implementation</div>
<div> because their interface is a bit different.</div>
<div> i.e. the lock and condition are merged, and/or notification</div>
<div> requires the caller to take the lock. We can't do this.</div>
<div> We allow signal/broadcast w/o holding a lock.</div>
<div> We'd have to resort to a global lock I guess.</div>
<div><br>
</div>
<div><br>
</div>
<div> I suspect nobody else uses this solution.</div>
<div><br>
</div>
<div><br>
</div>
<div>I believe we can solve it better and worse than others.</div>
<div><br>
</div>
<div><br>
</div>
<div>Most libraries do not have their own "CreateThread" function.</div>
<div>That is, most libraries let you call pthread_create or Win32 CreateThread,</div>
<div>and the library will interoperate with any thread that comes its way.</div>
<div>And most libraries don't seem to use thread locals in their solution.</div>
<div>By "most libraries", I mean the pthreads on win32 package and Boost.</div>
<div><br>
</div>
<div><br>
</div>
<div>Modula-3 isn't clearly this way. This isn't great, but it is the current</div>
<div>situation. I believe it is fixable.</div>
<div>That is, Modula-3 I believe requires using its library to create threads.</div>
<div>This is not great for interoperation. You want to be able to be used</div>
<div>by code that isn't away of you, that just creates threads in the "normal" way</div>
<div>for their platform.</div>
<div><br>
</div>
<div><br>
</div>
<div>And then, either the current state, or a fix I have in mind, can be taken</div>
<div>advantage of to do "directed notify" w/o creating a kernel event</div>
<div>per wait or notify, like other solutions do.</div>
<div><br>
</div>
<div><br>
</div>
<div>As to the fix, to not require threads go through our "CreateThread",</div>
<div>I believe on Windows (which is all that is relevant here), we should have</div>
<div>a DllMain that allocates thread locals.</div>
<div><br>
</div>
<div><br>
</div>
<div>We can only easily have a DllMain if we are a .dll.</div>
<div>Or we can use __declspec(thread).</div>
<div>__declspec(thread) works if the .exe is statically linked to the exe or the dll,</div>
<div>or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.</div>
<div>Perhaps that is good enough.</div>
<div><br>
</div>
<div><br>
</div>
<div>If we require Vista then we can just drop a bunch of our code and use</div>
<div>the Win32 condition variables presumably.</div>
<div>I remain curious how the condition variables work there, i.e. we can't</div>
<div>implement them ourselves, but it is possible they either require kernel</div>
<div>support or are well beyond our abilities.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div> - Jay</div>
<br>
<br>
<br>
<br>
<div>> Date: Thu, 28 Jul 2016 15:00:45 -0500<br>
> From: <a href="mailto:rodney_bates@lcwb.coop">rodney_bates@lcwb.coop</a><br>
> To: <a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a>; <a href="mailto:jay.krell@cornell.edu">
jay.krell@cornell.edu</a><br>
> Subject: A win32 thread test case<br>
> <br>
> I have attached a test case program designed to expose one of the bugs<br>
> I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX.<br>
> I expect it to fail on Windows, but don't have a Windows machine I<br>
> can try it on.<br>
> <br>
> Anybody willing to try it? It's a self-contained Main module. Just<br>
> compile and run it. It will announce what it finds.<br>
> <br>
> <br>
> -- <br>
> Rodney Bates<br>
> <a href="mailto:rodney.m.bates@acm.org">rodney.m.bates@acm.org</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>M3devel mailing list</span><br>
<span><a href="mailto:M3devel@elegosoft.com">M3devel@elegosoft.com</a></span><br>
<span><a href="https://m3lists.elegosoft.com/mailman/listinfo/m3devel">https://m3lists.elegosoft.com/mailman/listinfo/m3devel</a></span><br>
</div>
</blockquote>
</div></blockquote></div></blockquote></body></html>