<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>I agree there was some cruft in there. I'm still working on it.<div><br></div><div>Also agreed "watEvent" == "cond".</div><div><br></div><div>It turns out, I guess, pthreads doesn't events, but an event is a subset of a condition variable, and that is all Modula-3 uses them for.</div><div><br></div><div>I've been spinning on this stuff a while.</div><div><br></div><div>Here are some dilemnas:</div><div> To expose to M3 the size of CRITICAL_SECTION and/or CONTEXT.</div><div> Advantages: slight performance</div><div><br></div><div> Disadvantges:</div><div> I hide the sizes on the pthread side already.</div><div> Pointers are easy to check for initializaton.</div><div> CONTEXT varies for every processor. CRITICAL_SECTION basically doesn't -- it has some pointers/alignment, but that is all.</div><div> I don't like saying "ADR" all over.</div><div><br></div><div>I'm leaning toward exposing CRITICAL_SECTION but not CONTEXT.</div><div><br></div><div>If/when Win32 and pthreads are merged though, this optimization will go away.</div><div><br></div><div>More importantly: To initialize the event on demand or not.</div><div>The thing is, under low resources, that can fail.</div><div>And then both the caller in ThreadWin32.m3 and its caller need to deal with it.</div><div>I think for simplicity, do more up front allocation i.e. in CreateT.</div><div><br></div><div>I wish we could do more.</div><div>That is, users say NEW(MUTEX) or NEW(Condition) and we don't get a chance to run InitializeCriticalSection at that point. We have to delay it until they call LOCK. This bothers me some.</div><div><br></div><div>I know zero initialization is good, but I'd like some infrastructural way to trigger non-zero initialization for a heap allocated type w/o dpeending on the caller to have to say .init() or such. i.e. C++ constructors.</div><div><br></div><div>I know zeros are good and efficient.</div><div>And I know putting off initialization can be profitable, because it might never be done.</div><div>But the other choice is nice too -- to aggressively do initialization earlier, without having to synchronize on-demand-ness.</div><div><br></div><div>I'll keep poking at this and send a revision later...could still be days.</div><div><br></div><div> - Jay<br><br><br><div>> Date: Mon, 1 Aug 2016 14:51:36 -0500<br>> From: rodney_bates@lcwb.coop<br>> To: jay.krell@cornell.edu; rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> Subject: Re: A win32 thread test case<br>> <br>> It looks very reasonable, overall. The same PThread mechanism for managing/<br>> awakening waiters on Mutexs and Conditions should fix both the ticket count bugs<br>> I mentioned.<br>> <br>> Some minor points:<br>> <br>> fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t<br>> are unused (and unneeded -- leftovers from the flawed generation count scheme).<br>> <br>> Comments on the parameters of the CreateEvent call would help a reader, especially<br>> the second, which importantly makes it an auto-reset event.<br>> <br>> I don't think you can assert m.initialized at the beginning of UnlockMutex.<br>> ThreadPThread initializes it, if necessary. UnlockMutex can be called directly<br>> by application code (well, almost directly, through a couple of wrappers),<br>> and it could pass an initialized M3 Mutex in.<br>> <br>> I am having trouble following the connection mentioned in various comments among a lock,<br>> a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION),<br>> a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear<br>> seem to be the lock and cond fields of an Activation? But while act.lock is held while<br>> calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require<br>> it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock<br>> even be known to or affect SetEvent?<br>> <br>> I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated<br>> LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by<br>> Windows calls.<br>> <br>> <br>> On 07/31/2016 03:45 PM, Jay K wrote:<br>> > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress.<br>> > It is attached.<br>> > I also removed the indirection on critical sections -- exposing the size to Modula3.<br>> > I went with just an event as I indicated.<br>> > So the ThreadPThread.m3 code, but with slight changes:<br>> > - waits specify time to wait until time to wait until<br>> > - event instead of pthread_cond<br>> > - same maintenance of list of waiters<br>> ><br>> > Reasonable?<br>> ><br>> > - Jay<br>> ><br>> ><br>> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!<br>> ---<br>> > From: jay.krell@cornell.edu<br>> > To: rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> > Subject: RE: A win32 thread test case<br>> > Date: Sun, 31 Jul 2016 10:37:30 +0000<br>> ><br>> > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.<br>> > An even per thread should work.<br>> ><br>> > - Jay<br>> ><br>> ><br>> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!<br>> ---<br>> > From: jay.krell@cornell.edu<br>> > To: rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> > Subject: RE: A win32 thread test case<br>> > Date: Sun, 31 Jul 2016 04:24:14 +0000<br>> ><br>> > Something like attached?<br>> > Still testing..might be crashing.<br>> > A lot of the code and patterns come from ThreadPThread.m3.<br>> ><br>> > - Jay<br>> ><br>> ><br>> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!<br>> ---<br>> > From: jay.krell@cornell.edu<br>> > To: rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> > Subject: RE: A win32 thread test case<br>> > Date: Sun, 31 Jul 2016 02:52:02 +0000<br>> ><br>> > While I agree this is confusing, it is holding together for me and I have mostly coded it up.<br>> ><br>> > Repeating myself:<br>> > In this scheme there are two types of condition variable.<br>> > There are Modula-3 condition variables and posix-like/schmidt-like condition variables.<br>> ><br>> > The posix-like/smidt-like condition variables:<br>> > Can be implemented directly via posix condition variables (if Windows pre-Vista had them).<br>> > They are implemented as a slight modification of the Smidt counter method.<br>> > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible<br>> > to hold the same lock as he holds when calling cond_wait.<br>> ><br>> > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.<br>> > I still have to work through your examples.<br>> ><br>> > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.<br>> ><br>> > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.<br>> > XPause changes to call XWait, essentially.<br>> ><br>> > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.<br>> ><br>> > If this works, we should be able to share more code between the pthread and win32 implementations.<br>> > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).<br>> ><br>> > - Jay<br>> ><br>> ><br>> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!<br>> ---<br>> > From: jay.krell@cornell.edu<br>> > To: rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> > Subject: RE: A win32 thread test case<br>> > Date: Sun, 31 Jul 2016 00:47:24 +0000<br>> ><br>> > ThreadPThread.m3 had the same narrow startup race condition.<br>> > I fixed it.<br>> > Again I'm not seeing commit mails.<br>> ><br>> > On the larger matter:<br>> ><br>> > I think there is a solution, in that, if you look at<br>> > ThreadPThread.m3 and how it uses Posix condition variables,<br>> > in that it does a fair amount of the work itself,<br>> > you can term this "condition within condition".<br>> ><br>> ><br>> > So I think maybe we can use the counting solution<br>> > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast.<br>> ><br>> ><br>> > But I have to look at it more.<br>> ><br>> ><br>> ><br>> > Specifically, in ThreadPThread.m3:<br>> ><br>> > Broadcast:<br>> > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;<br>> > next := t.nextWaiter;<br>> > t.nextWaiter := NIL;<br>> > t.waitingOn := NIL;<br>> > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;<br>> ><br>> > Signal:<br>> > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;<br>> > t.nextWaiter := NIL;<br>> > t.waitingOn := NIL;<br>> > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;<br>> ><br>> ><br>> > Becaise they signal while locked, we can implement this limited<br>> > form of condition variable, and then use the Modula-3 pthread logic<br>> > using that.<br>> ><br>> ><br>> > Make some sense?<br>> ><br>> ><br>> > Essentially we will have almost-posix condition variables,<br>> > and exactly-posix mutexes, and then we can use just the pthread code<br>> > on top of that.<br>> ><br>> ><br>> > Almost-posix condition variable is a posix condition variable, except<br>> > it is guaranteed you will call pthread_cond_signal with the same<br>> > lock held as when you call pthread_cond_wait.<br>> ><br>> ><br>> > On a pthread system, the almost-posix condition variables and exactly-posix<br>> > mutexes are just pass through to the underlying pthreads.<br>> ><br>> ><br>> > On a win32 system, an almost-posix condition variable is similar<br>> > to what we have now, except, because we know this lock is held,<br>> > "XWait" can remove some of the leave/enter cycles and remove race conditions.<br>> ><br>> ><br>> > Essentially we only need a "monitor", which is one lock and one condition variable.<br>> ><br>> ><br>> > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.<br>> ><br>> > - Jay<br>> ><br>> ><br>> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!<br>> ---<br>> > From: jay.krell@cornell.edu<br>> > To: rodney.m.bates@acm.org; m3devel@elegosoft.com<br>> > Subject: RE: A win32 thread test case<br>> > Date: Sat, 30 Jul 2016 21:18:08 +0000<br>> ><br>> > I fixed the new_slots problem.<br>> ><br>> ><br>> > I agree there are problems here.<br>> > I think Java got away with a similar implementation<br>> > because their interface is a bit different.<br>> > i.e. the lock and condition are merged, and/or notification<br>> > requires the caller to take the lock. We can't do this.<br>> > We allow signal/broadcast w/o holding a lock.<br>> > We'd have to resort to a global lock I guess.<br>> ><br>> ><br>> > I suspect nobody else uses this solution.<br>> ><br>> ><br>> > I believe we can solve it better and worse than others.<br>> ><br>> ><br>> > Most libraries do not have their own "CreateThread" function.<br>> > That is, most libraries let you call pthread_create or Win32 CreateThread,<br>> > and the library will interoperate with any thread that comes its way.<br>> > And most libraries don't seem to use thread locals in their solution.<br>> > By "most libraries", I mean the pthreads on win32 package and Boost.<br>> ><br>> ><br>> > Modula-3 isn't clearly this way. This isn't great, but it is the current<br>> > situation. I believe it is fixable.<br>> > That is, Modula-3 I believe requires using its library to create threads.<br>> > This is not great for interoperation. You want to be able to be used<br>> > by code that isn't away of you, that just creates threads in the "normal" way<br>> > for their platform.<br>> ><br>> ><br>> > And then, either the current state, or a fix I have in mind, can be taken<br>> > advantage of to do "directed notify" w/o creating a kernel event<br>> > per wait or notify, like other solutions do.<br>> ><br>> ><br>> > As to the fix, to not require threads go through our "CreateThread",<br>> > I believe on Windows (which is all that is relevant here), we should have<br>> > a DllMain that allocates thread locals.<br>> ><br>> ><br>> > We can only easily have a DllMain if we are a .dll.<br>> > Or we can use __declspec(thread).<br>> > __declspec(thread) works if the .exe is statically linked to the exe or the dll,<br>> > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.<br>> > Perhaps that is good enough.<br>> ><br>> ><br>> > If we require Vista then we can just drop a bunch of our code and use<br>> > the Win32 condition variables presumably.<br>> > I remain curious how the condition variables work there, i.e. we can't<br>> > implement them ourselves, but it is possible they either require kernel<br>> > support or are well beyond our abilities.<br>> ><br>> ><br>> ><br>> > - Jay<br>> ><br>> ><br>> ><br>> ><br>> > > Date: Thu, 28 Jul 2016 15:00:45 -0500<br>> > > From: rodney_bates@lcwb.coop<br>> > > To: m3devel@elegosoft.com; jay.krell@cornell.edu<br>> > > Subject: A win32 thread test case<br>> > ><br>> > > I have attached a test case program designed to expose one of the bugs<br>> > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX.<br>> > > I expect it to fail on Windows, but don't have a Windows machine I<br>> > > can try it on.<br>> > ><br>> > > Anybody willing to try it? It's a self-contained Main module. Just<br>> > > compile and run it. It will announce what it finds.<br>> > ><br>> > ><br>> > > --<br>> > > Rodney Bates<br>> > > rodney.m.bates@acm.org<br>> <br>> -- <br>> Rodney Bates<br>> rodney.m.bates@acm.org<br></div></div> </div></body>
</html>