[M3devel] A win32 thread test case
Jay
jay.krell at cornell.edu
Tue Aug 30 03:22:49 CEST 2016
The summary though: my "recent" (years ago) implementation of condition variables et. al. was unconvincing, or more likely wrong. "Condition variables et. al." -- tied into threads and mutexes. I replaced it again. The replacement is based closely on the pthreads version. I noticed though that the pthread use of pthread condition variables does not require much, so win32 pre-vista mapped easily enough.
And now with the strong resemblance, more code sharing should be possible, but hasn't been done yet. A thin abstraction layer will be needed.
- Jay
> On Aug 29, 2016, at 1:03 PM, Jay <jay.krell at cornell.edu> wrote:
>
> There was a lull in mails, but I haven't made this jump yet anyway.
>
> - Jay
>
>> On Aug 29, 2016, at 6:28 AM, "Hosking, Antony L" <hosking at purdue.edu> wrote:
>>
>> Jay, I've not seen these commits or accompanying emails. (Not sure why!)
>> Can you summarize what is going on here?
>>
>> Thanks,
>>
>> Tony
>>
>> Sent from my iPad
>>
>> On Aug 29, 2016, at 10:49 PM, Jay K <jay.krell at cornell.edu> wrote:
>>
>>> I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code.
>>>
>>> Win32 will then, for now, be one of the "Posix" platforms with "direct suspend".
>>>
>>>
>>> Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected).
>>>
>>> There are still a few steps to get there.
>>>
>>> One of them is naming.
>>> So far I'm thinking of names like:
>>> ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion
>>> ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event
>>>
>>> Another thin abstraction is needed around computing times/timeouts.
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sun, 31 Jul 2016 20:45:48 +0000
>>>
>>> I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress.
>>> It is attached.
>>> I also removed the indirection on critical sections -- exposing the size to Modula3.
>>>
>>> I went with just an event as I indicated.
>>> So the ThreadPThread.m3 code, but with slight changes:
>>> - waits specify time to wait until time to wait until
>>> - event instead of pthread_cond
>>> - same maintenance of list of waiters
>>>
>>> Reasonable?
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sun, 31 Jul 2016 10:37:30 +0000
>>>
>>> a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.
>>> An even per thread should work.
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sun, 31 Jul 2016 04:24:14 +0000
>>>
>>> Something like attached?
>>> Still testing..might be crashing.
>>> A lot of the code and patterns come from ThreadPThread.m3.
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sun, 31 Jul 2016 02:52:02 +0000
>>>
>>> While I agree this is confusing, it is holding together for me and I have mostly coded it up.
>>>
>>> Repeating myself:
>>> In this scheme there are two types of condition variable.
>>> There are Modula-3 condition variables and posix-like/schmidt-like condition variables.
>>>
>>> The posix-like/smidt-like condition variables:
>>> Can be implemented directly via posix condition variables (if Windows pre-Vista had them).
>>> They are implemented as a slight modification of the Smidt counter method.
>>> The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible
>>> to hold the same lock as he holds when calling cond_wait.
>>>
>>> Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.
>>> I still have to work through your examples.
>>>
>>> Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.
>>>
>>> This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.
>>> XPause changes to call XWait, essentially.
>>>
>>> I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.
>>>
>>> If this works, we should be able to share more code between the pthread and win32 implementations.
>>> We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sun, 31 Jul 2016 00:47:24 +0000
>>>
>>> ThreadPThread.m3 had the same narrow startup race condition.
>>> I fixed it.
>>> Again I'm not seeing commit mails.
>>>
>>> On the larger matter:
>>>
>>> I think there is a solution, in that, if you look at
>>> ThreadPThread.m3 and how it uses Posix condition variables,
>>> in that it does a fair amount of the work itself,
>>> you can term this "condition within condition".
>>>
>>>
>>> So I think maybe we can use the counting solution
>>> and 1) merge the locks and/or 2) take the lock in Signal/Broadcast.
>>>
>>>
>>> But I have to look at it more.
>>>
>>>
>>>
>>> Specifically, in ThreadPThread.m3:
>>>
>>> Broadcast:
>>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
>>> next := t.nextWaiter;
>>> t.nextWaiter := NIL;
>>> t.waitingOn := NIL;
>>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
>>>
>>> Signal:
>>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
>>> t.nextWaiter := NIL;
>>> t.waitingOn := NIL;
>>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
>>>
>>>
>>> Becaise they signal while locked, we can implement this limited
>>> form of condition variable, and then use the Modula-3 pthread logic
>>> using that.
>>>
>>>
>>> Make some sense?
>>>
>>>
>>> Essentially we will have almost-posix condition variables,
>>> and exactly-posix mutexes, and then we can use just the pthread code
>>> on top of that.
>>>
>>>
>>> Almost-posix condition variable is a posix condition variable, except
>>> it is guaranteed you will call pthread_cond_signal with the same
>>> lock held as when you call pthread_cond_wait.
>>>
>>>
>>> On a pthread system, the almost-posix condition variables and exactly-posix
>>> mutexes are just pass through to the underlying pthreads.
>>>
>>>
>>> On a win32 system, an almost-posix condition variable is similar
>>> to what we have now, except, because we know this lock is held,
>>> "XWait" can remove some of the leave/enter cycles and remove race conditions.
>>>
>>>
>>> Essentially we only need a "monitor", which is one lock and one condition variable.
>>>
>>>
>>> OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.
>>>
>>> - Jay
>>>
>>>
>>> From: jay.krell at cornell.edu
>>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>>> Subject: RE: A win32 thread test case
>>> Date: Sat, 30 Jul 2016 21:18:08 +0000
>>>
>>> I fixed the new_slots problem.
>>>
>>>
>>> I agree there are problems here.
>>> I think Java got away with a similar implementation
>>> because their interface is a bit different.
>>> i.e. the lock and condition are merged, and/or notification
>>> requires the caller to take the lock. We can't do this.
>>> We allow signal/broadcast w/o holding a lock.
>>> We'd have to resort to a global lock I guess.
>>>
>>>
>>> I suspect nobody else uses this solution.
>>>
>>>
>>> I believe we can solve it better and worse than others.
>>>
>>>
>>> Most libraries do not have their own "CreateThread" function.
>>> That is, most libraries let you call pthread_create or Win32 CreateThread,
>>> and the library will interoperate with any thread that comes its way.
>>> And most libraries don't seem to use thread locals in their solution.
>>> By "most libraries", I mean the pthreads on win32 package and Boost.
>>>
>>>
>>> Modula-3 isn't clearly this way. This isn't great, but it is the current
>>> situation. I believe it is fixable.
>>> That is, Modula-3 I believe requires using its library to create threads.
>>> This is not great for interoperation. You want to be able to be used
>>> by code that isn't away of you, that just creates threads in the "normal" way
>>> for their platform.
>>>
>>>
>>> And then, either the current state, or a fix I have in mind, can be taken
>>> advantage of to do "directed notify" w/o creating a kernel event
>>> per wait or notify, like other solutions do.
>>>
>>>
>>> As to the fix, to not require threads go through our "CreateThread",
>>> I believe on Windows (which is all that is relevant here), we should have
>>> a DllMain that allocates thread locals.
>>>
>>>
>>> We can only easily have a DllMain if we are a .dll.
>>> Or we can use __declspec(thread).
>>> __declspec(thread) works if the .exe is statically linked to the exe or the dll,
>>> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.
>>> Perhaps that is good enough.
>>>
>>>
>>> If we require Vista then we can just drop a bunch of our code and use
>>> the Win32 condition variables presumably.
>>> I remain curious how the condition variables work there, i.e. we can't
>>> implement them ourselves, but it is possible they either require kernel
>>> support or are well beyond our abilities.
>>>
>>>
>>>
>>> - Jay
>>>
>>>
>>>
>>>
>>> > Date: Thu, 28 Jul 2016 15:00:45 -0500
>>> > From: rodney_bates at lcwb.coop
>>> > To: m3devel at elegosoft.com; jay.krell at cornell.edu
>>> > Subject: A win32 thread test case
>>> >
>>> > I have attached a test case program designed to expose one of the bugs
>>> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX.
>>> > I expect it to fail on Windows, but don't have a Windows machine I
>>> > can try it on.
>>> >
>>> > Anybody willing to try it? It's a self-contained Main module. Just
>>> > compile and run it. It will announce what it finds.
>>> >
>>> >
>>> > --
>>> > Rodney Bates
>>> > rodney.m.bates at acm.org
>>> _______________________________________________
>>> M3devel mailing list
>>> M3devel at elegosoft.com
>>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20160829/eec96653/attachment-0001.html>
More information about the M3devel
mailing list