[M3devel] A win32 thread test case
Jay
jay.krell at cornell.edu
Mon Aug 29 22:03:21 CEST 2016
There was a lull in mails, but I haven't made this jump yet anyway.
- Jay
> On Aug 29, 2016, at 6:28 AM, "Hosking, Antony L" <hosking at purdue.edu> wrote:
>
> Jay, I've not seen these commits or accompanying emails. (Not sure why!)
> Can you summarize what is going on here?
>
> Thanks,
>
> Tony
>
> Sent from my iPad
>
> On Aug 29, 2016, at 10:49 PM, Jay K <jay.krell at cornell.edu> wrote:
>
>> I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code.
>>
>> Win32 will then, for now, be one of the "Posix" platforms with "direct suspend".
>>
>>
>> Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected).
>>
>> There are still a few steps to get there.
>>
>> One of them is naming.
>> So far I'm thinking of names like:
>> ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion
>> ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event
>>
>> Another thin abstraction is needed around computing times/timeouts.
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sun, 31 Jul 2016 20:45:48 +0000
>>
>> I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress.
>> It is attached.
>> I also removed the indirection on critical sections -- exposing the size to Modula3.
>>
>> I went with just an event as I indicated.
>> So the ThreadPThread.m3 code, but with slight changes:
>> - waits specify time to wait until time to wait until
>> - event instead of pthread_cond
>> - same maintenance of list of waiters
>>
>> Reasonable?
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sun, 31 Jul 2016 10:37:30 +0000
>>
>> a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.
>> An even per thread should work.
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sun, 31 Jul 2016 04:24:14 +0000
>>
>> Something like attached?
>> Still testing..might be crashing.
>> A lot of the code and patterns come from ThreadPThread.m3.
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sun, 31 Jul 2016 02:52:02 +0000
>>
>> While I agree this is confusing, it is holding together for me and I have mostly coded it up.
>>
>> Repeating myself:
>> In this scheme there are two types of condition variable.
>> There are Modula-3 condition variables and posix-like/schmidt-like condition variables.
>>
>> The posix-like/smidt-like condition variables:
>> Can be implemented directly via posix condition variables (if Windows pre-Vista had them).
>> They are implemented as a slight modification of the Smidt counter method.
>> The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible
>> to hold the same lock as he holds when calling cond_wait.
>>
>> Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.
>> I still have to work through your examples.
>>
>> Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.
>>
>> This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.
>> XPause changes to call XWait, essentially.
>>
>> I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.
>>
>> If this works, we should be able to share more code between the pthread and win32 implementations.
>> We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sun, 31 Jul 2016 00:47:24 +0000
>>
>> ThreadPThread.m3 had the same narrow startup race condition.
>> I fixed it.
>> Again I'm not seeing commit mails.
>>
>> On the larger matter:
>>
>> I think there is a solution, in that, if you look at
>> ThreadPThread.m3 and how it uses Posix condition variables,
>> in that it does a fair amount of the work itself,
>> you can term this "condition within condition".
>>
>>
>> So I think maybe we can use the counting solution
>> and 1) merge the locks and/or 2) take the lock in Signal/Broadcast.
>>
>>
>> But I have to look at it more.
>>
>>
>>
>> Specifically, in ThreadPThread.m3:
>>
>> Broadcast:
>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
>> next := t.nextWaiter;
>> t.nextWaiter := NIL;
>> t.waitingOn := NIL;
>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
>>
>> Signal:
>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
>> t.nextWaiter := NIL;
>> t.waitingOn := NIL;
>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
>>
>>
>> Becaise they signal while locked, we can implement this limited
>> form of condition variable, and then use the Modula-3 pthread logic
>> using that.
>>
>>
>> Make some sense?
>>
>>
>> Essentially we will have almost-posix condition variables,
>> and exactly-posix mutexes, and then we can use just the pthread code
>> on top of that.
>>
>>
>> Almost-posix condition variable is a posix condition variable, except
>> it is guaranteed you will call pthread_cond_signal with the same
>> lock held as when you call pthread_cond_wait.
>>
>>
>> On a pthread system, the almost-posix condition variables and exactly-posix
>> mutexes are just pass through to the underlying pthreads.
>>
>>
>> On a win32 system, an almost-posix condition variable is similar
>> to what we have now, except, because we know this lock is held,
>> "XWait" can remove some of the leave/enter cycles and remove race conditions.
>>
>>
>> Essentially we only need a "monitor", which is one lock and one condition variable.
>>
>>
>> OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.
>>
>> - Jay
>>
>>
>> From: jay.krell at cornell.edu
>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com
>> Subject: RE: A win32 thread test case
>> Date: Sat, 30 Jul 2016 21:18:08 +0000
>>
>> I fixed the new_slots problem.
>>
>>
>> I agree there are problems here.
>> I think Java got away with a similar implementation
>> because their interface is a bit different.
>> i.e. the lock and condition are merged, and/or notification
>> requires the caller to take the lock. We can't do this.
>> We allow signal/broadcast w/o holding a lock.
>> We'd have to resort to a global lock I guess.
>>
>>
>> I suspect nobody else uses this solution.
>>
>>
>> I believe we can solve it better and worse than others.
>>
>>
>> Most libraries do not have their own "CreateThread" function.
>> That is, most libraries let you call pthread_create or Win32 CreateThread,
>> and the library will interoperate with any thread that comes its way.
>> And most libraries don't seem to use thread locals in their solution.
>> By "most libraries", I mean the pthreads on win32 package and Boost.
>>
>>
>> Modula-3 isn't clearly this way. This isn't great, but it is the current
>> situation. I believe it is fixable.
>> That is, Modula-3 I believe requires using its library to create threads.
>> This is not great for interoperation. You want to be able to be used
>> by code that isn't away of you, that just creates threads in the "normal" way
>> for their platform.
>>
>>
>> And then, either the current state, or a fix I have in mind, can be taken
>> advantage of to do "directed notify" w/o creating a kernel event
>> per wait or notify, like other solutions do.
>>
>>
>> As to the fix, to not require threads go through our "CreateThread",
>> I believe on Windows (which is all that is relevant here), we should have
>> a DllMain that allocates thread locals.
>>
>>
>> We can only easily have a DllMain if we are a .dll.
>> Or we can use __declspec(thread).
>> __declspec(thread) works if the .exe is statically linked to the exe or the dll,
>> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.
>> Perhaps that is good enough.
>>
>>
>> If we require Vista then we can just drop a bunch of our code and use
>> the Win32 condition variables presumably.
>> I remain curious how the condition variables work there, i.e. we can't
>> implement them ourselves, but it is possible they either require kernel
>> support or are well beyond our abilities.
>>
>>
>>
>> - Jay
>>
>>
>>
>>
>> > Date: Thu, 28 Jul 2016 15:00:45 -0500
>> > From: rodney_bates at lcwb.coop
>> > To: m3devel at elegosoft.com; jay.krell at cornell.edu
>> > Subject: A win32 thread test case
>> >
>> > I have attached a test case program designed to expose one of the bugs
>> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX.
>> > I expect it to fail on Windows, but don't have a Windows machine I
>> > can try it on.
>> >
>> > Anybody willing to try it? It's a self-contained Main module. Just
>> > compile and run it. It will announce what it finds.
>> >
>> >
>> > --
>> > Rodney Bates
>> > rodney.m.bates at acm.org
>> _______________________________________________
>> M3devel mailing list
>> M3devel at elegosoft.com
>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20160829/1f6d770b/attachment-0001.html>
More information about the M3devel
mailing list