[M3devel] A win32 thread test case

Mon Aug 1 01:54:56 CEST 2016

I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code.
Win32 will then, for now, be one of the "Posix" platforms with "direct suspend".

Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected).
There are still a few steps to get there.
One of them is naming.So far I'm thinking of names like:    ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion   ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event
Another thin abstraction is needed around computing times/timeouts.
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sun, 31 Jul 2016 20:45:48 +0000

I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. It is attached. I also removed the indirection on critical sections -- exposing the size to Modula3.  I went with just an event as I indicated. So the ThreadPThread.m3 code, but with slight changes:    - waits specify time to wait until time to wait until    - event instead of pthread_cond    - same maintenance of list of waiters 
Reasonable?
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sun, 31 Jul 2016 10:37:30 +0000

a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.An even per thread should work.
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sun, 31 Jul 2016 04:24:14 +0000

Something like attached?Still testing..might be crashing.A lot of the code and patterns come from ThreadPThread.m3.
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sun, 31 Jul 2016 02:52:02 +0000

While I agree this is confusing, it is holding together for me and I have mostly coded it up.
Repeating myself:  In this scheme there are two types of condition variable. There are Modula-3 condition variables and posix-like/schmidt-like condition variables.
The posix-like/smidt-like condition variables:   Can be implemented directly via posix condition variables (if Windows pre-Vista had them).   They are implemented as a slight modification of the Smidt counter method.   The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible   to hold the same lock as he holds when calling cond_wait.
  Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.  I still have to work through your examples.
Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.
This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.XPause changes to call XWait, essentially.
I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.
If this works, we should be able to share more code between the pthread and win32 implementations.We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sun, 31 Jul 2016 00:47:24 +0000

ThreadPThread.m3 had the same narrow startup race condition. I fixed it.Again I'm not seeing commit mails.
On the larger matter:
I think there is a solution, in that, if you look atThreadPThread.m3 and how it uses Posix condition variables,in that it does a fair amount of the work itself,you can term this "condition within condition".

So I think maybe we can use the counting solutionand 1) merge the locks and/or 2) take the lock in Signal/Broadcast.

But I have to look at it more.

Specifically, in ThreadPThread.m3:
Broadcast:        WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;        next := t.nextWaiter;        t.nextWaiter := NIL;        t.waitingOn := NIL;        WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
Signal:      WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;      t.nextWaiter := NIL;      t.waitingOn := NIL;      WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;

Becaise they signal while locked, we can implement this limitedform of condition variable, and then use the Modula-3 pthread logicusing that.

Make some sense?

Essentially we will have almost-posix condition variables,and exactly-posix mutexes, and then we can use just the pthread codeon top of that.

Almost-posix condition variable is a posix condition variable, exceptit is guaranteed you will call pthread_cond_signal with the samelock held as when you call pthread_cond_wait.

On a pthread system, the almost-posix condition variables and exactly-posixmutexes are just pass through to the underlying pthreads.

On a win32 system, an almost-posix condition variable is similarto what we have now, except, because we know this lock is held,"XWait" can remove some of the leave/enter cycles and remove race conditions.

Essentially we only need a "monitor", which is one lock and one condition variable.

OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.
 - Jay

From: jay.krell at cornell.edu
To: rodney.m.bates at acm.org; m3devel at elegosoft.com
Subject: RE: A win32 thread test case
Date: Sat, 30 Jul 2016 21:18:08 +0000

I fixed the new_slots problem.

I agree there are problems here.  I think Java got away with a similar implementation  because their interface is a bit different.  i.e. the lock and condition are merged, and/or notification  requires the caller to take the lock. We can't do this.  We allow signal/broadcast w/o holding a lock.  We'd have to resort to a global lock I guess.

  I suspect nobody else uses this solution.

I believe we can solve it better and worse than others.

Most libraries do not have their own "CreateThread" function.That is, most libraries let you call pthread_create or Win32 CreateThread,and the library will interoperate with any thread that comes its way.And most libraries don't seem to use thread locals in their solution.By "most libraries", I mean the pthreads on win32 package and Boost.

Modula-3 isn't clearly this way. This isn't great, but it is the currentsituation. I believe it is fixable.That is, Modula-3 I believe requires using its library to create threads.This is not great for interoperation. You want to be able to be usedby code that isn't away of you, that just creates threads in the "normal" wayfor their platform.

And then, either the current state, or a fix I have in mind, can be takenadvantage of to do "directed notify" w/o creating a kernel eventper wait or notify, like other solutions do.

As to the fix, to not require threads go through our "CreateThread",I believe on Windows (which is all that is relevant here), we should havea DllMain that allocates thread locals.

We can only easily have a DllMain if we are a .dll.Or we can use __declspec(thread).__declspec(thread) works if the .exe is statically linked to the exe or the dll,or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.Perhaps that is good enough.

If we require Vista then we can just drop a bunch of our code and usethe Win32 condition variables presumably.I remain curious how the condition variables work there, i.e. we can'timplement them ourselves, but it is possible they either require kernelsupport or are well beyond our abilities.

 - Jay

> Date: Thu, 28 Jul 2016 15:00:45 -0500
> From: rodney_bates at lcwb.coop
> To: m3devel at elegosoft.com; jay.krell at cornell.edu
> Subject: A win32 thread test case
> 
> I have attached a test case program designed to expose one of the bugs
> I think I see in ThreadWin32.m3.  It runs correctly on AMD64_LINUX.
> I expect it to fail on Windows, but don't have a Windows machine I
> can try it on.
> 
> Anybody willing to try it?  It's a self-contained Main module.  Just
> compile and run it.  It will announce what it finds.
> 
> 
> -- 
> Rodney Bates
> rodney.m.bates at acm.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20160731/e89ada4e/attachment-0001.html>