[M3devel] A win32 thread test case
Jay K
jay.krell at cornell.edu
Wed Aug 3 12:05:50 CEST 2016
I agree there was some cruft in there. I'm still working on it.
Also agreed "watEvent" == "cond".
It turns out, I guess, pthreads doesn't events, but an event is a subset of a condition variable, and that is all Modula-3 uses them for.
I've been spinning on this stuff a while.
Here are some dilemnas: To expose to M3 the size of CRITICAL_SECTION and/or CONTEXT. Advantages: slight performance
Disadvantges: I hide the sizes on the pthread side already. Pointers are easy to check for initializaton. CONTEXT varies for every processor. CRITICAL_SECTION basically doesn't -- it has some pointers/alignment, but that is all. I don't like saying "ADR" all over.
I'm leaning toward exposing CRITICAL_SECTION but not CONTEXT.
If/when Win32 and pthreads are merged though, this optimization will go away.
More importantly: To initialize the event on demand or not.The thing is, under low resources, that can fail.And then both the caller in ThreadWin32.m3 and its caller need to deal with it.I think for simplicity, do more up front allocation i.e. in CreateT.
I wish we could do more.That is, users say NEW(MUTEX) or NEW(Condition) and we don't get a chance to run InitializeCriticalSection at that point. We have to delay it until they call LOCK. This bothers me some.
I know zero initialization is good, but I'd like some infrastructural way to trigger non-zero initialization for a heap allocated type w/o dpeending on the caller to have to say .init() or such. i.e. C++ constructors.
I know zeros are good and efficient.And I know putting off initialization can be profitable, because it might never be done.But the other choice is nice too -- to aggressively do initialization earlier, without having to synchronize on-demand-ness.
I'll keep poking at this and send a revision later...could still be days.
- Jay
> Date: Mon, 1 Aug 2016 14:51:36 -0500
> From: rodney_bates at lcwb.coop
> To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com
> Subject: Re: A win32 thread test case
>
> It looks very reasonable, overall. The same PThread mechanism for managing/
> awakening waiters on Mutexs and Conditions should fix both the ticket count bugs
> I mentioned.
>
> Some minor points:
>
> fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t
> are unused (and unneeded -- leftovers from the flawed generation count scheme).
>
> Comments on the parameters of the CreateEvent call would help a reader, especially
> the second, which importantly makes it an auto-reset event.
>
> I don't think you can assert m.initialized at the beginning of UnlockMutex.
> ThreadPThread initializes it, if necessary. UnlockMutex can be called directly
> by application code (well, almost directly, through a couple of wrappers),
> and it could pass an initialized M3 Mutex in.
>
> I am having trouble following the connection mentioned in various comments among a lock,
> a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION),
> a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear
> seem to be the lock and cond fields of an Activation? But while act.lock is held while
> calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require
> it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock
> even be known to or affect SetEvent?
>
> I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated
> LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by
> Windows calls.
>
>
> On 07/31/2016 03:45 PM, Jay K wrote:
> > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress.
> > It is attached.
> > I also removed the indirection on critical sections -- exposing the size to Modula3.
> > I went with just an event as I indicated.
> > So the ThreadPThread.m3 code, but with slight changes:
> > - waits specify time to wait until time to wait until
> > - event instead of pthread_cond
> > - same maintenance of list of waiters
> >
> > Reasonable?
> >
> > - Jay
> >
> >
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!
> ---
> > From: jay.krell at cornell.edu
> > To: rodney.m.bates at acm.org; m3devel at elegosoft.com
> > Subject: RE: A win32 thread test case
> > Date: Sun, 31 Jul 2016 10:37:30 +0000
> >
> > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.
> > An even per thread should work.
> >
> > - Jay
> >
> >
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!
> ---
> > From: jay.krell at cornell.edu
> > To: rodney.m.bates at acm.org; m3devel at elegosoft.com
> > Subject: RE: A win32 thread test case
> > Date: Sun, 31 Jul 2016 04:24:14 +0000
> >
> > Something like attached?
> > Still testing..might be crashing.
> > A lot of the code and patterns come from ThreadPThread.m3.
> >
> > - Jay
> >
> >
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!
> ---
> > From: jay.krell at cornell.edu
> > To: rodney.m.bates at acm.org; m3devel at elegosoft.com
> > Subject: RE: A win32 thread test case
> > Date: Sun, 31 Jul 2016 02:52:02 +0000
> >
> > While I agree this is confusing, it is holding together for me and I have mostly coded it up.
> >
> > Repeating myself:
> > In this scheme there are two types of condition variable.
> > There are Modula-3 condition variables and posix-like/schmidt-like condition variables.
> >
> > The posix-like/smidt-like condition variables:
> > Can be implemented directly via posix condition variables (if Windows pre-Vista had them).
> > They are implemented as a slight modification of the Smidt counter method.
> > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible
> > to hold the same lock as he holds when calling cond_wait.
> >
> > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed.
> > I still have to work through your examples.
> >
> > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3.
> >
> > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.
> > XPause changes to call XWait, essentially.
> >
> > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate.
> >
> > If this works, we should be able to share more code between the pthread and win32 implementations.
> > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast).
> >
> > - Jay
> >
> >
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!
> ---
> > From: jay.krell at cornell.edu
> > To: rodney.m.bates at acm.org; m3devel at elegosoft.com
> > Subject: RE: A win32 thread test case
> > Date: Sun, 31 Jul 2016 00:47:24 +0000
> >
> > ThreadPThread.m3 had the same narrow startup race condition.
> > I fixed it.
> > Again I'm not seeing commit mails.
> >
> > On the larger matter:
> >
> > I think there is a solution, in that, if you look at
> > ThreadPThread.m3 and how it uses Posix condition variables,
> > in that it does a fair amount of the work itself,
> > you can term this "condition within condition".
> >
> >
> > So I think maybe we can use the counting solution
> > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast.
> >
> >
> > But I have to look at it more.
> >
> >
> >
> > Specifically, in ThreadPThread.m3:
> >
> > Broadcast:
> > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
> > next := t.nextWaiter;
> > t.nextWaiter := NIL;
> > t.waitingOn := NIL;
> > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
> >
> > Signal:
> > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END;
> > t.nextWaiter := NIL;
> > t.waitingOn := NIL;
> > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END;
> >
> >
> > Becaise they signal while locked, we can implement this limited
> > form of condition variable, and then use the Modula-3 pthread logic
> > using that.
> >
> >
> > Make some sense?
> >
> >
> > Essentially we will have almost-posix condition variables,
> > and exactly-posix mutexes, and then we can use just the pthread code
> > on top of that.
> >
> >
> > Almost-posix condition variable is a posix condition variable, except
> > it is guaranteed you will call pthread_cond_signal with the same
> > lock held as when you call pthread_cond_wait.
> >
> >
> > On a pthread system, the almost-posix condition variables and exactly-posix
> > mutexes are just pass through to the underlying pthreads.
> >
> >
> > On a win32 system, an almost-posix condition variable is similar
> > to what we have now, except, because we know this lock is held,
> > "XWait" can remove some of the leave/enter cycles and remove race conditions.
> >
> >
> > Essentially we only need a "monitor", which is one lock and one condition variable.
> >
> >
> > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do.
> >
> > - Jay
> >
> >
> > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------!
> ---
> > From: jay.krell at cornell.edu
> > To: rodney.m.bates at acm.org; m3devel at elegosoft.com
> > Subject: RE: A win32 thread test case
> > Date: Sat, 30 Jul 2016 21:18:08 +0000
> >
> > I fixed the new_slots problem.
> >
> >
> > I agree there are problems here.
> > I think Java got away with a similar implementation
> > because their interface is a bit different.
> > i.e. the lock and condition are merged, and/or notification
> > requires the caller to take the lock. We can't do this.
> > We allow signal/broadcast w/o holding a lock.
> > We'd have to resort to a global lock I guess.
> >
> >
> > I suspect nobody else uses this solution.
> >
> >
> > I believe we can solve it better and worse than others.
> >
> >
> > Most libraries do not have their own "CreateThread" function.
> > That is, most libraries let you call pthread_create or Win32 CreateThread,
> > and the library will interoperate with any thread that comes its way.
> > And most libraries don't seem to use thread locals in their solution.
> > By "most libraries", I mean the pthreads on win32 package and Boost.
> >
> >
> > Modula-3 isn't clearly this way. This isn't great, but it is the current
> > situation. I believe it is fixable.
> > That is, Modula-3 I believe requires using its library to create threads.
> > This is not great for interoperation. You want to be able to be used
> > by code that isn't away of you, that just creates threads in the "normal" way
> > for their platform.
> >
> >
> > And then, either the current state, or a fix I have in mind, can be taken
> > advantage of to do "directed notify" w/o creating a kernel event
> > per wait or notify, like other solutions do.
> >
> >
> > As to the fix, to not require threads go through our "CreateThread",
> > I believe on Windows (which is all that is relevant here), we should have
> > a DllMain that allocates thread locals.
> >
> >
> > We can only easily have a DllMain if we are a .dll.
> > Or we can use __declspec(thread).
> > __declspec(thread) works if the .exe is statically linked to the exe or the dll,
> > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.
> > Perhaps that is good enough.
> >
> >
> > If we require Vista then we can just drop a bunch of our code and use
> > the Win32 condition variables presumably.
> > I remain curious how the condition variables work there, i.e. we can't
> > implement them ourselves, but it is possible they either require kernel
> > support or are well beyond our abilities.
> >
> >
> >
> > - Jay
> >
> >
> >
> >
> > > Date: Thu, 28 Jul 2016 15:00:45 -0500
> > > From: rodney_bates at lcwb.coop
> > > To: m3devel at elegosoft.com; jay.krell at cornell.edu
> > > Subject: A win32 thread test case
> > >
> > > I have attached a test case program designed to expose one of the bugs
> > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX.
> > > I expect it to fail on Windows, but don't have a Windows machine I
> > > can try it on.
> > >
> > > Anybody willing to try it? It's a self-contained Main module. Just
> > > compile and run it. It will announce what it finds.
> > >
> > >
> > > --
> > > Rodney Bates
> > > rodney.m.bates at acm.org
>
> --
> Rodney Bates
> rodney.m.bates at acm.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20160803/40c944e1/attachment-0002.html>
More information about the M3devel
mailing list