From jay.krell at cornell.edu Mon Aug 1 01:54:56 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 31 Jul 2016 23:54:56 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , , Message-ID: I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code. Win32 will then, for now, be one of the "Posix" platforms with "direct suspend". Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected). There are still a few steps to get there. One of them is naming.So far I'm thinking of names like: ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event Another thin abstraction is needed around computing times/timeouts. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 20:45:48 +0000 I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. It is attached. I also removed the indirection on critical sections -- exposing the size to Modula3. I went with just an event as I indicated. So the ThreadPThread.m3 code, but with slight changes: - waits specify time to wait until time to wait until - event instead of pthread_cond - same maintenance of list of waiters Reasonable? - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 10:37:30 +0000 a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.An even per thread should work. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 04:24:14 +0000 Something like attached?Still testing..might be crashing.A lot of the code and patterns come from ThreadPThread.m3. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 02:52:02 +0000 While I agree this is confusing, it is holding together for me and I have mostly coded it up. Repeating myself: In this scheme there are two types of condition variable. There are Modula-3 condition variables and posix-like/schmidt-like condition variables. The posix-like/smidt-like condition variables: Can be implemented directly via posix condition variables (if Windows pre-Vista had them). They are implemented as a slight modification of the Smidt counter method. The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible to hold the same lock as he holds when calling cond_wait. Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. I still have to work through your examples. Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.XPause changes to call XWait, essentially. I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. If this works, we should be able to share more code between the pthread and win32 implementations.We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 00:47:24 +0000 ThreadPThread.m3 had the same narrow startup race condition. I fixed it.Again I'm not seeing commit mails. On the larger matter: I think there is a solution, in that, if you look atThreadPThread.m3 and how it uses Posix condition variables,in that it does a fair amount of the work itself,you can term this "condition within condition". So I think maybe we can use the counting solutionand 1) merge the locks and/or 2) take the lock in Signal/Broadcast. But I have to look at it more. Specifically, in ThreadPThread.m3: Broadcast: WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; next := t.nextWaiter; t.nextWaiter := NIL; t.waitingOn := NIL; WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; Signal: WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; t.nextWaiter := NIL; t.waitingOn := NIL; WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; Becaise they signal while locked, we can implement this limitedform of condition variable, and then use the Modula-3 pthread logicusing that. Make some sense? Essentially we will have almost-posix condition variables,and exactly-posix mutexes, and then we can use just the pthread codeon top of that. Almost-posix condition variable is a posix condition variable, exceptit is guaranteed you will call pthread_cond_signal with the samelock held as when you call pthread_cond_wait. On a pthread system, the almost-posix condition variables and exactly-posixmutexes are just pass through to the underlying pthreads. On a win32 system, an almost-posix condition variable is similarto what we have now, except, because we know this lock is held,"XWait" can remove some of the leave/enter cycles and remove race conditions. Essentially we only need a "monitor", which is one lock and one condition variable. OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sat, 30 Jul 2016 21:18:08 +0000 I fixed the new_slots problem. I agree there are problems here. I think Java got away with a similar implementation because their interface is a bit different. i.e. the lock and condition are merged, and/or notification requires the caller to take the lock. We can't do this. We allow signal/broadcast w/o holding a lock. We'd have to resort to a global lock I guess. I suspect nobody else uses this solution. I believe we can solve it better and worse than others. Most libraries do not have their own "CreateThread" function.That is, most libraries let you call pthread_create or Win32 CreateThread,and the library will interoperate with any thread that comes its way.And most libraries don't seem to use thread locals in their solution.By "most libraries", I mean the pthreads on win32 package and Boost. Modula-3 isn't clearly this way. This isn't great, but it is the currentsituation. I believe it is fixable.That is, Modula-3 I believe requires using its library to create threads.This is not great for interoperation. You want to be able to be usedby code that isn't away of you, that just creates threads in the "normal" wayfor their platform. And then, either the current state, or a fix I have in mind, can be takenadvantage of to do "directed notify" w/o creating a kernel eventper wait or notify, like other solutions do. As to the fix, to not require threads go through our "CreateThread",I believe on Windows (which is all that is relevant here), we should havea DllMain that allocates thread locals. We can only easily have a DllMain if we are a .dll.Or we can use __declspec(thread).__declspec(thread) works if the .exe is statically linked to the exe or the dll,or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.Perhaps that is good enough. If we require Vista then we can just drop a bunch of our code and usethe Win32 condition variables presumably.I remain curious how the condition variables work there, i.e. we can'timplement them ourselves, but it is possible they either require kernelsupport or are well beyond our abilities. - Jay > Date: Thu, 28 Jul 2016 15:00:45 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: A win32 thread test case > > I have attached a test case program designed to expose one of the bugs > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > I expect it to fail on Windows, but don't have a Windows machine I > can try it on. > > Anybody willing to try it? It's a self-contained Main module. Just > compile and run it. It will announce what it finds. > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 1 02:43:16 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Aug 2016 00:43:16 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop> Message-ID: <579E9B0A.2010305@lcwb.coop> On 07/30/2016 04:18 PM, Jay K wrote: > I fixed the new_slots problem. > > > I agree there are problems here. > I think Java got away with a similar implementation > because their interface is a bit different. > i.e. the lock and condition are merged, and/or notification > requires the caller to take the lock. We can't do this. > We allow signal/broadcast w/o holding a lock. > We'd have to resort to a global lock I guess. > Ah, I think I am finally getting what you mean by merging the mutex and Condition. Java's primitives merge these at the level the application sees, not inside the implementation. Off hand, I think this might be more awkward for some use cases, but anyway, we can't change the semantics of our primitives. > > I suspect nobody else uses this solution. > > > I believe we can solve it better and worse than others. > > > Most libraries do not have their own "CreateThread" function. > That is, most libraries let you call pthread_create or Win32 CreateThread, > and the library will interoperate with any thread that comes its way. > And most libraries don't seem to use thread locals in their solution. > By "most libraries", I mean the pthreads on win32 package and Boost. > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > situation. I believe it is fixable. > That is, Modula-3 I believe requires using its library to create threads. > This is not great for interoperation. You want to be able to be used > by code that isn't away of you, that just creates threads in the "normal" way > for their platform. > > > And then, either the current state, or a fix I have in mind, can be taken > advantage of to do "directed notify" w/o creating a kernel event > per wait or notify, like other solutions do. > > > As to the fix, to not require threads go through our "CreateThread", > I believe on Windows (which is all that is relevant here), we should have > a DllMain that allocates thread locals. > > While it is appealing from a performance point of view, I doubt directly using other synchronization primitives from other libraries can be made to work. We have a number of differences. Threads, Mutexs, and Conditions, all three, are M3 objects, and can be subtyped in Modula-3's object subtype semantics. Even without this, they all contain various properties that other libraries' objects almost certainly don't. They are traced and garbage-collected. Threads, at least, actively participate in the GC implementation. The M3 Fork and Join mechanism, the Alert mechanism, and the exception handling implementation mechanism are all intertwined. There are some automatic cleanup things happening. There are just too many differences. > We can only easily have a DllMain if we are a .dll. > Or we can use __declspec(thread). > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > Perhaps that is good enough. > > > If we require Vista then we can just drop a bunch of our code and use > the Win32 condition variables presumably. > I remain curious how the condition variables work there, i.e. we can't > implement them ourselves, but it is possible they either require kernel > support or are well beyond our abilities. > > > > - Jay > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: A win32 thread test case > > > > I have attached a test case program designed to expose one of the bugs > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > I expect it to fail on Windows, but don't have a Windows machine I > > can try it on. > > > > Anybody willing to try it? It's a self-contained Main module. Just > > compile and run it. It will announce what it finds. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 1 03:02:58 2016 From: jay.krell at cornell.edu (Jay) Date: Mon, 01 Aug 2016 01:02:58 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579E9B0A.2010305@lcwb.coop> References: <579A646D.3020404@lcwb.coop> <579E9B0A.2010305@lcwb.coop> Message-ID: Right. But by merging at the application layer, they can be merged in the implementation as well. But right, we can't. I don't expect other languages to use the m3 runtime but for other languages to call into m3 code, which uses the m3 runtime. The other languages all Interoperate with each other: C++, C#, Java. They can all call into each other without having to know they are and without using other than native platform thread creation. For example on Windows they can all be in-proc COM servers. It is a flaw to require other than platform native thread creation. - Jay > On Jul 31, 2016, at 5:42 PM, "Rodney M. Bates" wrote: > > > >> On 07/30/2016 04:18 PM, Jay K wrote: >> I fixed the new_slots problem. >> >> >> I agree there are problems here. >> I think Java got away with a similar implementation >> because their interface is a bit different. >> i.e. the lock and condition are merged, and/or notification >> requires the caller to take the lock. We can't do this. >> We allow signal/broadcast w/o holding a lock. >> We'd have to resort to a global lock I guess. > > Ah, I think I am finally getting what you mean by merging the mutex and > Condition. Java's primitives merge these at the level the application > sees, not inside the implementation. Off hand, I think this might be more > awkward for some use cases, but anyway, we can't change the semantics of > our primitives. > >> >> I suspect nobody else uses this solution. >> >> >> I believe we can solve it better and worse than others. >> >> >> Most libraries do not have their own "CreateThread" function. >> That is, most libraries let you call pthread_create or Win32 CreateThread, >> and the library will interoperate with any thread that comes its way. >> And most libraries don't seem to use thread locals in their solution. >> By "most libraries", I mean the pthreads on win32 package and Boost. >> >> >> Modula-3 isn't clearly this way. This isn't great, but it is the current >> situation. I believe it is fixable. >> That is, Modula-3 I believe requires using its library to create threads. >> This is not great for interoperation. You want to be able to be used >> by code that isn't away of you, that just creates threads in the "normal" way >> for their platform. >> >> >> And then, either the current state, or a fix I have in mind, can be taken >> advantage of to do "directed notify" w/o creating a kernel event >> per wait or notify, like other solutions do. >> >> >> As to the fix, to not require threads go through our "CreateThread", >> I believe on Windows (which is all that is relevant here), we should have >> a DllMain that allocates thread locals. > > While it is appealing from a performance point of view, I doubt directly using > other synchronization primitives from other libraries can be made to work. We > have a number of differences. Threads, Mutexs, and Conditions, all three, > are M3 objects, and can be subtyped in Modula-3's object subtype semantics. > Even without this, they all contain various properties that other libraries' > objects almost certainly don't. They are traced and garbage-collected. Threads, > at least, actively participate in the GC implementation. The M3 Fork and Join > mechanism, the Alert mechanism, and the exception handling implementation > mechanism are all intertwined. There are some automatic cleanup things happening. > There are just too many differences. > > >> We can only easily have a DllMain if we are a .dll. >> Or we can use __declspec(thread). >> __declspec(thread) works if the .exe is statically linked to the exe or the dll, >> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. >> Perhaps that is good enough. >> >> >> If we require Vista then we can just drop a bunch of our code and use >> the Win32 condition variables presumably. >> I remain curious how the condition variables work there, i.e. we can't >> implement them ourselves, but it is possible they either require kernel >> support or are well beyond our abilities. >> >> >> >> - Jay >> >> >> >> >> > Date: Thu, 28 Jul 2016 15:00:45 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: A win32 thread test case >> > >> > I have attached a test case program designed to expose one of the bugs >> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. >> > I expect it to fail on Windows, but don't have a Windows machine I >> > can try it on. >> > >> > Anybody willing to try it? It's a self-contained Main module. Just >> > compile and run it. It will announce what it finds. >> > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 1 09:31:52 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:31:52 -0000 Subject: [M3devel] pthread_join? Message-ID: Should we use pthread_join? http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_join.html "it is a convenience and you can do X" And we do that X. I ask for one or two reasons. In making ThreadWin32.m3 resemble ThreadPThread.m3 more, there is usually no loss of efficiency.However this is an area where I percieve, just from the design/implementation, some loss. So, wondering..I noticed pthread_join exists. I suspect if we used pthread_join, then both Win32 could still resemble pthread, but regain the efficiency.In particular, so that multiple independent joins do not contend on the same mutex. Is there the problem of: http://man7.org/linux/man-pages/man3/pthread_join.3.html " Joining with a thread that has previously been joined results in undefined behavior." yes we want it to be well defined to raise an exception?Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 1 09:35:02 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:35:02 -0000 Subject: [M3devel] Modula-3 thread alert vs. Win32 thread alert? Message-ID: Modula-3 threads and Win32 threads both a notion of "alert". It seems it is almost the same notion. Where did it originate, I wonder? It seems we could implement Modula-3 on top of Win32. However I'm unsure. In Modula-3, the only way to alert a thread is to call a specific alert function. In Windows there are approximately two ways to be alerted: 1. complete an asynchronous I/O. 2. Request an "APC" (asynchronous procedure call) unrelated to I/O. One could simulate Modula-3 alerting by queueing an APC, that does nothing. However, where I'm worried, is that completing asynchronous I/O would alert a thread, unexpectedly to Modula-3. So don't use it? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 1 09:52:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:52:07 -0000 Subject: [M3devel] RegisterFinalCleanup multiple for same object? Message-ID: Can I call RTHeapRep.RegisterFinalCleanup multiple times for the same object?i.e. if it has multiple on-demand initialized resources, not all initialized at the same time? I guess I can test it.From the implementation, I think so.I think it creates a weak reference per call, and I can have any number per object. Thank you, -Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 1 21:55:36 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Aug 2016 19:55:36 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , Message-ID: <579FA848.6010400@lcwb.coop> It looks very reasonable, overall. The same PThread mechanism for managing/ awakening waiters on Mutexs and Conditions should fix both the ticket count bugs I mentioned. Some minor points: fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t are unused (and unneeded -- leftovers from the flawed generation count scheme). Comments on the parameters of the CreateEvent call would help a reader, especially the second, which importantly makes it an auto-reset event. I don't think you can assert m.initialized at the beginning of UnlockMutex. ThreadPThread initializes it, if necessary. UnlockMutex can be called directly by application code (well, almost directly, through a couple of wrappers), and it could pass an initialized M3 Mutex in. I am having trouble following the connection mentioned in various comments among a lock, a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear seem to be the lock and cond fields of an Activation? But while act.lock is held while calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock even be known to or affect SetEvent? I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by Windows calls. On 07/31/2016 03:45 PM, Jay K wrote: > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > It is attached. > I also removed the indirection on critical sections -- exposing the size to Modula3. > I went with just an event as I indicated. > So the ThreadPThread.m3 code, but with slight changes: > - waits specify time to wait until time to wait until > - event instead of pthread_cond > - same maintenance of list of waiters > > Reasonable? > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > An even per thread should work. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > Something like attached? > Still testing..might be crashing. > A lot of the code and patterns come from ThreadPThread.m3. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > Repeating myself: > In this scheme there are two types of condition variable. > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > The posix-like/smidt-like condition variables: > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > They are implemented as a slight modification of the Smidt counter method. > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > to hold the same lock as he holds when calling cond_wait. > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > I still have to work through your examples. > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > XPause changes to call XWait, essentially. > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > If this works, we should be able to share more code between the pthread and win32 implementations. > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > ThreadPThread.m3 had the same narrow startup race condition. > I fixed it. > Again I'm not seeing commit mails. > > On the larger matter: > > I think there is a solution, in that, if you look at > ThreadPThread.m3 and how it uses Posix condition variables, > in that it does a fair amount of the work itself, > you can term this "condition within condition". > > > So I think maybe we can use the counting solution > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > But I have to look at it more. > > > > Specifically, in ThreadPThread.m3: > > Broadcast: > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > next := t.nextWaiter; > t.nextWaiter := NIL; > t.waitingOn := NIL; > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > Signal: > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > t.nextWaiter := NIL; > t.waitingOn := NIL; > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > Becaise they signal while locked, we can implement this limited > form of condition variable, and then use the Modula-3 pthread logic > using that. > > > Make some sense? > > > Essentially we will have almost-posix condition variables, > and exactly-posix mutexes, and then we can use just the pthread code > on top of that. > > > Almost-posix condition variable is a posix condition variable, except > it is guaranteed you will call pthread_cond_signal with the same > lock held as when you call pthread_cond_wait. > > > On a pthread system, the almost-posix condition variables and exactly-posix > mutexes are just pass through to the underlying pthreads. > > > On a win32 system, an almost-posix condition variable is similar > to what we have now, except, because we know this lock is held, > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > I fixed the new_slots problem. > > > I agree there are problems here. > I think Java got away with a similar implementation > because their interface is a bit different. > i.e. the lock and condition are merged, and/or notification > requires the caller to take the lock. We can't do this. > We allow signal/broadcast w/o holding a lock. > We'd have to resort to a global lock I guess. > > > I suspect nobody else uses this solution. > > > I believe we can solve it better and worse than others. > > > Most libraries do not have their own "CreateThread" function. > That is, most libraries let you call pthread_create or Win32 CreateThread, > and the library will interoperate with any thread that comes its way. > And most libraries don't seem to use thread locals in their solution. > By "most libraries", I mean the pthreads on win32 package and Boost. > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > situation. I believe it is fixable. > That is, Modula-3 I believe requires using its library to create threads. > This is not great for interoperation. You want to be able to be used > by code that isn't away of you, that just creates threads in the "normal" way > for their platform. > > > And then, either the current state, or a fix I have in mind, can be taken > advantage of to do "directed notify" w/o creating a kernel event > per wait or notify, like other solutions do. > > > As to the fix, to not require threads go through our "CreateThread", > I believe on Windows (which is all that is relevant here), we should have > a DllMain that allocates thread locals. > > > We can only easily have a DllMain if we are a .dll. > Or we can use __declspec(thread). > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > Perhaps that is good enough. > > > If we require Vista then we can just drop a bunch of our code and use > the Win32 condition variables presumably. > I remain curious how the condition variables work there, i.e. we can't > implement them ourselves, but it is possible they either require kernel > support or are well beyond our abilities. > > > > - Jay > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: A win32 thread test case > > > > I have attached a test case program designed to expose one of the bugs > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > I expect it to fail on Windows, but don't have a Windows machine I > > can try it on. > > > > Anybody willing to try it? It's a self-contained Main module. Just > > compile and run it. It will announce what it finds. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 3 12:05:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 03 Aug 2016 10:05:50 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579FA848.6010400@lcwb.coop> References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop> Message-ID: I agree there was some cruft in there. I'm still working on it. Also agreed "watEvent" == "cond". It turns out, I guess, pthreads doesn't events, but an event is a subset of a condition variable, and that is all Modula-3 uses them for. I've been spinning on this stuff a while. Here are some dilemnas: To expose to M3 the size of CRITICAL_SECTION and/or CONTEXT. Advantages: slight performance Disadvantges: I hide the sizes on the pthread side already. Pointers are easy to check for initializaton. CONTEXT varies for every processor. CRITICAL_SECTION basically doesn't -- it has some pointers/alignment, but that is all. I don't like saying "ADR" all over. I'm leaning toward exposing CRITICAL_SECTION but not CONTEXT. If/when Win32 and pthreads are merged though, this optimization will go away. More importantly: To initialize the event on demand or not.The thing is, under low resources, that can fail.And then both the caller in ThreadWin32.m3 and its caller need to deal with it.I think for simplicity, do more up front allocation i.e. in CreateT. I wish we could do more.That is, users say NEW(MUTEX) or NEW(Condition) and we don't get a chance to run InitializeCriticalSection at that point. We have to delay it until they call LOCK. This bothers me some. I know zero initialization is good, but I'd like some infrastructural way to trigger non-zero initialization for a heap allocated type w/o dpeending on the caller to have to say .init() or such. i.e. C++ constructors. I know zeros are good and efficient.And I know putting off initialization can be profitable, because it might never be done.But the other choice is nice too -- to aggressively do initialization earlier, without having to synchronize on-demand-ness. I'll keep poking at this and send a revision later...could still be days. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 4 09:25:58 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 04 Aug 2016 07:25:58 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579FA848.6010400@lcwb.coop> References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop> Message-ID: Right, it wasn't done. I have "staged" the maybe final version here: https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32.m3-2 There is a minor update then here, just removing some code. https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32C.c-2 Some notes: I removed some of the on-demand initialization. It can/could fail, so it has to be very careful, not to fail with partial state. I expose sizeof(CRITICAL_SECTION) into Modula-3. There is fairly portable, in that Win64 will work already. If we further merge with ThreadPThread.m3 though, this will become hidden again, at slight cost, and potential reintroduction of failure points (failure to heap allocate the CRITICAL_SECTION in LockMutex). ThreadPThread.m3 already is like this. I need to check if it is robust under low memory. Similarly I call CreateEvent early instead of on-demand. I based even more of it on ThreadPThread.m3. Perhaps unnecessarily so and with some efficiency loss. In particular, all thread joins are now serialized. And alerting is probably less efficient. The joining part could be better in ThreadPThread.m3 as well, I think, and then maybe follow suite here? Your test passes. I can rebuild the world, which I know exercises the code some, since I managed to break it while changing it. Thoughts? Commit? I believe this structuring is now quite amenable to sharing the vast bulk of the code with ThreadPThread.m3. We really just need to rename some things so they are an "abstraction layer" that directly maps to pthread and win32, instead of also reusing the pthread/win32 names. Specifically, pthread mutex, win32 critical_section, pthread cond, and Win32 event. The heap locking would need attention do. PThread and Win32 vary in structure there. Maybe stay that way, but still move most of the code to somewhere common. And I have to see if user threads can share code too. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 5 05:53:59 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 05 Aug 2016 03:53:59 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop>, Message-ID: This is commited now, with one slight change per Rodney -- checking in UnlockMutex for uninitialized (and therefore unlocked) mutex. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Thu, 4 Aug 2016 07:25:54 +0000 Right, it wasn't done. I have "staged" the maybe final version here: https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32.m3-2 There is a minor update then here, just removing some code. https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32C.c-2 Some notes: I removed some of the on-demand initialization. It can/could fail, so it has to be very careful, not to fail with partial state. I expose sizeof(CRITICAL_SECTION) into Modula-3. There is fairly portable, in that Win64 will work already. If we further merge with ThreadPThread.m3 though, this will become hidden again, at slight cost, and potential reintroduction of failure points (failure to heap allocate the CRITICAL_SECTION in LockMutex). ThreadPThread.m3 already is like this. I need to check if it is robust under low memory. Similarly I call CreateEvent early instead of on-demand. I based even more of it on ThreadPThread.m3. Perhaps unnecessarily so and with some efficiency loss. In particular, all thread joins are now serialized. And alerting is probably less efficient. The joining part could be better in ThreadPThread.m3 as well, I think, and then maybe follow suite here? Your test passes. I can rebuild the world, which I know exercises the code some, since I managed to break it while changing it. Thoughts? Commit? I believe this structuring is now quite amenable to sharing the vast bulk of the code with ThreadPThread.m3. We really just need to rename some things so they are an "abstraction layer" that directly maps to pthread and win32, instead of also reusing the pthread/win32 names. Specifically, pthread mutex, win32 critical_section, pthread cond, and Win32 event. The heap locking would need attention do. PThread and Win32 vary in structure there. Maybe stay that way, but still move most of the code to somewhere common. And I have to see if user threads can share code too. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 9 02:54:30 2016 From: lists at darko.org (Darko Volaric) Date: Tue, 09 Aug 2016 00:54:30 -0000 Subject: [M3devel] GitHub commit logging to M3 Commit mailing list gone? Message-ID: I'm no longer seeing commits from GitHub in the mailing list. Did someone change somethig? - Darko -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 25 19:58:47 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 25 Aug 2016 17:58:47 +0000 (UTC) Subject: [M3devel] Information lost, now recovered? References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 | ? | | ? | ? | ? | ? | ? | | HLRL: DEC Prism Project Documentation, Retrospective | HoffmanLabsFrom the HoffmanLabs Reading List (HLRL), some Digital Equipment Corporation (DEC) Prism project history has become available on BitSavers. | | | | Ver en labs.hoffmanlabs.com | Vista previa por Yahoo | | | | ? | Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Sat Aug 27 11:26:53 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 27 Aug 2016 11:26:53 +0200 Subject: [M3devel] Fw: Just an upate for modula3 Message-ID: <20160827112653.ac12dbed8a29349ab6f9cc34@elegosoft.com> FYI Begin forwarded message: Date: Fri, 26 Aug 2016 16:42:23 -0700 From: Brian Davis To: webmaster at modula3.org Subject: Just an upate for modula3 Modula3 Cm3 compiler including all 17 packages compile successfully on Fedora 24 kernel-4.6.7-300.fc24.x86_64 #1 SMP Everything works. :) -- Brian Davis brpia at fastmail.com -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Sun Aug 28 19:03:20 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Aug 2016 17:03:20 +0000 (UTC) Subject: [M3devel] Fw: Information lost, now recovered? In-Reply-To: <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> Message-ID: <378750309.1624269.1472403800154@mail.yahoo.com> On thursday August 25th, 2016 12:58, Daniel Alejandro Benavides D. wrote: Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sun Aug 28 19:32:53 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Aug 2016 17:32:53 +0000 (UTC) Subject: [M3devel] Fw2: Information lost, now recovered? In-Reply-To: <378750309.1624269.1472403800154@mail.yahoo.com> References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> <378750309.1624269.1472403800154@mail.yahoo.com> Message-ID: <623087746.1678529.1472405573085@mail.yahoo.com> Sorry if this repeatsOn Sunday August 28th, 2016 12:03, Daniel Alejandro Benavides D. wrote: On thursday August 25th, 2016 12:58, Daniel Alejandro Benavides D. wrote: Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 1 01:54:56 2016 From: jay.krell at cornell.edu (Jay K) Date: Sun, 31 Jul 2016 23:54:56 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , , Message-ID: I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code. Win32 will then, for now, be one of the "Posix" platforms with "direct suspend". Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected). There are still a few steps to get there. One of them is naming.So far I'm thinking of names like: ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event Another thin abstraction is needed around computing times/timeouts. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 20:45:48 +0000 I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. It is attached. I also removed the indirection on critical sections -- exposing the size to Modula3. I went with just an event as I indicated. So the ThreadPThread.m3 code, but with slight changes: - waits specify time to wait until time to wait until - event instead of pthread_cond - same maintenance of list of waiters Reasonable? - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 10:37:30 +0000 a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed.An even per thread should work. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 04:24:14 +0000 Something like attached?Still testing..might be crashing.A lot of the code and patterns come from ThreadPThread.m3. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 02:52:02 +0000 While I agree this is confusing, it is holding together for me and I have mostly coded it up. Repeating myself: In this scheme there are two types of condition variable. There are Modula-3 condition variables and posix-like/schmidt-like condition variables. The posix-like/smidt-like condition variables: Can be implemented directly via posix condition variables (if Windows pre-Vista had them). They are implemented as a slight modification of the Smidt counter method. The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible to hold the same lock as he holds when calling cond_wait. Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. I still have to work through your examples. Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal.XPause changes to call XWait, essentially. I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. If this works, we should be able to share more code between the pthread and win32 implementations.We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sun, 31 Jul 2016 00:47:24 +0000 ThreadPThread.m3 had the same narrow startup race condition. I fixed it.Again I'm not seeing commit mails. On the larger matter: I think there is a solution, in that, if you look atThreadPThread.m3 and how it uses Posix condition variables,in that it does a fair amount of the work itself,you can term this "condition within condition". So I think maybe we can use the counting solutionand 1) merge the locks and/or 2) take the lock in Signal/Broadcast. But I have to look at it more. Specifically, in ThreadPThread.m3: Broadcast: WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; next := t.nextWaiter; t.nextWaiter := NIL; t.waitingOn := NIL; WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; Signal: WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; t.nextWaiter := NIL; t.waitingOn := NIL; WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; Becaise they signal while locked, we can implement this limitedform of condition variable, and then use the Modula-3 pthread logicusing that. Make some sense? Essentially we will have almost-posix condition variables,and exactly-posix mutexes, and then we can use just the pthread codeon top of that. Almost-posix condition variable is a posix condition variable, exceptit is guaranteed you will call pthread_cond_signal with the samelock held as when you call pthread_cond_wait. On a pthread system, the almost-posix condition variables and exactly-posixmutexes are just pass through to the underlying pthreads. On a win32 system, an almost-posix condition variable is similarto what we have now, except, because we know this lock is held,"XWait" can remove some of the leave/enter cycles and remove race conditions. Essentially we only need a "monitor", which is one lock and one condition variable. OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Sat, 30 Jul 2016 21:18:08 +0000 I fixed the new_slots problem. I agree there are problems here. I think Java got away with a similar implementation because their interface is a bit different. i.e. the lock and condition are merged, and/or notification requires the caller to take the lock. We can't do this. We allow signal/broadcast w/o holding a lock. We'd have to resort to a global lock I guess. I suspect nobody else uses this solution. I believe we can solve it better and worse than others. Most libraries do not have their own "CreateThread" function.That is, most libraries let you call pthread_create or Win32 CreateThread,and the library will interoperate with any thread that comes its way.And most libraries don't seem to use thread locals in their solution.By "most libraries", I mean the pthreads on win32 package and Boost. Modula-3 isn't clearly this way. This isn't great, but it is the currentsituation. I believe it is fixable.That is, Modula-3 I believe requires using its library to create threads.This is not great for interoperation. You want to be able to be usedby code that isn't away of you, that just creates threads in the "normal" wayfor their platform. And then, either the current state, or a fix I have in mind, can be takenadvantage of to do "directed notify" w/o creating a kernel eventper wait or notify, like other solutions do. As to the fix, to not require threads go through our "CreateThread",I believe on Windows (which is all that is relevant here), we should havea DllMain that allocates thread locals. We can only easily have a DllMain if we are a .dll.Or we can use __declspec(thread).__declspec(thread) works if the .exe is statically linked to the exe or the dll,or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista.Perhaps that is good enough. If we require Vista then we can just drop a bunch of our code and usethe Win32 condition variables presumably.I remain curious how the condition variables work there, i.e. we can'timplement them ourselves, but it is possible they either require kernelsupport or are well beyond our abilities. - Jay > Date: Thu, 28 Jul 2016 15:00:45 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: A win32 thread test case > > I have attached a test case program designed to expose one of the bugs > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > I expect it to fail on Windows, but don't have a Windows machine I > can try it on. > > Anybody willing to try it? It's a self-contained Main module. Just > compile and run it. It will announce what it finds. > > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 1 02:43:16 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Aug 2016 00:43:16 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop> Message-ID: <579E9B0A.2010305@lcwb.coop> On 07/30/2016 04:18 PM, Jay K wrote: > I fixed the new_slots problem. > > > I agree there are problems here. > I think Java got away with a similar implementation > because their interface is a bit different. > i.e. the lock and condition are merged, and/or notification > requires the caller to take the lock. We can't do this. > We allow signal/broadcast w/o holding a lock. > We'd have to resort to a global lock I guess. > Ah, I think I am finally getting what you mean by merging the mutex and Condition. Java's primitives merge these at the level the application sees, not inside the implementation. Off hand, I think this might be more awkward for some use cases, but anyway, we can't change the semantics of our primitives. > > I suspect nobody else uses this solution. > > > I believe we can solve it better and worse than others. > > > Most libraries do not have their own "CreateThread" function. > That is, most libraries let you call pthread_create or Win32 CreateThread, > and the library will interoperate with any thread that comes its way. > And most libraries don't seem to use thread locals in their solution. > By "most libraries", I mean the pthreads on win32 package and Boost. > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > situation. I believe it is fixable. > That is, Modula-3 I believe requires using its library to create threads. > This is not great for interoperation. You want to be able to be used > by code that isn't away of you, that just creates threads in the "normal" way > for their platform. > > > And then, either the current state, or a fix I have in mind, can be taken > advantage of to do "directed notify" w/o creating a kernel event > per wait or notify, like other solutions do. > > > As to the fix, to not require threads go through our "CreateThread", > I believe on Windows (which is all that is relevant here), we should have > a DllMain that allocates thread locals. > > While it is appealing from a performance point of view, I doubt directly using other synchronization primitives from other libraries can be made to work. We have a number of differences. Threads, Mutexs, and Conditions, all three, are M3 objects, and can be subtyped in Modula-3's object subtype semantics. Even without this, they all contain various properties that other libraries' objects almost certainly don't. They are traced and garbage-collected. Threads, at least, actively participate in the GC implementation. The M3 Fork and Join mechanism, the Alert mechanism, and the exception handling implementation mechanism are all intertwined. There are some automatic cleanup things happening. There are just too many differences. > We can only easily have a DllMain if we are a .dll. > Or we can use __declspec(thread). > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > Perhaps that is good enough. > > > If we require Vista then we can just drop a bunch of our code and use > the Win32 condition variables presumably. > I remain curious how the condition variables work there, i.e. we can't > implement them ourselves, but it is possible they either require kernel > support or are well beyond our abilities. > > > > - Jay > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: A win32 thread test case > > > > I have attached a test case program designed to expose one of the bugs > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > I expect it to fail on Windows, but don't have a Windows machine I > > can try it on. > > > > Anybody willing to try it? It's a self-contained Main module. Just > > compile and run it. It will announce what it finds. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 1 03:02:58 2016 From: jay.krell at cornell.edu (Jay) Date: Mon, 01 Aug 2016 01:02:58 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579E9B0A.2010305@lcwb.coop> References: <579A646D.3020404@lcwb.coop> <579E9B0A.2010305@lcwb.coop> Message-ID: Right. But by merging at the application layer, they can be merged in the implementation as well. But right, we can't. I don't expect other languages to use the m3 runtime but for other languages to call into m3 code, which uses the m3 runtime. The other languages all Interoperate with each other: C++, C#, Java. They can all call into each other without having to know they are and without using other than native platform thread creation. For example on Windows they can all be in-proc COM servers. It is a flaw to require other than platform native thread creation. - Jay > On Jul 31, 2016, at 5:42 PM, "Rodney M. Bates" wrote: > > > >> On 07/30/2016 04:18 PM, Jay K wrote: >> I fixed the new_slots problem. >> >> >> I agree there are problems here. >> I think Java got away with a similar implementation >> because their interface is a bit different. >> i.e. the lock and condition are merged, and/or notification >> requires the caller to take the lock. We can't do this. >> We allow signal/broadcast w/o holding a lock. >> We'd have to resort to a global lock I guess. > > Ah, I think I am finally getting what you mean by merging the mutex and > Condition. Java's primitives merge these at the level the application > sees, not inside the implementation. Off hand, I think this might be more > awkward for some use cases, but anyway, we can't change the semantics of > our primitives. > >> >> I suspect nobody else uses this solution. >> >> >> I believe we can solve it better and worse than others. >> >> >> Most libraries do not have their own "CreateThread" function. >> That is, most libraries let you call pthread_create or Win32 CreateThread, >> and the library will interoperate with any thread that comes its way. >> And most libraries don't seem to use thread locals in their solution. >> By "most libraries", I mean the pthreads on win32 package and Boost. >> >> >> Modula-3 isn't clearly this way. This isn't great, but it is the current >> situation. I believe it is fixable. >> That is, Modula-3 I believe requires using its library to create threads. >> This is not great for interoperation. You want to be able to be used >> by code that isn't away of you, that just creates threads in the "normal" way >> for their platform. >> >> >> And then, either the current state, or a fix I have in mind, can be taken >> advantage of to do "directed notify" w/o creating a kernel event >> per wait or notify, like other solutions do. >> >> >> As to the fix, to not require threads go through our "CreateThread", >> I believe on Windows (which is all that is relevant here), we should have >> a DllMain that allocates thread locals. > > While it is appealing from a performance point of view, I doubt directly using > other synchronization primitives from other libraries can be made to work. We > have a number of differences. Threads, Mutexs, and Conditions, all three, > are M3 objects, and can be subtyped in Modula-3's object subtype semantics. > Even without this, they all contain various properties that other libraries' > objects almost certainly don't. They are traced and garbage-collected. Threads, > at least, actively participate in the GC implementation. The M3 Fork and Join > mechanism, the Alert mechanism, and the exception handling implementation > mechanism are all intertwined. There are some automatic cleanup things happening. > There are just too many differences. > > >> We can only easily have a DllMain if we are a .dll. >> Or we can use __declspec(thread). >> __declspec(thread) works if the .exe is statically linked to the exe or the dll, >> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. >> Perhaps that is good enough. >> >> >> If we require Vista then we can just drop a bunch of our code and use >> the Win32 condition variables presumably. >> I remain curious how the condition variables work there, i.e. we can't >> implement them ourselves, but it is possible they either require kernel >> support or are well beyond our abilities. >> >> >> >> - Jay >> >> >> >> >> > Date: Thu, 28 Jul 2016 15:00:45 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: A win32 thread test case >> > >> > I have attached a test case program designed to expose one of the bugs >> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. >> > I expect it to fail on Windows, but don't have a Windows machine I >> > can try it on. >> > >> > Anybody willing to try it? It's a self-contained Main module. Just >> > compile and run it. It will announce what it finds. >> > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org From jay.krell at cornell.edu Mon Aug 1 09:31:52 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:31:52 -0000 Subject: [M3devel] pthread_join? Message-ID: Should we use pthread_join? http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_join.html "it is a convenience and you can do X" And we do that X. I ask for one or two reasons. In making ThreadWin32.m3 resemble ThreadPThread.m3 more, there is usually no loss of efficiency.However this is an area where I percieve, just from the design/implementation, some loss. So, wondering..I noticed pthread_join exists. I suspect if we used pthread_join, then both Win32 could still resemble pthread, but regain the efficiency.In particular, so that multiple independent joins do not contend on the same mutex. Is there the problem of: http://man7.org/linux/man-pages/man3/pthread_join.3.html " Joining with a thread that has previously been joined results in undefined behavior." yes we want it to be well defined to raise an exception?Thoughts? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 1 09:35:02 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:35:02 -0000 Subject: [M3devel] Modula-3 thread alert vs. Win32 thread alert? Message-ID: Modula-3 threads and Win32 threads both a notion of "alert". It seems it is almost the same notion. Where did it originate, I wonder? It seems we could implement Modula-3 on top of Win32. However I'm unsure. In Modula-3, the only way to alert a thread is to call a specific alert function. In Windows there are approximately two ways to be alerted: 1. complete an asynchronous I/O. 2. Request an "APC" (asynchronous procedure call) unrelated to I/O. One could simulate Modula-3 alerting by queueing an APC, that does nothing. However, where I'm worried, is that completing asynchronous I/O would alert a thread, unexpectedly to Modula-3. So don't use it? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 1 09:52:07 2016 From: jay.krell at cornell.edu (Jay K) Date: Mon, 01 Aug 2016 07:52:07 -0000 Subject: [M3devel] RegisterFinalCleanup multiple for same object? Message-ID: Can I call RTHeapRep.RegisterFinalCleanup multiple times for the same object?i.e. if it has multiple on-demand initialized resources, not all initialized at the same time? I guess I can test it.From the implementation, I think so.I think it creates a weak reference per call, and I can have any number per object. Thank you, -Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon Aug 1 21:55:36 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Mon, 01 Aug 2016 19:55:36 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , Message-ID: <579FA848.6010400@lcwb.coop> It looks very reasonable, overall. The same PThread mechanism for managing/ awakening waiters on Mutexs and Conditions should fix both the ticket count bugs I mentioned. Some minor points: fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t are unused (and unneeded -- leftovers from the flawed generation count scheme). Comments on the parameters of the CreateEvent call would help a reader, especially the second, which importantly makes it an auto-reset event. I don't think you can assert m.initialized at the beginning of UnlockMutex. ThreadPThread initializes it, if necessary. UnlockMutex can be called directly by application code (well, almost directly, through a couple of wrappers), and it could pass an initialized M3 Mutex in. I am having trouble following the connection mentioned in various comments among a lock, a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear seem to be the lock and cond fields of an Activation? But while act.lock is held while calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock even be known to or affect SetEvent? I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by Windows calls. On 07/31/2016 03:45 PM, Jay K wrote: > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > It is attached. > I also removed the indirection on critical sections -- exposing the size to Modula3. > I went with just an event as I indicated. > So the ThreadPThread.m3 code, but with slight changes: > - waits specify time to wait until time to wait until > - event instead of pthread_cond > - same maintenance of list of waiters > > Reasonable? > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > An even per thread should work. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > Something like attached? > Still testing..might be crashing. > A lot of the code and patterns come from ThreadPThread.m3. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > Repeating myself: > In this scheme there are two types of condition variable. > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > The posix-like/smidt-like condition variables: > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > They are implemented as a slight modification of the Smidt counter method. > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > to hold the same lock as he holds when calling cond_wait. > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > I still have to work through your examples. > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > XPause changes to call XWait, essentially. > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > If this works, we should be able to share more code between the pthread and win32 implementations. > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > ThreadPThread.m3 had the same narrow startup race condition. > I fixed it. > Again I'm not seeing commit mails. > > On the larger matter: > > I think there is a solution, in that, if you look at > ThreadPThread.m3 and how it uses Posix condition variables, > in that it does a fair amount of the work itself, > you can term this "condition within condition". > > > So I think maybe we can use the counting solution > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > But I have to look at it more. > > > > Specifically, in ThreadPThread.m3: > > Broadcast: > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > next := t.nextWaiter; > t.nextWaiter := NIL; > t.waitingOn := NIL; > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > Signal: > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > t.nextWaiter := NIL; > t.waitingOn := NIL; > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > Becaise they signal while locked, we can implement this limited > form of condition variable, and then use the Modula-3 pthread logic > using that. > > > Make some sense? > > > Essentially we will have almost-posix condition variables, > and exactly-posix mutexes, and then we can use just the pthread code > on top of that. > > > Almost-posix condition variable is a posix condition variable, except > it is guaranteed you will call pthread_cond_signal with the same > lock held as when you call pthread_cond_wait. > > > On a pthread system, the almost-posix condition variables and exactly-posix > mutexes are just pass through to the underlying pthreads. > > > On a win32 system, an almost-posix condition variable is similar > to what we have now, except, because we know this lock is held, > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > - Jay > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > From: jay.krell at cornell.edu > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: RE: A win32 thread test case > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > I fixed the new_slots problem. > > > I agree there are problems here. > I think Java got away with a similar implementation > because their interface is a bit different. > i.e. the lock and condition are merged, and/or notification > requires the caller to take the lock. We can't do this. > We allow signal/broadcast w/o holding a lock. > We'd have to resort to a global lock I guess. > > > I suspect nobody else uses this solution. > > > I believe we can solve it better and worse than others. > > > Most libraries do not have their own "CreateThread" function. > That is, most libraries let you call pthread_create or Win32 CreateThread, > and the library will interoperate with any thread that comes its way. > And most libraries don't seem to use thread locals in their solution. > By "most libraries", I mean the pthreads on win32 package and Boost. > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > situation. I believe it is fixable. > That is, Modula-3 I believe requires using its library to create threads. > This is not great for interoperation. You want to be able to be used > by code that isn't away of you, that just creates threads in the "normal" way > for their platform. > > > And then, either the current state, or a fix I have in mind, can be taken > advantage of to do "directed notify" w/o creating a kernel event > per wait or notify, like other solutions do. > > > As to the fix, to not require threads go through our "CreateThread", > I believe on Windows (which is all that is relevant here), we should have > a DllMain that allocates thread locals. > > > We can only easily have a DllMain if we are a .dll. > Or we can use __declspec(thread). > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > Perhaps that is good enough. > > > If we require Vista then we can just drop a bunch of our code and use > the Win32 condition variables presumably. > I remain curious how the condition variables work there, i.e. we can't > implement them ourselves, but it is possible they either require kernel > support or are well beyond our abilities. > > > > - Jay > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > Subject: A win32 thread test case > > > > I have attached a test case program designed to expose one of the bugs > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > I expect it to fail on Windows, but don't have a Windows machine I > > can try it on. > > > > Anybody willing to try it? It's a self-contained Main module. Just > > compile and run it. It will announce what it finds. > > > > > > -- > > Rodney Bates > > rodney.m.bates at acm.org -- Rodney Bates rodney.m.bates at acm.org From jay.krell at cornell.edu Wed Aug 3 12:05:50 2016 From: jay.krell at cornell.edu (Jay K) Date: Wed, 03 Aug 2016 10:05:50 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579FA848.6010400@lcwb.coop> References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop> Message-ID: I agree there was some cruft in there. I'm still working on it. Also agreed "watEvent" == "cond". It turns out, I guess, pthreads doesn't events, but an event is a subset of a condition variable, and that is all Modula-3 uses them for. I've been spinning on this stuff a while. Here are some dilemnas: To expose to M3 the size of CRITICAL_SECTION and/or CONTEXT. Advantages: slight performance Disadvantges: I hide the sizes on the pthread side already. Pointers are easy to check for initializaton. CONTEXT varies for every processor. CRITICAL_SECTION basically doesn't -- it has some pointers/alignment, but that is all. I don't like saying "ADR" all over. I'm leaning toward exposing CRITICAL_SECTION but not CONTEXT. If/when Win32 and pthreads are merged though, this optimization will go away. More importantly: To initialize the event on demand or not.The thing is, under low resources, that can fail.And then both the caller in ThreadWin32.m3 and its caller need to deal with it.I think for simplicity, do more up front allocation i.e. in CreateT. I wish we could do more.That is, users say NEW(MUTEX) or NEW(Condition) and we don't get a chance to run InitializeCriticalSection at that point. We have to delay it until they call LOCK. This bothers me some. I know zero initialization is good, but I'd like some infrastructural way to trigger non-zero initialization for a heap allocated type w/o dpeending on the caller to have to say .init() or such. i.e. C++ constructors. I know zeros are good and efficient.And I know putting off initialization can be profitable, because it might never be done.But the other choice is nice too -- to aggressively do initialization earlier, without having to synchronize on-demand-ness. I'll keep poking at this and send a revision later...could still be days. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Aug 4 09:25:58 2016 From: jay.krell at cornell.edu (Jay K) Date: Thu, 04 Aug 2016 07:25:58 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: <579FA848.6010400@lcwb.coop> References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop> Message-ID: Right, it wasn't done. I have "staged" the maybe final version here: https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32.m3-2 There is a minor update then here, just removing some code. https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32C.c-2 Some notes: I removed some of the on-demand initialization. It can/could fail, so it has to be very careful, not to fail with partial state. I expose sizeof(CRITICAL_SECTION) into Modula-3. There is fairly portable, in that Win64 will work already. If we further merge with ThreadPThread.m3 though, this will become hidden again, at slight cost, and potential reintroduction of failure points (failure to heap allocate the CRITICAL_SECTION in LockMutex). ThreadPThread.m3 already is like this. I need to check if it is robust under low memory. Similarly I call CreateEvent early instead of on-demand. I based even more of it on ThreadPThread.m3. Perhaps unnecessarily so and with some efficiency loss. In particular, all thread joins are now serialized. And alerting is probably less efficient. The joining part could be better in ThreadPThread.m3 as well, I think, and then maybe follow suite here? Your test passes. I can rebuild the world, which I know exercises the code some, since I managed to break it while changing it. Thoughts? Commit? I believe this structuring is now quite amenable to sharing the vast bulk of the code with ThreadPThread.m3. We really just need to rename some things so they are an "abstraction layer" that directly maps to pthread and win32, instead of also reusing the pthread/win32 names. Specifically, pthread mutex, win32 critical_section, pthread cond, and Win32 event. The heap locking would need attention do. PThread and Win32 vary in structure there. Maybe stay that way, but still move most of the code to somewhere common. And I have to see if user threads can share code too. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 5 05:53:59 2016 From: jay.krell at cornell.edu (Jay K) Date: Fri, 05 Aug 2016 03:53:59 -0000 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop>, , , , , , , <579FA848.6010400@lcwb.coop>, Message-ID: This is commited now, with one slight change per Rodney -- checking in UnlockMutex for uninitialized (and therefore unlocked) mutex. - Jay From: jay.krell at cornell.edu To: rodney.m.bates at acm.org; m3devel at elegosoft.com Subject: RE: A win32 thread test case Date: Thu, 4 Aug 2016 07:25:54 +0000 Right, it wasn't done. I have "staged" the maybe final version here: https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32.m3-2 There is a minor update then here, just removing some code. https://github.com/modula3/cm3/blob/master/m3-libs/m3core/src/thread/WIN32/ThreadWin32C.c-2 Some notes: I removed some of the on-demand initialization. It can/could fail, so it has to be very careful, not to fail with partial state. I expose sizeof(CRITICAL_SECTION) into Modula-3. There is fairly portable, in that Win64 will work already. If we further merge with ThreadPThread.m3 though, this will become hidden again, at slight cost, and potential reintroduction of failure points (failure to heap allocate the CRITICAL_SECTION in LockMutex). ThreadPThread.m3 already is like this. I need to check if it is robust under low memory. Similarly I call CreateEvent early instead of on-demand. I based even more of it on ThreadPThread.m3. Perhaps unnecessarily so and with some efficiency loss. In particular, all thread joins are now serialized. And alerting is probably less efficient. The joining part could be better in ThreadPThread.m3 as well, I think, and then maybe follow suite here? Your test passes. I can rebuild the world, which I know exercises the code some, since I managed to break it while changing it. Thoughts? Commit? I believe this structuring is now quite amenable to sharing the vast bulk of the code with ThreadPThread.m3. We really just need to rename some things so they are an "abstraction layer" that directly maps to pthread and win32, instead of also reusing the pthread/win32 names. Specifically, pthread mutex, win32 critical_section, pthread cond, and Win32 event. The heap locking would need attention do. PThread and Win32 vary in structure there. Maybe stay that way, but still move most of the code to somewhere common. And I have to see if user threads can share code too. - Jay > Date: Mon, 1 Aug 2016 14:51:36 -0500 > From: rodney_bates at lcwb.coop > To: jay.krell at cornell.edu; rodney.m.bates at acm.org; m3devel at elegosoft.com > Subject: Re: A win32 thread test case > > It looks very reasonable, overall. The same PThread mechanism for managing/ > awakening waiters on Mutexs and Conditions should fix both the ticket count bugs > I mentioned. > > Some minor points: > > fields wait-generation_count, release_count, and waiters of PThreadLikeCondition_t > are unused (and unneeded -- leftovers from the flawed generation count scheme). > > Comments on the parameters of the CreateEvent call would help a reader, especially > the second, which importantly makes it an auto-reset event. > > I don't think you can assert m.initialized at the beginning of UnlockMutex. > ThreadPThread initializes it, if necessary. UnlockMutex can be called directly > by application code (well, almost directly, through a couple of wrappers), > and it could pass an initialized M3 Mutex in. > > I am having trouble following the connection mentioned in various comments among a lock, > a condition, wait, and signal. I am guessing these are a Win32 lock (a CRITICAL_SECTION), > a Win32 Event, Win32 WaitForSingleEvent, and Win32 SetEvent. The lock and condition appear > seem to be the lock and cond fields of an Activation? But while act.lock is held while > calling SetEvent, it is released before calling WaitForSingleObject. Does SetEvent require > it? With act.lock not passed to WaitForSingleObject, how could the identity of the lock > even be known to or affect SetEvent? > > I do see that there is an unstated LL=act.lock on entrance to Event_wait(act) and unstated > LL=t.lock when Event_Signal(t.cond) is called, and both seem needed locally, not by > Windows calls. > > > On 07/31/2016 03:45 PM, Jay K wrote: > > I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. > > It is attached. > > I also removed the indirection on critical sections -- exposing the size to Modula3. > > I went with just an event as I indicated. > > So the ThreadPThread.m3 code, but with slight changes: > > - waits specify time to wait until time to wait until > > - event instead of pthread_cond > > - same maintenance of list of waiters > > > > Reasonable? > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 10:37:30 +0000 > > > > a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. > > An even per thread should work. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 04:24:14 +0000 > > > > Something like attached? > > Still testing..might be crashing. > > A lot of the code and patterns come from ThreadPThread.m3. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 02:52:02 +0000 > > > > While I agree this is confusing, it is holding together for me and I have mostly coded it up. > > > > Repeating myself: > > In this scheme there are two types of condition variable. > > There are Modula-3 condition variables and posix-like/schmidt-like condition variables. > > > > The posix-like/smidt-like condition variables: > > Can be implemented directly via posix condition variables (if Windows pre-Vista had them). > > They are implemented as a slight modification of the Smidt counter method. > > The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible > > to hold the same lock as he holds when calling cond_wait. > > > > Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. > > I still have to work through your examples. > > > > Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. > > > > This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. > > XPause changes to call XWait, essentially. > > > > I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. > > > > If this works, we should be able to share more code between the pthread and win32 implementations. > > We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sun, 31 Jul 2016 00:47:24 +0000 > > > > ThreadPThread.m3 had the same narrow startup race condition. > > I fixed it. > > Again I'm not seeing commit mails. > > > > On the larger matter: > > > > I think there is a solution, in that, if you look at > > ThreadPThread.m3 and how it uses Posix condition variables, > > in that it does a fair amount of the work itself, > > you can term this "condition within condition". > > > > > > So I think maybe we can use the counting solution > > and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. > > > > > > But I have to look at it more. > > > > > > > > Specifically, in ThreadPThread.m3: > > > > Broadcast: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > next := t.nextWaiter; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > Signal: > > WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; > > t.nextWaiter := NIL; > > t.waitingOn := NIL; > > WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; > > > > > > Becaise they signal while locked, we can implement this limited > > form of condition variable, and then use the Modula-3 pthread logic > > using that. > > > > > > Make some sense? > > > > > > Essentially we will have almost-posix condition variables, > > and exactly-posix mutexes, and then we can use just the pthread code > > on top of that. > > > > > > Almost-posix condition variable is a posix condition variable, except > > it is guaranteed you will call pthread_cond_signal with the same > > lock held as when you call pthread_cond_wait. > > > > > > On a pthread system, the almost-posix condition variables and exactly-posix > > mutexes are just pass through to the underlying pthreads. > > > > > > On a win32 system, an almost-posix condition variable is similar > > to what we have now, except, because we know this lock is held, > > "XWait" can remove some of the leave/enter cycles and remove race conditions. > > > > > > Essentially we only need a "monitor", which is one lock and one condition variable. > > > > > > OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. > > > > - Jay > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! > --- > > From: jay.krell at cornell.edu > > To: rodney.m.bates at acm.org; m3devel at elegosoft.com > > Subject: RE: A win32 thread test case > > Date: Sat, 30 Jul 2016 21:18:08 +0000 > > > > I fixed the new_slots problem. > > > > > > I agree there are problems here. > > I think Java got away with a similar implementation > > because their interface is a bit different. > > i.e. the lock and condition are merged, and/or notification > > requires the caller to take the lock. We can't do this. > > We allow signal/broadcast w/o holding a lock. > > We'd have to resort to a global lock I guess. > > > > > > I suspect nobody else uses this solution. > > > > > > I believe we can solve it better and worse than others. > > > > > > Most libraries do not have their own "CreateThread" function. > > That is, most libraries let you call pthread_create or Win32 CreateThread, > > and the library will interoperate with any thread that comes its way. > > And most libraries don't seem to use thread locals in their solution. > > By "most libraries", I mean the pthreads on win32 package and Boost. > > > > > > Modula-3 isn't clearly this way. This isn't great, but it is the current > > situation. I believe it is fixable. > > That is, Modula-3 I believe requires using its library to create threads. > > This is not great for interoperation. You want to be able to be used > > by code that isn't away of you, that just creates threads in the "normal" way > > for their platform. > > > > > > And then, either the current state, or a fix I have in mind, can be taken > > advantage of to do "directed notify" w/o creating a kernel event > > per wait or notify, like other solutions do. > > > > > > As to the fix, to not require threads go through our "CreateThread", > > I believe on Windows (which is all that is relevant here), we should have > > a DllMain that allocates thread locals. > > > > > > We can only easily have a DllMain if we are a .dll. > > Or we can use __declspec(thread). > > __declspec(thread) works if the .exe is statically linked to the exe or the dll, > > or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. > > Perhaps that is good enough. > > > > > > If we require Vista then we can just drop a bunch of our code and use > > the Win32 condition variables presumably. > > I remain curious how the condition variables work there, i.e. we can't > > implement them ourselves, but it is possible they either require kernel > > support or are well beyond our abilities. > > > > > > > > - Jay > > > > > > > > > > > Date: Thu, 28 Jul 2016 15:00:45 -0500 > > > From: rodney_bates at lcwb.coop > > > To: m3devel at elegosoft.com; jay.krell at cornell.edu > > > Subject: A win32 thread test case > > > > > > I have attached a test case program designed to expose one of the bugs > > > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. > > > I expect it to fail on Windows, but don't have a Windows machine I > > > can try it on. > > > > > > Anybody willing to try it? It's a self-contained Main module. Just > > > compile and run it. It will announce what it finds. > > > > > > > > > -- > > > Rodney Bates > > > rodney.m.bates at acm.org > > -- > Rodney Bates > rodney.m.bates at acm.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 9 02:54:30 2016 From: lists at darko.org (Darko Volaric) Date: Tue, 09 Aug 2016 00:54:30 -0000 Subject: [M3devel] GitHub commit logging to M3 Commit mailing list gone? Message-ID: I'm no longer seeing commits from GitHub in the mailing list. Did someone change somethig? - Darko -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Thu Aug 25 19:58:47 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 25 Aug 2016 17:58:47 +0000 (UTC) Subject: [M3devel] Information lost, now recovered? References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 | ? | | ? | ? | ? | ? | ? | | HLRL: DEC Prism Project Documentation, Retrospective | HoffmanLabsFrom the HoffmanLabs Reading List (HLRL), some Digital Equipment Corporation (DEC) Prism project history has become available on BitSavers. | | | | Ver en labs.hoffmanlabs.com | Vista previa por Yahoo | | | | ? | Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From wagner at elegosoft.com Sat Aug 27 11:26:53 2016 From: wagner at elegosoft.com (Olaf Wagner) Date: Sat, 27 Aug 2016 11:26:53 +0200 Subject: [M3devel] Fw: Just an upate for modula3 Message-ID: <20160827112653.ac12dbed8a29349ab6f9cc34@elegosoft.com> FYI Begin forwarded message: Date: Fri, 26 Aug 2016 16:42:23 -0700 From: Brian Davis To: webmaster at modula3.org Subject: Just an upate for modula3 Modula3 Cm3 compiler including all 17 packages compile successfully on Fedora 24 kernel-4.6.7-300.fc24.x86_64 #1 SMP Everything works. :) -- Brian Davis brpia at fastmail.com -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Sun Aug 28 19:03:20 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Aug 2016 17:03:20 +0000 (UTC) Subject: [M3devel] Fw: Information lost, now recovered? In-Reply-To: <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> Message-ID: <378750309.1624269.1472403800154@mail.yahoo.com> On thursday August 25th, 2016 12:58, Daniel Alejandro Benavides D. wrote: Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sun Aug 28 19:32:53 2016 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 28 Aug 2016 17:32:53 +0000 (UTC) Subject: [M3devel] Fw2: Information lost, now recovered? In-Reply-To: <378750309.1624269.1472403800154@mail.yahoo.com> References: <998753846.3439608.1472147927151.JavaMail.yahoo.ref@mail.yahoo.com> <998753846.3439608.1472147927151.JavaMail.yahoo@mail.yahoo.com> <378750309.1624269.1472403800154@mail.yahoo.com> Message-ID: <623087746.1678529.1472405573085@mail.yahoo.com> Sorry if this repeatsOn Sunday August 28th, 2016 12:03, Daniel Alejandro Benavides D. wrote: On thursday August 25th, 2016 12:58, Daniel Alejandro Benavides D. wrote: Hello all:I was just searching about Digital microprocessors and found a language Called Pillar, the systems programming language of Mica OS of the Prism and VAX architectures.Has anyone checked it before or know what happened with it, specially after its cancellation? https://people.cs.clemson.edu/~mark/prism.htmlhttp://labs.hoffmanlabs.com/node/1877 Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Mon Aug 29 22:03:21 2016 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Aug 2016 13:03:21 -0700 Subject: [M3devel] A win32 thread test case In-Reply-To: References: <579A646D.3020404@lcwb.coop> Message-ID: <9D31FD75-2449-4255-8624-6A4D32807D9B@gmail.com> There was a lull in mails, but I haven't made this jump yet anyway. - Jay > On Aug 29, 2016, at 6:28 AM, "Hosking, Antony L" wrote: > > Jay, I've not seen these commits or accompanying emails. (Not sure why!) > Can you summarize what is going on here? > > Thanks, > > Tony > > Sent from my iPad > > On Aug 29, 2016, at 10:49 PM, Jay K wrote: > >> I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code. >> >> Win32 will then, for now, be one of the "Posix" platforms with "direct suspend". >> >> >> Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected). >> >> There are still a few steps to get there. >> >> One of them is naming. >> So far I'm thinking of names like: >> ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion >> ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event >> >> Another thin abstraction is needed around computing times/timeouts. >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sun, 31 Jul 2016 20:45:48 +0000 >> >> I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. >> It is attached. >> I also removed the indirection on critical sections -- exposing the size to Modula3. >> >> I went with just an event as I indicated. >> So the ThreadPThread.m3 code, but with slight changes: >> - waits specify time to wait until time to wait until >> - event instead of pthread_cond >> - same maintenance of list of waiters >> >> Reasonable? >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sun, 31 Jul 2016 10:37:30 +0000 >> >> a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. >> An even per thread should work. >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sun, 31 Jul 2016 04:24:14 +0000 >> >> Something like attached? >> Still testing..might be crashing. >> A lot of the code and patterns come from ThreadPThread.m3. >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sun, 31 Jul 2016 02:52:02 +0000 >> >> While I agree this is confusing, it is holding together for me and I have mostly coded it up. >> >> Repeating myself: >> In this scheme there are two types of condition variable. >> There are Modula-3 condition variables and posix-like/schmidt-like condition variables. >> >> The posix-like/smidt-like condition variables: >> Can be implemented directly via posix condition variables (if Windows pre-Vista had them). >> They are implemented as a slight modification of the Smidt counter method. >> The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible >> to hold the same lock as he holds when calling cond_wait. >> >> Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. >> I still have to work through your examples. >> >> Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. >> >> This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. >> XPause changes to call XWait, essentially. >> >> I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. >> >> If this works, we should be able to share more code between the pthread and win32 implementations. >> We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sun, 31 Jul 2016 00:47:24 +0000 >> >> ThreadPThread.m3 had the same narrow startup race condition. >> I fixed it. >> Again I'm not seeing commit mails. >> >> On the larger matter: >> >> I think there is a solution, in that, if you look at >> ThreadPThread.m3 and how it uses Posix condition variables, >> in that it does a fair amount of the work itself, >> you can term this "condition within condition". >> >> >> So I think maybe we can use the counting solution >> and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. >> >> >> But I have to look at it more. >> >> >> >> Specifically, in ThreadPThread.m3: >> >> Broadcast: >> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; >> next := t.nextWaiter; >> t.nextWaiter := NIL; >> t.waitingOn := NIL; >> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; >> >> Signal: >> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; >> t.nextWaiter := NIL; >> t.waitingOn := NIL; >> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; >> >> >> Becaise they signal while locked, we can implement this limited >> form of condition variable, and then use the Modula-3 pthread logic >> using that. >> >> >> Make some sense? >> >> >> Essentially we will have almost-posix condition variables, >> and exactly-posix mutexes, and then we can use just the pthread code >> on top of that. >> >> >> Almost-posix condition variable is a posix condition variable, except >> it is guaranteed you will call pthread_cond_signal with the same >> lock held as when you call pthread_cond_wait. >> >> >> On a pthread system, the almost-posix condition variables and exactly-posix >> mutexes are just pass through to the underlying pthreads. >> >> >> On a win32 system, an almost-posix condition variable is similar >> to what we have now, except, because we know this lock is held, >> "XWait" can remove some of the leave/enter cycles and remove race conditions. >> >> >> Essentially we only need a "monitor", which is one lock and one condition variable. >> >> >> OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >> Subject: RE: A win32 thread test case >> Date: Sat, 30 Jul 2016 21:18:08 +0000 >> >> I fixed the new_slots problem. >> >> >> I agree there are problems here. >> I think Java got away with a similar implementation >> because their interface is a bit different. >> i.e. the lock and condition are merged, and/or notification >> requires the caller to take the lock. We can't do this. >> We allow signal/broadcast w/o holding a lock. >> We'd have to resort to a global lock I guess. >> >> >> I suspect nobody else uses this solution. >> >> >> I believe we can solve it better and worse than others. >> >> >> Most libraries do not have their own "CreateThread" function. >> That is, most libraries let you call pthread_create or Win32 CreateThread, >> and the library will interoperate with any thread that comes its way. >> And most libraries don't seem to use thread locals in their solution. >> By "most libraries", I mean the pthreads on win32 package and Boost. >> >> >> Modula-3 isn't clearly this way. This isn't great, but it is the current >> situation. I believe it is fixable. >> That is, Modula-3 I believe requires using its library to create threads. >> This is not great for interoperation. You want to be able to be used >> by code that isn't away of you, that just creates threads in the "normal" way >> for their platform. >> >> >> And then, either the current state, or a fix I have in mind, can be taken >> advantage of to do "directed notify" w/o creating a kernel event >> per wait or notify, like other solutions do. >> >> >> As to the fix, to not require threads go through our "CreateThread", >> I believe on Windows (which is all that is relevant here), we should have >> a DllMain that allocates thread locals. >> >> >> We can only easily have a DllMain if we are a .dll. >> Or we can use __declspec(thread). >> __declspec(thread) works if the .exe is statically linked to the exe or the dll, >> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. >> Perhaps that is good enough. >> >> >> If we require Vista then we can just drop a bunch of our code and use >> the Win32 condition variables presumably. >> I remain curious how the condition variables work there, i.e. we can't >> implement them ourselves, but it is possible they either require kernel >> support or are well beyond our abilities. >> >> >> >> - Jay >> >> >> >> >> > Date: Thu, 28 Jul 2016 15:00:45 -0500 >> > From: rodney_bates at lcwb.coop >> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >> > Subject: A win32 thread test case >> > >> > I have attached a test case program designed to expose one of the bugs >> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. >> > I expect it to fail on Windows, but don't have a Windows machine I >> > can try it on. >> > >> > Anybody willing to try it? It's a self-contained Main module. Just >> > compile and run it. It will announce what it finds. >> > >> > >> > -- >> > Rodney Bates >> > rodney.m.bates at acm.org >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 30 03:22:49 2016 From: jay.krell at cornell.edu (Jay) Date: Mon, 29 Aug 2016 18:22:49 -0700 Subject: [M3devel] A win32 thread test case In-Reply-To: <9D31FD75-2449-4255-8624-6A4D32807D9B@gmail.com> References: <579A646D.3020404@lcwb.coop> <9D31FD75-2449-4255-8624-6A4D32807D9B@gmail.com> Message-ID: <557F1D56-BA5B-4EE6-A89A-A1CE11B9EE34@gmail.com> The summary though: my "recent" (years ago) implementation of condition variables et. al. was unconvincing, or more likely wrong. "Condition variables et. al." -- tied into threads and mutexes. I replaced it again. The replacement is based closely on the pthreads version. I noticed though that the pthread use of pthread condition variables does not require much, so win32 pre-vista mapped easily enough. And now with the strong resemblance, more code sharing should be possible, but hasn't been done yet. A thin abstraction layer will be needed. - Jay > On Aug 29, 2016, at 1:03 PM, Jay wrote: > > There was a lull in mails, but I haven't made this jump yet anyway. > > - Jay > >> On Aug 29, 2016, at 6:28 AM, "Hosking, Antony L" wrote: >> >> Jay, I've not seen these commits or accompanying emails. (Not sure why!) >> Can you summarize what is going on here? >> >> Thanks, >> >> Tony >> >> Sent from my iPad >> >> On Aug 29, 2016, at 10:49 PM, Jay K wrote: >> >>> I have high hopes that with this direction we will soon be able to fold ThreadPThread.m3 and ThreadWin32.m3 into one common code. >>> >>> Win32 will then, for now, be one of the "Posix" platforms with "direct suspend". >>> >>> >>> Later still, we'll have cooperative suspend -- goal being better portability and correctness on emulators that don't work great like PPC_DARWIN on x86/amd64 (the context APIs don't work) or wow64 (where the context APIs don't work quite as expected). >>> >>> There are still a few steps to get there. >>> >>> One of them is naming. >>> So far I'm thinking of names like: >>> ThreadInternalLock -- for pthread mutex and Win32 critical_section, taking the minimum functionality -- no recursion >>> ThreadInternalEvent -- for the use of thread pthread condition variables and Win32 event >>> >>> Another thin abstraction is needed around computing times/timeouts. >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sun, 31 Jul 2016 20:45:48 +0000 >>> >>> I confirmed your test case fails with unmodified ThreadWin32.m3 and passes with my current work in progress. >>> It is attached. >>> I also removed the indirection on critical sections -- exposing the size to Modula3. >>> >>> I went with just an event as I indicated. >>> So the ThreadPThread.m3 code, but with slight changes: >>> - waits specify time to wait until time to wait until >>> - event instead of pthread_cond >>> - same maintenance of list of waiters >>> >>> Reasonable? >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sun, 31 Jul 2016 10:37:30 +0000 >>> >>> a fallacy here is that..broadcast isn't used, only signal, so the implementation is more general than needed. >>> An even per thread should work. >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sun, 31 Jul 2016 04:24:14 +0000 >>> >>> Something like attached? >>> Still testing..might be crashing. >>> A lot of the code and patterns come from ThreadPThread.m3. >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sun, 31 Jul 2016 02:52:02 +0000 >>> >>> While I agree this is confusing, it is holding together for me and I have mostly coded it up. >>> >>> Repeating myself: >>> In this scheme there are two types of condition variable. >>> There are Modula-3 condition variables and posix-like/schmidt-like condition variables. >>> >>> The posix-like/smidt-like condition variables: >>> Can be implemented directly via posix condition variables (if Windows pre-Vista had them). >>> They are implemented as a slight modification of the Smidt counter method. >>> The differences are that condition.lock is removed. Caller of cond_signal/cond_broadcast is responsible >>> to hold the same lock as he holds when calling cond_wait. >>> >>> Some of the lock/unlock cycles are removed, and I hope therefore all the bugs are fixed. >>> I still have to work through your examples. >>> >>> Modula-3 condition variables are then built on top of pthread-like/schmidt condition variables just like ThreadPThread.m3. >>> >>> This includes "alertable". The separate event previously in the Win32 implementation becomes a boolean plus cond_signal. >>> XPause changes to call XWait, essentially. >>> >>> I was thinking we could reuse the Win32 thread alerting feature, but this alerts for other reasons so maybe isn't approprirate. >>> >>> If this works, we should be able to share more code between the pthread and win32 implementations. >>> We just need a name for this slightly constrained pthread interface, i.e. where the condition variables assume the lock is held by the caller of signal/broadcast (we don't use broadcast). >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sun, 31 Jul 2016 00:47:24 +0000 >>> >>> ThreadPThread.m3 had the same narrow startup race condition. >>> I fixed it. >>> Again I'm not seeing commit mails. >>> >>> On the larger matter: >>> >>> I think there is a solution, in that, if you look at >>> ThreadPThread.m3 and how it uses Posix condition variables, >>> in that it does a fair amount of the work itself, >>> you can term this "condition within condition". >>> >>> >>> So I think maybe we can use the counting solution >>> and 1) merge the locks and/or 2) take the lock in Signal/Broadcast. >>> >>> >>> But I have to look at it more. >>> >>> >>> >>> Specifically, in ThreadPThread.m3: >>> >>> Broadcast: >>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; >>> next := t.nextWaiter; >>> t.nextWaiter := NIL; >>> t.waitingOn := NIL; >>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; >>> >>> Signal: >>> WITH r = pthread_mutex_lock(t.mutex) DO <*ASSERT r=0*> END; >>> t.nextWaiter := NIL; >>> t.waitingOn := NIL; >>> WITH r = pthread_cond_signal(t.cond) DO <*ASSERT r=0*> END; >>> >>> >>> Becaise they signal while locked, we can implement this limited >>> form of condition variable, and then use the Modula-3 pthread logic >>> using that. >>> >>> >>> Make some sense? >>> >>> >>> Essentially we will have almost-posix condition variables, >>> and exactly-posix mutexes, and then we can use just the pthread code >>> on top of that. >>> >>> >>> Almost-posix condition variable is a posix condition variable, except >>> it is guaranteed you will call pthread_cond_signal with the same >>> lock held as when you call pthread_cond_wait. >>> >>> >>> On a pthread system, the almost-posix condition variables and exactly-posix >>> mutexes are just pass through to the underlying pthreads. >>> >>> >>> On a win32 system, an almost-posix condition variable is similar >>> to what we have now, except, because we know this lock is held, >>> "XWait" can remove some of the leave/enter cycles and remove race conditions. >>> >>> >>> Essentially we only need a "monitor", which is one lock and one condition variable. >>> >>> >>> OR, we can use "directed notification" where each thread has an event, and threads are linked through a "wait block" i.e. the condition. This should work well too. It depends on us owing CreateThread, which we already do. >>> >>> - Jay >>> >>> >>> From: jay.krell at cornell.edu >>> To: rodney.m.bates at acm.org; m3devel at elegosoft.com >>> Subject: RE: A win32 thread test case >>> Date: Sat, 30 Jul 2016 21:18:08 +0000 >>> >>> I fixed the new_slots problem. >>> >>> >>> I agree there are problems here. >>> I think Java got away with a similar implementation >>> because their interface is a bit different. >>> i.e. the lock and condition are merged, and/or notification >>> requires the caller to take the lock. We can't do this. >>> We allow signal/broadcast w/o holding a lock. >>> We'd have to resort to a global lock I guess. >>> >>> >>> I suspect nobody else uses this solution. >>> >>> >>> I believe we can solve it better and worse than others. >>> >>> >>> Most libraries do not have their own "CreateThread" function. >>> That is, most libraries let you call pthread_create or Win32 CreateThread, >>> and the library will interoperate with any thread that comes its way. >>> And most libraries don't seem to use thread locals in their solution. >>> By "most libraries", I mean the pthreads on win32 package and Boost. >>> >>> >>> Modula-3 isn't clearly this way. This isn't great, but it is the current >>> situation. I believe it is fixable. >>> That is, Modula-3 I believe requires using its library to create threads. >>> This is not great for interoperation. You want to be able to be used >>> by code that isn't away of you, that just creates threads in the "normal" way >>> for their platform. >>> >>> >>> And then, either the current state, or a fix I have in mind, can be taken >>> advantage of to do "directed notify" w/o creating a kernel event >>> per wait or notify, like other solutions do. >>> >>> >>> As to the fix, to not require threads go through our "CreateThread", >>> I believe on Windows (which is all that is relevant here), we should have >>> a DllMain that allocates thread locals. >>> >>> >>> We can only easily have a DllMain if we are a .dll. >>> Or we can use __declspec(thread). >>> __declspec(thread) works if the .exe is statically linked to the exe or the dll, >>> or on Vista and newer, but not in a .dll loaded by LoadLibrary prior to Vista. >>> Perhaps that is good enough. >>> >>> >>> If we require Vista then we can just drop a bunch of our code and use >>> the Win32 condition variables presumably. >>> I remain curious how the condition variables work there, i.e. we can't >>> implement them ourselves, but it is possible they either require kernel >>> support or are well beyond our abilities. >>> >>> >>> >>> - Jay >>> >>> >>> >>> >>> > Date: Thu, 28 Jul 2016 15:00:45 -0500 >>> > From: rodney_bates at lcwb.coop >>> > To: m3devel at elegosoft.com; jay.krell at cornell.edu >>> > Subject: A win32 thread test case >>> > >>> > I have attached a test case program designed to expose one of the bugs >>> > I think I see in ThreadWin32.m3. It runs correctly on AMD64_LINUX. >>> > I expect it to fail on Windows, but don't have a Windows machine I >>> > can try it on. >>> > >>> > Anybody willing to try it? It's a self-contained Main module. Just >>> > compile and run it. It will announce what it finds. >>> > >>> > >>> > -- >>> > Rodney Bates >>> > rodney.m.bates at acm.org >>> _______________________________________________ >>> M3devel mailing list >>> M3devel at elegosoft.com >>> https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at darko.org Tue Aug 30 06:10:29 2016 From: lists at darko.org (Darko Volaric) Date: Tue, 30 Aug 2016 06:10:29 +0200 Subject: [M3devel] GitHub commit logging to M3 Commit mailing list gone? In-Reply-To: <5B41F1E2-A015-4AA6-979F-621433E1F60D@purdue.edu> References: <5B41F1E2-A015-4AA6-979F-621433E1F60D@purdue.edu> Message-ID: It came back all at once yesterday with a backlog of posts, along with the devel list. I actually sent that mail 3 weeks ago! On Mon, Aug 29, 2016 at 3:29 PM, Hosking, Antony L wrote: > Me too! > > Sent from my iPad > > > On Aug 29, 2016, at 10:09 PM, Darko Volaric wrote: > > > > I'm no longer seeing commits from GitHub in the mailing list. Did > someone change somethig? > > > > - Darko > > _______________________________________________ > > M3devel mailing list > > M3devel at elegosoft.com > > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Aug 30 21:05:22 2016 From: jay.krell at cornell.edu (Jay) Date: Tue, 30 Aug 2016 12:05:22 -0700 Subject: [M3devel] why thread slots? In-Reply-To: <3420B2A3-3569-468F-9F6F-1712F62B8481@purdue.edu> References: <3420B2A3-3569-468F-9F6F-1712F62B8481@purdue.edu> Message-ID: <54E41B6C-69E7-4D9E-9F01-E7FCC49BEAC2@gmail.com> "Garbage collector can't see thread locals." Understood. - Jay > On Aug 30, 2016, at 4:51 AM, "Hosking, Antony L" wrote: > > You need gc roots for the collector! > > Sent from my iPad > > On Aug 30, 2016, at 6:36 PM, Jay K wrote: > >> What is the point of all this "slot" stuff in the thread library? >> >> It seems kind of self-referential and pointless. >> >> I might try removing it and see what the results are. >> >> - Jay >> _______________________________________________ >> M3devel mailing list >> M3devel at elegosoft.com >> https://m3lists.elegosoft.com/mailman/listinfo/m3devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Wed Aug 31 19:45:24 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 31 Aug 2016 12:45:24 -0500 Subject: [M3devel] idioms for tracking initialization state and raising errors? In-Reply-To: References: Message-ID: <57C717B4.1040108@lcwb.coop> On 07/31/2016 04:07 AM, Jay K wrote: > Is this a correct idiom? > If so, I think it is fairly reasonable. > It looks reasonable to me, however ... > > That is: > be very willing to "early return" > put all cleanup in finally > raise errors in finally > > > In particular, I don't want to repeat the cleanup. You might want to reconsider this. The TRY-FINALLY construct has some significant behind-the-scenes overhead, and this could be a somewhat speed-critical operation. The compiler has to create a procedure containing the code in the FINALLY block, put calls on it in all the places where it is statically known it needs to execute (e.g., RETURNs and falling out of the main TRY block), and further register it with the runtime system, so, for the statically unpredictable cases where an exception comes out of some explicitly coded call in the main block, the RTS can do a callback on it, before propagating the exception out another level. It may be less elegant, but it could be a good bit faster to just duplicate cleanup code at the 3 relevant places, i.e., the two RETURNs and falling out of the main block and eliminate the TRY-FINALLY overhead. Moreover, in each one of these places, the cleanup code can be specialized, eliminating both IF tests and omitting some of the cleanup actions. For example, at the second RETURN inside the TRY block, it is certain that lock#NIL, so the first LeaveCriticalSection can be unconditional, so the 1st cleanup statement can be eliminated altogether, and that mutex.initialized, so the 3rd cleanup statement can be eliminated altogether. It remains to consider exceptions raised from inside the TRY main block, and while it looks doubtful to me that these can matter or even happen, I think more thought is warranted. Since NewCriticalSection is a Windows call, it probably can't raise a Modula3 exception. Or can it? Could it segfault and have that turn into a M3 exception? If that happened, is there any point in any cleanup, or is it guaranteed to terminate the program? RegisterFinalCleanup has an empty RAISES set. But with all this unsafe code, is that enough to rely on? Presumably it could segfault (I see it directly contains some array subscripting), but would cleanup matter in that case either? > I don't want to raise before leaving critical sections. > > > I don't intend this to raise within a raise, but only > from the "normal" exit of the finally block. > > > I also don't want extra local booleans, i.e. raise: boolean to indicate failure. > Just the one to indicate success. > > > PROCEDURE InitMutex (mutex: Mutex) = > VAR > lock: PCRITICAL_SECTION := NIL; > locked: PCRITICAL_SECTION := NIL; > > BEGIN > IF mutex.initialized THEN RETURN END; > > TRY > > lock := NewCriticalSection(); > IF lock = NIL THEN RETURN; END; > > EnterCriticalSection(ADR(initLock)); > locked := ADR(initLock); > > IF mutex.initialized THEN RETURN END; > > (* We won the race. *) > RTHeapRep.RegisterFinalCleanup (mutex, CleanMutex); > mutex.lock := lock; > lock := NIL; > mutex.initialized := TRUE; > > FINALLY > IF locked # NIL THEN LeaveCriticalSection(locked); END; > DelCriticalSection(lock); > IF NOT mutex.initialized THEN (* Raise after leaving critical section. *) > RuntimeError.Raise (RuntimeError.T.OutOfMemory); > END; > END; > > END InitMutex; > > > Thank you, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org From rodney_bates at lcwb.coop Wed Aug 31 19:58:37 2016 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Wed, 31 Aug 2016 12:58:37 -0500 Subject: [M3devel] idioms for tracking initialization state and raising errors? In-Reply-To: References: Message-ID: <57C71ACD.1000202@lcwb.coop> On 07/31/2016 04:07 AM, Jay K wrote: > Is this a correct idiom? > If so, I think it is fairly reasonable. > > > That is: > be very willing to "early return" > put all cleanup in finally > raise errors in finally > > > In particular, I don't want to repeat the cleanup. > I don't want to raise before leaving critical sections. > > > I don't intend this to raise within a raise, but only > from the "normal" exit of the finally block. > > > I also don't want extra local booleans, i.e. raise: boolean to indicate failure. > Just the one to indicate success. > > > PROCEDURE InitMutex (mutex: Mutex) = > VAR > lock: PCRITICAL_SECTION := NIL; > locked: PCRITICAL_SECTION := NIL; > > BEGIN > IF mutex.initialized THEN RETURN END; > > TRY > > lock := NewCriticalSection(); > IF lock = NIL THEN RETURN; END; > ^I suggest moving the two statements above inside the critical section on initLock, after the inner check on mutex.initialized. That way, if we lose a race, it avoids executing creation then deletion of an unneeded CriticalSection, and further eliminates the cleanup action in the code. BTW, I am assuming initLock is a single global CriticalSection? > EnterCriticalSection(ADR(initLock)); > locked := ADR(initLock); > > IF mutex.initialized THEN RETURN END; > > (* We won the race. *) > RTHeapRep.RegisterFinalCleanup (mutex, CleanMutex); > mutex.lock := lock; > lock := NIL; > mutex.initialized := TRUE; > > FINALLY > IF locked # NIL THEN LeaveCriticalSection(locked); END; > DelCriticalSection(lock); > IF NOT mutex.initialized THEN (* Raise after leaving critical section. *) > RuntimeError.Raise (RuntimeError.T.OutOfMemory); > END; > END; > > END InitMutex; > > > Thank you, > - Jay > > > _______________________________________________ > M3devel mailing list > M3devel at elegosoft.com > https://m3lists.elegosoft.com/mailman/listinfo/m3devel > -- Rodney Bates rodney.m.bates at acm.org