From jkrell at elego.de Fri Aug 1 08:00:22 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:00:22 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060024.DD2DB10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:00:22 Modified files: cm3/scripts/python/: pylib.py Log message: use another popular (at least with me) guess for where the cm3 executable is -- /cm3/bin/cm3 From jkrell at elego.de Fri Aug 1 08:02:31 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:02:31 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060232.3DFDA10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:02:31 Modified files: cm3/scripts/python/: pylib.py Log message: also the D: drive, probably should check SystemDrive environment variable and iterate a..z or c..z From jkrell at elego.de Fri Aug 1 11:46:18 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 11:46:18 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801094618.94B8710D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 11:46:18 Modified files: cm3/m3-ui/ui/src/winvbt/: WinScrnColorMap.m3 Log message: fix warning From jkrell at elego.de Fri Aug 1 15:17:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:17:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801131751.6158010D4DD2@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:17:51 Modified files: cm3/m3-libs/m3core/src/win32/: WinGDI.i3 WinGDI.m3 WinUser.i3 WinUser.m3 Log message: some header updates and repair not all of it compatible -- identifiers not in headers generally removed "max" identifiers that aren't constants, removed The Modula-3 headers appear to be from VERY old Windows headers, possibly NT 3.1 beta since the level of stuff the Modula-3 headers make up that isn't in headers is surprising. only added result of findstr #define on winuser.h did not check if things #if'ed out did not add new functions this triggered by alarm over unrecognized output of windows message tracing, which was merely a message that didn't exist long ago when this stuff was written ie: pre NT 4 -- by a lot I think. always normalize some booleans to true/false the extra check of is_chicago is not worthwhile not worth the cost to initialize and use the global though Modula-3 ought to define boolean as a full 32 bit integer that is 0 or non-zero like C... probably should just do this for all extern functions????? it is not really probably a bug for C to return other values for true note that Modula-3 doesn't have the "#if WINVER >= 0x4000" stuff. everything is unconditionally declared m3core.dll et. al. should probably take care to GetProcAddress anything they use that was added after some agreed upon lowest common denominator all this header duplication could really use some automation/verification none introduced here today.. From jkrell at elego.de Fri Aug 1 15:43:44 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:43:44 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134344.5D47E10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:43:44 Modified files: cm3/m3-libs/m3core/src/win32/: WinUser.i3 Log message: a few more updates, still quite out of date but arguably that is just fine -- only declare what the m3 libs need From jkrell at elego.de Fri Aug 1 15:46:50 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:46:50 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134650.8A59C10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:46:50 Modified files: cm3/m3-ui/ui/src/winvbt/: WinMsg.m3 cm3/m3-ui/anim3D/src/win-opengl/: Win_OpenGL_Base.m3 Log message: trace a little more, sure could use some code consolidation here.. From jkrell at elego.de Sat Aug 2 03:36:36 2008 From: jkrell at elego.de (Jay Krell) Date: Sat, 2 Aug 2008 3:36:36 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080802013636.7D26C10D4DD9@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/02 03:36:36 Modified files: cm3/m3-ui/vbtkit/src/vbtkitutils/: VBTKitEnv.m3 Log message: allow getting the emacs text editing model via the environment variable From wagner at elego.de Sun Aug 3 11:44:54 2008 From: wagner at elego.de (Olaf Wagner) Date: Sun, 3 Aug 2008 11:44:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080803094454.764EA10D4E29@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/03 11:44:54 Modified files: cm3/m3-ui/vbtkit/src/: m3overrides cm3/m3-ui/vbtkit/src/etext/: MacModel.m3 TextPort.m3 Log message: tentative fix for text selection and insertion in the Mac TextPortModel: The strange behaviour reported by Randy Coleburn (mouse click, left, left, insert --> first character inserted at mouse click point) should now be gone; cursor up and down at the right end of ragged paragraphs still looks a bit strange, but that seems to be not worse than before. It still should be fixed though. These changes should be reviewed and tested by others; they seem to do the right thing for me, but I haven't done extensive testing, especially in other modes. From jkrell at elego.de Tue Aug 5 12:47:56 2008 From: jkrell at elego.de (Jay Krell) Date: Tue, 5 Aug 2008 12:47:56 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080805104756.C117410D4E5D@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/05 12:47:56 Modified files: cm3/m3-sys/m3cc/src/: m3makefile Log message: dependency on libiconv.so still present, use the previous code From jkrell at elego.de Fri Aug 8 15:14:40 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:14:40 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808131440.33CCF10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:14:40 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common Log message: fix this: LINK : warning LNK4044: unrecognized option '/cm3\lib\hand.obj'; ignored From jkrell at elego.de Fri Aug 8 15:29:48 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:29:48 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808132948.C7F0F10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:29:48 Modified files: cm3/m3-libs/m3core/src/win32/: WinDef.i3 WinNT.i3 WinUser.i3 WinUser.m3 m3makefile Added files: cm3/m3-libs/m3core/src/win32/: WinNTc.c Log message: some header file updates WinNT* in particular is used by the upcoming pixmap change (I'd say "fix" but I can't test it, no high dpi system available, nor perhaps apps with noticable deformity) From jkrell at elego.de Fri Aug 8 15:34:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:34:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133451.B6E0810D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:34:51 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Added files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: make all uses of Image.Raw.xres and yres go through accessor functions in fact, renaming xres/yres so any failure to catch uses will fail to compile On Windows, compute xres and yres from Devicecaps(desktop window) and GetSystemMetrics on X Windows, keep the same old 75 (but this looks not difficult to fix) this has a small dependency on WinNT* change from a minute ago From jkrell at elego.de Fri Aug 8 15:38:08 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:38:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133808.8766310D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:38:08 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: remove unused import (I was going to use InterlockedIncrement, but that is perhaps overkill, and unsafe apparently) From jkrell at elego.de Fri Aug 8 15:42:29 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:42:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134229.7A65510D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:42:29 Modified files: cm3/m3-libs/m3core/src/win32/: m3makefile Log message: include WinNTc.c in all platforms, I expect it is needed From jkrell at elego.de Fri Aug 8 15:45:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:45:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134534.4E2C410D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:45:33 Modified files: cm3/m3-libs/m3core/src/win32/: WinNT.i3 Log message: small Win64 fix (not relevant to anything at this time) From jkrell at elego.de Fri Aug 8 15:46:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:46:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134633.7700C10D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:46:33 Added files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 Log message: forgot to add file From rcoleburn at elego.de Sat Aug 9 07:09:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 7:09:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809050943.8325B10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 07:09:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Compute scroll bar slider width based on screen resolution rather than making it a fixed size in millimeters. Allows scroll bar to adapt to different screen resolutions. Used new procedures in ImageInit provided by Jay Krell.---Randy Coleburn From rcoleburn at elego.de Sat Aug 9 10:02:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:02:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809080243.B029A10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:02:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Ensure millimeters are whole, not fractional.---R.Coleburn From rcoleburn at elego.de Sat Aug 9 10:50:13 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:50:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809085014.538C510D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:50:13 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Revert to prior version. My idea about truncating the millimeters was wrong, even though I thought it tested out ok. Sorry. --R.Coleburn From wagner at elego.de Sat Aug 9 11:03:13 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:03:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090313.CB2E410D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:03:13 Modified files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 Log message: make it compile (fix syntax error) From wagner at elego.de Sat Aug 9 11:08:24 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:08:24 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090824.BD1BE10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:08:24 Modified files: cm3/m3-sys/cm3ide/src/: m3overrides Log message: fix overrides (adapt to cm3 build conventions) From jkrell at elego.de Mon Aug 11 12:43:29 2008 From: jkrell at elego.de (Jay Krell) Date: Mon, 11 Aug 2008 12:43:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080811104330.09F1F10D4EF4@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/11 12:43:29 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: When I am copy/pasting code I don't really understand, it helps to look for more nearby code that should also probably be copied. Specifically: Call ReleaseDC after GetDC. From jkrell at elego.de Wed Aug 13 11:41:07 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 11:41:07 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813094107.37C8A10D4DFA@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 11:41:07 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Removed files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: go back one version -- back to hard coding Image xres/yres = 75 and leaving them publically accessible From jkrell at elego.de Wed Aug 13 13:10:59 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 13:10:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813111059.33D8010D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 13:10:59 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: fix it to compile, at least and eliminate common subexpressions will see if we can get Windows to draw much more of this using DrawFrameControl From jkrell at elego.de Thu Aug 14 16:26:59 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:26:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814142659.91A4A10D4F35@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From jkrell at elego.de Thu Aug 14 16:54:08 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:54:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814145408.C970510D4D27@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:54:08 Modified files: cm3/scripts/: find-packages.sh pkginfo.txt cm3/scripts/python/: make-dist.py pylib.py cm3/scripts/win/: find-packages.cmd Log message: add webcat, deskscape, webscape they might not be particularly useful, but they do build and startup From jkrell at elego.de Thu Aug 14 17:05:54 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 17:05:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814150554.AE35010D4F2A@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 17:05:54 Modified files: cm3/scripts/: find-packages.sh cm3/scripts/win/: find-packages.cmd Log message: go back a version unless proven to be needed From rcoleburn at scires.com Thu Aug 14 17:13:25 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 11:13:25 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <20080814142659.91A4A10D4F35@birch.elegosoft.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> Message-ID: <48A4134F.1E75.00D7.1@scires.com> Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:05:13 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A4134F.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:08:40 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:08:40 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From rcoleburn at scires.com Fri Aug 15 01:46:38 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 19:46:38 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A48B97.1E75.00D7.1@scires.com> Jay, this comment is indeed disturbing. Since, I didn't have anything to do with the Trestle implementation for Windows, I can't really offer an authoritative opinion here. The phrase "the only reason this *might* work" is especially disturbing as it sounds as if the idea was still being tested and not known to be sound. I've done quite a bit of multi-threaded programming and the complexity gets out of hand quickly without some rigor and the help of a good thread-aware language like Modula-3. When I say rigor, I mean the programmers have to set some ground rules that are enforced throughout the code that will ensure correct multi-threaded operation. Arbitrarily releasing locks to prevent potential recursive locks seems like a recipe for disaster and hints that the necessary rigor was not established. Regards, Randy >>> Jay 8/14/2008 6:05 PM >>> How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Fri Aug 15 02:11:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 20:11:19 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A49161.1E75.00D7.1@scires.com> Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy >>> Jay 8/14/2008 6:08 PM >>> Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 03:08:23 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 01:08:23 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A49161.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: We should probably add a new type Thread.RecursiveExclusiveLock? And then just use that here? The basic Win32 critical section is such a thing precisely. Not that it is all that great. But it is widely used. Vista adds the "SRWLock" which is a small read/write lock and maybe reduced recursive functionality. It is indeed small, just the size of a pointer, whereas a critical section is fairly large. - Jay Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy>>> Jay 8/14/2008 6:08 PM >>>Maybe we should have another lock type that allows recursive acquires?To be used sparingly, but used here?I know that is often frowned upon -- the need for it implies a design flaw -- butmaybe it is ok sometimes?I don't understand when/why recursive locks are ok or not.- Jay________________________________From: jay.krell at cornell.eduTo: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.comDate: Thu, 14 Aug 2008 22:05:13 +0000Subject: Re: [M3commit] CVS Update: cm3How about this comment in the code:(*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*)?I also need to check if any state has been modified, or cached in locals, withthe locks held that are being released.-Jay________________________________Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3Jay:I haven't studied this code in depth, but on the surface I doubt your change is "correct".I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released.Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon.I would suggest reverting these changes and looking to implement the Release procedure.Regards,Randy>>> Jay Krell 8/14/2008 4:26 PM>>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 12:41:23 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 11:41:23 +0100 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> I am unclear as to why this abstraction is needed for Windows threads when it is not needed for pthreads. Please explain. On Aug 15, 2008, at 2:08 AM, Jay wrote: > > We should probably add a new type Thread.RecursiveExclusiveLock? And > then just use that here? > The basic Win32 critical section is such a thing precisely. Not that > it is all that great. But it is widely used. > Vista adds the "SRWLock" which is a small read/write lock and maybe > reduced recursive functionality. > It is indeed small, just the size of a pointer, whereas a critical > section is fairly large. > > - Jay > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > Update: cm3 > > Jay: > > There is the abstraction of multiple concurrent "readers" but only > one "writer". I actually have such a module that I use sometimes in > my programs. The idea is one of acquiring/releasing read locks and > write locks. The difficulty here is in preventing starvation and > usually requires adherence to some rules (again rigor). By > starvation I mean that writers must wait for all readers to finish > before they get access, so if there is always at least one reader, > the writer will "starve" and never gain access. > > As for recursive locks, again that is usually indicative of a design > flaw. I did once implement a system that permitted the same thread > to acquire a lock multiple times, provided that it eventually > released the lock the same number of times that it acquired it. In > other words, after the first lock, subsequent locks by the same > thread just increment a counter. Then, unlocks decrement the > counter until it is zero with the final unlock actually releasing > the mutex. > > In the code you modified, you could implement such a scheme to > handle the recursive locks, provided that at some point the thread > releases for each lock. The Thread interface has the ability to > note the current thread (i.e., Thread.Self()). So you could make a > stack of locks by same thread. Other threads would have to block > until the first thread's lock stack was empty. > > For example: > > ACQUIRE: If resource available, lock resource mutex and put current > thread on granted stack. Otherwise, if current thread already on > granted stack, push it on stack again and let it proceed. Otherwise > (this is a new thread wanting access), so push this thread onto a > waiting stack and wait From dragisha at m3w.org Fri Aug 15 12:57:36 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 15 Aug 2008 12:57:36 +0200 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <1218797856.6212.4.camel@faramir.m3w.org> Just because we can do something is not a reason to do it. Minimalistic philosophy of Modula-3 API is, as opposed to Java for example, one of it's greatest strenghts. I still can remember a moment when I opened Java book on Thread class. I see it in my nightmares from time to time :). On Fri, 2008-08-15 at 11:41 +0100, Tony Hosking wrote: > I am unclear as to why this abstraction is needed for Windows threads > when it is not needed for pthreads. Please explain. > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? And > > then just use that here? > > The basic Win32 critical section is such a thing precisely. Not that > > it is all that great. But it is widely used. > > Vista adds the "SRWLock" which is a small read/write lock and maybe > > reduced recursive functionality. > > It is indeed small, just the size of a pointer, whereas a critical > > section is fairly large. > > > > - Jay > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > > Update: cm3 > > > > Jay: > > > > There is the abstraction of multiple concurrent "readers" but only > > one "writer". I actually have such a module that I use sometimes in > > my programs. The idea is one of acquiring/releasing read locks and > > write locks. The difficulty here is in preventing starvation and > > usually requires adherence to some rules (again rigor). By > > starvation I mean that writers must wait for all readers to finish > > before they get access, so if there is always at least one reader, > > the writer will "starve" and never gain access. > > > > As for recursive locks, again that is usually indicative of a design > > flaw. I did once implement a system that permitted the same thread > > to acquire a lock multiple times, provided that it eventually > > released the lock the same number of times that it acquired it. In > > other words, after the first lock, subsequent locks by the same > > thread just increment a counter. Then, unlocks decrement the > > counter until it is zero with the final unlock actually releasing > > the mutex. > > > > In the code you modified, you could implement such a scheme to > > handle the recursive locks, provided that at some point the thread > > releases for each lock. The Thread interface has the ability to > > note the current thread (i.e., Thread.Self()). So you could make a > > stack of locks by same thread. Other threads would have to block > > until the first thread's lock stack was empty. > > > > For example: > > > > ACQUIRE: If resource available, lock resource mutex and put current > > thread on granted stack. Otherwise, if current thread already on > > granted stack, push it on stack again and let it proceed. Otherwise > > (this is a new thread wanting access), so push this thread onto a > > waiting stack and wait > -- Dragi?a Duri? From jay.krell at cornell.edu Fri Aug 15 13:29:29 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 11:29:29 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree. pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 13:51:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 12:51:54 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Umm, no, we do not need this for pthreads, since the windowing on non- Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 21:37:46 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 19:37:46 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:03:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:03:41 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 22:12:25 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 20:12:25 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: My understanding, could be all wrong: The thread is not intended to hold no locks, just not while calling out. The thread obeys the usual Trestle locking rules. I *believe* it is not a special thread, it just happens to be the thread that all gui stuff happens on. I believe other threads can still do some Trestle stuff, just not that result to calls to Windows. It's obvious why this is happening, from the callstack. Less obvious what to do about it. Is reentrance and working on some data while already working on that data on the same thread really always such a terrible thing? If you know where the points you might reenter from are? That is, while data is locked, it can and will likely go through some states in which it is "incoherent", in which it is "in the middle" of being updated. But isn't the incoherence likely to be over short runs of code? Not likely to be at a point where a function is called? Otherwise, it becomes difficult for even the holder of the lock to keep things straight. I mean, you know, data is usually coherent, but must often pass through incoherence between coherent states. I would expect very little code to deal with the incoherence, for the updating code to be as short as possible. - Jay CC: m3commit at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3commit] recursive locking in WinTrestleDate: Fri, 15 Aug 2008 21:03:41 +0100 Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:28:09 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:28:09 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: <32788D5B-45A7-4D6B-BBDE-758DE5F1C92A@cs.purdue.edu> That thread is supposed to process an event queue, taking events from the Trestle threads, calling out to Windows on their behalf, and sending the result back to the waiting Trestle thread. What lock is it holding? Re-entrance is not a Modula-3 concept, for good reason. I think you may be oversimplifying the need for re-entrance. On Aug 15, 2008, at 9:12 PM, Jay wrote: > My understanding, could be all wrong: > The thread is not intended to hold no locks, just not while calling > out. > The thread obeys the usual Trestle locking rules. > I *believe* it is not a special thread, it just happens to be the > thread that all gui stuff happens on. I believe other threads can > still do some Trestle stuff, just not that result to calls to Windows. > It's obvious why this is happening, from the callstack. > Less obvious what to do about it. > > Is reentrance and working on some data while already working on that > data on the same thread really always such a terrible thing? If you > know where the points you might reenter from are? > That is, while data is locked, it can and will likely go through > some states in which it is "incoherent", in which it is "in the > middle" of being updated. But isn't the incoherence likely to be > over short runs of code? Not likely to be at a point where a > function is called? Otherwise, it becomes difficult for even the > holder of the lock to keep things straight. I mean, you know, data > is usually coherent, but must often pass through incoherence between > coherent states. I would expect very little code to deal with the > incoherence, for the updating code to be as short as possible. > > - Jay > > > CC: m3commit at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3commit] recursive locking in WinTrestle > Date: Fri, 15 Aug 2008 21:03:41 +0100 > > > Then the question is why that thread, which is intended *not* to > hold any locks so it *can* call out to Windows, actually does hold a > lock. Sounds like a bug to me. The fix is to figure out where it > is acquiring the lock and deduce what should actually be happening. > > On Aug 15, 2008, at 8:37 PM, Jay wrote: > > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at elego.de Fri Aug 22 16:06:53 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:06:53 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822140654.29EBF10D4FE6@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:06:53 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 Log message: Add comment about Raw.x/yres referring to the resolution at which the original image was constructed. Add procedure ForceUnScaledBehavior(). ---Randy Coleburn From rcoleburn at elego.de Fri Aug 22 16:32:47 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:32:47 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822143247.E873510D4E08@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:32:47 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: The following changes greatly improve appearance and functionality of scroll bars on Windows platforms. Internally, deal with all sizes in pixels, vs. millimeters. Provide conversions between millimeters and pixels as needed. Since FormsVBT scales pixmaps by default, scale the slider and trough to match the size of the button pixmaps. Change scroll behavior to more closely resemble user expectation in Windows. Formerly, scrolling was proportional to distance between mouse click and thumb. Now, clicking up/down buttons scrolls backward/forward by one line, and clicking in the slider trough above/below the thumb scrolls backward/forward by one window at a time. Unfortunately, the interface comments still reference the proportional behavior, but that behavior is still done in the POSIX implementation, so the interface comments are correct for POSIX. Perhaps we should add a comment about the Windows implementation; after all, the interface does say the implementation is free to change the behavior.---Randy Coleburn From jkrell at elego.de Fri Aug 1 08:00:22 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:00:22 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060024.DD2DB10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:00:22 Modified files: cm3/scripts/python/: pylib.py Log message: use another popular (at least with me) guess for where the cm3 executable is -- /cm3/bin/cm3 From jkrell at elego.de Fri Aug 1 08:02:31 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:02:31 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060232.3DFDA10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:02:31 Modified files: cm3/scripts/python/: pylib.py Log message: also the D: drive, probably should check SystemDrive environment variable and iterate a..z or c..z From jkrell at elego.de Fri Aug 1 11:46:18 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 11:46:18 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801094618.94B8710D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 11:46:18 Modified files: cm3/m3-ui/ui/src/winvbt/: WinScrnColorMap.m3 Log message: fix warning From jkrell at elego.de Fri Aug 1 15:17:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:17:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801131751.6158010D4DD2@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:17:51 Modified files: cm3/m3-libs/m3core/src/win32/: WinGDI.i3 WinGDI.m3 WinUser.i3 WinUser.m3 Log message: some header updates and repair not all of it compatible -- identifiers not in headers generally removed "max" identifiers that aren't constants, removed The Modula-3 headers appear to be from VERY old Windows headers, possibly NT 3.1 beta since the level of stuff the Modula-3 headers make up that isn't in headers is surprising. only added result of findstr #define on winuser.h did not check if things #if'ed out did not add new functions this triggered by alarm over unrecognized output of windows message tracing, which was merely a message that didn't exist long ago when this stuff was written ie: pre NT 4 -- by a lot I think. always normalize some booleans to true/false the extra check of is_chicago is not worthwhile not worth the cost to initialize and use the global though Modula-3 ought to define boolean as a full 32 bit integer that is 0 or non-zero like C... probably should just do this for all extern functions????? it is not really probably a bug for C to return other values for true note that Modula-3 doesn't have the "#if WINVER >= 0x4000" stuff. everything is unconditionally declared m3core.dll et. al. should probably take care to GetProcAddress anything they use that was added after some agreed upon lowest common denominator all this header duplication could really use some automation/verification none introduced here today.. From jkrell at elego.de Fri Aug 1 15:43:44 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:43:44 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134344.5D47E10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:43:44 Modified files: cm3/m3-libs/m3core/src/win32/: WinUser.i3 Log message: a few more updates, still quite out of date but arguably that is just fine -- only declare what the m3 libs need From jkrell at elego.de Fri Aug 1 15:46:50 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:46:50 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134650.8A59C10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:46:50 Modified files: cm3/m3-ui/ui/src/winvbt/: WinMsg.m3 cm3/m3-ui/anim3D/src/win-opengl/: Win_OpenGL_Base.m3 Log message: trace a little more, sure could use some code consolidation here.. From jkrell at elego.de Sat Aug 2 03:36:36 2008 From: jkrell at elego.de (Jay Krell) Date: Sat, 2 Aug 2008 3:36:36 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080802013636.7D26C10D4DD9@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/02 03:36:36 Modified files: cm3/m3-ui/vbtkit/src/vbtkitutils/: VBTKitEnv.m3 Log message: allow getting the emacs text editing model via the environment variable From wagner at elego.de Sun Aug 3 11:44:54 2008 From: wagner at elego.de (Olaf Wagner) Date: Sun, 3 Aug 2008 11:44:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080803094454.764EA10D4E29@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/03 11:44:54 Modified files: cm3/m3-ui/vbtkit/src/: m3overrides cm3/m3-ui/vbtkit/src/etext/: MacModel.m3 TextPort.m3 Log message: tentative fix for text selection and insertion in the Mac TextPortModel: The strange behaviour reported by Randy Coleburn (mouse click, left, left, insert --> first character inserted at mouse click point) should now be gone; cursor up and down at the right end of ragged paragraphs still looks a bit strange, but that seems to be not worse than before. It still should be fixed though. These changes should be reviewed and tested by others; they seem to do the right thing for me, but I haven't done extensive testing, especially in other modes. From jkrell at elego.de Tue Aug 5 12:47:56 2008 From: jkrell at elego.de (Jay Krell) Date: Tue, 5 Aug 2008 12:47:56 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080805104756.C117410D4E5D@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/05 12:47:56 Modified files: cm3/m3-sys/m3cc/src/: m3makefile Log message: dependency on libiconv.so still present, use the previous code From jkrell at elego.de Fri Aug 8 15:14:40 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:14:40 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808131440.33CCF10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:14:40 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common Log message: fix this: LINK : warning LNK4044: unrecognized option '/cm3\lib\hand.obj'; ignored From jkrell at elego.de Fri Aug 8 15:29:48 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:29:48 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808132948.C7F0F10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:29:48 Modified files: cm3/m3-libs/m3core/src/win32/: WinDef.i3 WinNT.i3 WinUser.i3 WinUser.m3 m3makefile Added files: cm3/m3-libs/m3core/src/win32/: WinNTc.c Log message: some header file updates WinNT* in particular is used by the upcoming pixmap change (I'd say "fix" but I can't test it, no high dpi system available, nor perhaps apps with noticable deformity) From jkrell at elego.de Fri Aug 8 15:34:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:34:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133451.B6E0810D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:34:51 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Added files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: make all uses of Image.Raw.xres and yres go through accessor functions in fact, renaming xres/yres so any failure to catch uses will fail to compile On Windows, compute xres and yres from Devicecaps(desktop window) and GetSystemMetrics on X Windows, keep the same old 75 (but this looks not difficult to fix) this has a small dependency on WinNT* change from a minute ago From jkrell at elego.de Fri Aug 8 15:38:08 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:38:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133808.8766310D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:38:08 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: remove unused import (I was going to use InterlockedIncrement, but that is perhaps overkill, and unsafe apparently) From jkrell at elego.de Fri Aug 8 15:42:29 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:42:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134229.7A65510D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:42:29 Modified files: cm3/m3-libs/m3core/src/win32/: m3makefile Log message: include WinNTc.c in all platforms, I expect it is needed From jkrell at elego.de Fri Aug 8 15:45:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:45:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134534.4E2C410D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:45:33 Modified files: cm3/m3-libs/m3core/src/win32/: WinNT.i3 Log message: small Win64 fix (not relevant to anything at this time) From jkrell at elego.de Fri Aug 8 15:46:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:46:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134633.7700C10D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:46:33 Added files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 Log message: forgot to add file From rcoleburn at elego.de Sat Aug 9 07:09:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 7:09:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809050943.8325B10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 07:09:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Compute scroll bar slider width based on screen resolution rather than making it a fixed size in millimeters. Allows scroll bar to adapt to different screen resolutions. Used new procedures in ImageInit provided by Jay Krell.---Randy Coleburn From rcoleburn at elego.de Sat Aug 9 10:02:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:02:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809080243.B029A10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:02:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Ensure millimeters are whole, not fractional.---R.Coleburn From rcoleburn at elego.de Sat Aug 9 10:50:13 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:50:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809085014.538C510D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:50:13 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Revert to prior version. My idea about truncating the millimeters was wrong, even though I thought it tested out ok. Sorry. --R.Coleburn From wagner at elego.de Sat Aug 9 11:03:13 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:03:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090313.CB2E410D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:03:13 Modified files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 Log message: make it compile (fix syntax error) From wagner at elego.de Sat Aug 9 11:08:24 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:08:24 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090824.BD1BE10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:08:24 Modified files: cm3/m3-sys/cm3ide/src/: m3overrides Log message: fix overrides (adapt to cm3 build conventions) From jkrell at elego.de Mon Aug 11 12:43:29 2008 From: jkrell at elego.de (Jay Krell) Date: Mon, 11 Aug 2008 12:43:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080811104330.09F1F10D4EF4@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/11 12:43:29 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: When I am copy/pasting code I don't really understand, it helps to look for more nearby code that should also probably be copied. Specifically: Call ReleaseDC after GetDC. From jkrell at elego.de Wed Aug 13 11:41:07 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 11:41:07 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813094107.37C8A10D4DFA@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 11:41:07 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Removed files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: go back one version -- back to hard coding Image xres/yres = 75 and leaving them publically accessible From jkrell at elego.de Wed Aug 13 13:10:59 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 13:10:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813111059.33D8010D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 13:10:59 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: fix it to compile, at least and eliminate common subexpressions will see if we can get Windows to draw much more of this using DrawFrameControl From jkrell at elego.de Thu Aug 14 16:26:59 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:26:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814142659.91A4A10D4F35@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From jkrell at elego.de Thu Aug 14 16:54:08 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:54:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814145408.C970510D4D27@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:54:08 Modified files: cm3/scripts/: find-packages.sh pkginfo.txt cm3/scripts/python/: make-dist.py pylib.py cm3/scripts/win/: find-packages.cmd Log message: add webcat, deskscape, webscape they might not be particularly useful, but they do build and startup From jkrell at elego.de Thu Aug 14 17:05:54 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 17:05:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814150554.AE35010D4F2A@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 17:05:54 Modified files: cm3/scripts/: find-packages.sh cm3/scripts/win/: find-packages.cmd Log message: go back a version unless proven to be needed From rcoleburn at scires.com Thu Aug 14 17:13:25 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 11:13:25 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <20080814142659.91A4A10D4F35@birch.elegosoft.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> Message-ID: <48A4134F.1E75.00D7.1@scires.com> Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:05:13 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A4134F.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:08:40 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:08:40 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From rcoleburn at scires.com Fri Aug 15 01:46:38 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 19:46:38 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A48B97.1E75.00D7.1@scires.com> Jay, this comment is indeed disturbing. Since, I didn't have anything to do with the Trestle implementation for Windows, I can't really offer an authoritative opinion here. The phrase "the only reason this *might* work" is especially disturbing as it sounds as if the idea was still being tested and not known to be sound. I've done quite a bit of multi-threaded programming and the complexity gets out of hand quickly without some rigor and the help of a good thread-aware language like Modula-3. When I say rigor, I mean the programmers have to set some ground rules that are enforced throughout the code that will ensure correct multi-threaded operation. Arbitrarily releasing locks to prevent potential recursive locks seems like a recipe for disaster and hints that the necessary rigor was not established. Regards, Randy >>> Jay 8/14/2008 6:05 PM >>> How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Fri Aug 15 02:11:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 20:11:19 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A49161.1E75.00D7.1@scires.com> Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy >>> Jay 8/14/2008 6:08 PM >>> Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 03:08:23 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 01:08:23 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A49161.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: We should probably add a new type Thread.RecursiveExclusiveLock? And then just use that here? The basic Win32 critical section is such a thing precisely. Not that it is all that great. But it is widely used. Vista adds the "SRWLock" which is a small read/write lock and maybe reduced recursive functionality. It is indeed small, just the size of a pointer, whereas a critical section is fairly large. - Jay Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy>>> Jay 8/14/2008 6:08 PM >>>Maybe we should have another lock type that allows recursive acquires?To be used sparingly, but used here?I know that is often frowned upon -- the need for it implies a design flaw -- butmaybe it is ok sometimes?I don't understand when/why recursive locks are ok or not.- Jay________________________________From: jay.krell at cornell.eduTo: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.comDate: Thu, 14 Aug 2008 22:05:13 +0000Subject: Re: [M3commit] CVS Update: cm3How about this comment in the code:(*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*)?I also need to check if any state has been modified, or cached in locals, withthe locks held that are being released.-Jay________________________________Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3Jay:I haven't studied this code in depth, but on the surface I doubt your change is "correct".I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released.Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon.I would suggest reverting these changes and looking to implement the Release procedure.Regards,Randy>>> Jay Krell 8/14/2008 4:26 PM>>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 12:41:23 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 11:41:23 +0100 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> I am unclear as to why this abstraction is needed for Windows threads when it is not needed for pthreads. Please explain. On Aug 15, 2008, at 2:08 AM, Jay wrote: > > We should probably add a new type Thread.RecursiveExclusiveLock? And > then just use that here? > The basic Win32 critical section is such a thing precisely. Not that > it is all that great. But it is widely used. > Vista adds the "SRWLock" which is a small read/write lock and maybe > reduced recursive functionality. > It is indeed small, just the size of a pointer, whereas a critical > section is fairly large. > > - Jay > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > Update: cm3 > > Jay: > > There is the abstraction of multiple concurrent "readers" but only > one "writer". I actually have such a module that I use sometimes in > my programs. The idea is one of acquiring/releasing read locks and > write locks. The difficulty here is in preventing starvation and > usually requires adherence to some rules (again rigor). By > starvation I mean that writers must wait for all readers to finish > before they get access, so if there is always at least one reader, > the writer will "starve" and never gain access. > > As for recursive locks, again that is usually indicative of a design > flaw. I did once implement a system that permitted the same thread > to acquire a lock multiple times, provided that it eventually > released the lock the same number of times that it acquired it. In > other words, after the first lock, subsequent locks by the same > thread just increment a counter. Then, unlocks decrement the > counter until it is zero with the final unlock actually releasing > the mutex. > > In the code you modified, you could implement such a scheme to > handle the recursive locks, provided that at some point the thread > releases for each lock. The Thread interface has the ability to > note the current thread (i.e., Thread.Self()). So you could make a > stack of locks by same thread. Other threads would have to block > until the first thread's lock stack was empty. > > For example: > > ACQUIRE: If resource available, lock resource mutex and put current > thread on granted stack. Otherwise, if current thread already on > granted stack, push it on stack again and let it proceed. Otherwise > (this is a new thread wanting access), so push this thread onto a > waiting stack and wait From dragisha at m3w.org Fri Aug 15 12:57:36 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 15 Aug 2008 12:57:36 +0200 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <1218797856.6212.4.camel@faramir.m3w.org> Just because we can do something is not a reason to do it. Minimalistic philosophy of Modula-3 API is, as opposed to Java for example, one of it's greatest strenghts. I still can remember a moment when I opened Java book on Thread class. I see it in my nightmares from time to time :). On Fri, 2008-08-15 at 11:41 +0100, Tony Hosking wrote: > I am unclear as to why this abstraction is needed for Windows threads > when it is not needed for pthreads. Please explain. > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? And > > then just use that here? > > The basic Win32 critical section is such a thing precisely. Not that > > it is all that great. But it is widely used. > > Vista adds the "SRWLock" which is a small read/write lock and maybe > > reduced recursive functionality. > > It is indeed small, just the size of a pointer, whereas a critical > > section is fairly large. > > > > - Jay > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > > Update: cm3 > > > > Jay: > > > > There is the abstraction of multiple concurrent "readers" but only > > one "writer". I actually have such a module that I use sometimes in > > my programs. The idea is one of acquiring/releasing read locks and > > write locks. The difficulty here is in preventing starvation and > > usually requires adherence to some rules (again rigor). By > > starvation I mean that writers must wait for all readers to finish > > before they get access, so if there is always at least one reader, > > the writer will "starve" and never gain access. > > > > As for recursive locks, again that is usually indicative of a design > > flaw. I did once implement a system that permitted the same thread > > to acquire a lock multiple times, provided that it eventually > > released the lock the same number of times that it acquired it. In > > other words, after the first lock, subsequent locks by the same > > thread just increment a counter. Then, unlocks decrement the > > counter until it is zero with the final unlock actually releasing > > the mutex. > > > > In the code you modified, you could implement such a scheme to > > handle the recursive locks, provided that at some point the thread > > releases for each lock. The Thread interface has the ability to > > note the current thread (i.e., Thread.Self()). So you could make a > > stack of locks by same thread. Other threads would have to block > > until the first thread's lock stack was empty. > > > > For example: > > > > ACQUIRE: If resource available, lock resource mutex and put current > > thread on granted stack. Otherwise, if current thread already on > > granted stack, push it on stack again and let it proceed. Otherwise > > (this is a new thread wanting access), so push this thread onto a > > waiting stack and wait > -- Dragi?a Duri? From jay.krell at cornell.edu Fri Aug 15 13:29:29 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 11:29:29 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree. pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 13:51:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 12:51:54 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Umm, no, we do not need this for pthreads, since the windowing on non- Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 21:37:46 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 19:37:46 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:03:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:03:41 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 22:12:25 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 20:12:25 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: My understanding, could be all wrong: The thread is not intended to hold no locks, just not while calling out. The thread obeys the usual Trestle locking rules. I *believe* it is not a special thread, it just happens to be the thread that all gui stuff happens on. I believe other threads can still do some Trestle stuff, just not that result to calls to Windows. It's obvious why this is happening, from the callstack. Less obvious what to do about it. Is reentrance and working on some data while already working on that data on the same thread really always such a terrible thing? If you know where the points you might reenter from are? That is, while data is locked, it can and will likely go through some states in which it is "incoherent", in which it is "in the middle" of being updated. But isn't the incoherence likely to be over short runs of code? Not likely to be at a point where a function is called? Otherwise, it becomes difficult for even the holder of the lock to keep things straight. I mean, you know, data is usually coherent, but must often pass through incoherence between coherent states. I would expect very little code to deal with the incoherence, for the updating code to be as short as possible. - Jay CC: m3commit at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3commit] recursive locking in WinTrestleDate: Fri, 15 Aug 2008 21:03:41 +0100 Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:28:09 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:28:09 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: <32788D5B-45A7-4D6B-BBDE-758DE5F1C92A@cs.purdue.edu> That thread is supposed to process an event queue, taking events from the Trestle threads, calling out to Windows on their behalf, and sending the result back to the waiting Trestle thread. What lock is it holding? Re-entrance is not a Modula-3 concept, for good reason. I think you may be oversimplifying the need for re-entrance. On Aug 15, 2008, at 9:12 PM, Jay wrote: > My understanding, could be all wrong: > The thread is not intended to hold no locks, just not while calling > out. > The thread obeys the usual Trestle locking rules. > I *believe* it is not a special thread, it just happens to be the > thread that all gui stuff happens on. I believe other threads can > still do some Trestle stuff, just not that result to calls to Windows. > It's obvious why this is happening, from the callstack. > Less obvious what to do about it. > > Is reentrance and working on some data while already working on that > data on the same thread really always such a terrible thing? If you > know where the points you might reenter from are? > That is, while data is locked, it can and will likely go through > some states in which it is "incoherent", in which it is "in the > middle" of being updated. But isn't the incoherence likely to be > over short runs of code? Not likely to be at a point where a > function is called? Otherwise, it becomes difficult for even the > holder of the lock to keep things straight. I mean, you know, data > is usually coherent, but must often pass through incoherence between > coherent states. I would expect very little code to deal with the > incoherence, for the updating code to be as short as possible. > > - Jay > > > CC: m3commit at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3commit] recursive locking in WinTrestle > Date: Fri, 15 Aug 2008 21:03:41 +0100 > > > Then the question is why that thread, which is intended *not* to > hold any locks so it *can* call out to Windows, actually does hold a > lock. Sounds like a bug to me. The fix is to figure out where it > is acquiring the lock and deduce what should actually be happening. > > On Aug 15, 2008, at 8:37 PM, Jay wrote: > > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at elego.de Fri Aug 22 16:06:53 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:06:53 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822140654.29EBF10D4FE6@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:06:53 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 Log message: Add comment about Raw.x/yres referring to the resolution at which the original image was constructed. Add procedure ForceUnScaledBehavior(). ---Randy Coleburn From rcoleburn at elego.de Fri Aug 22 16:32:47 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:32:47 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822143247.E873510D4E08@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:32:47 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: The following changes greatly improve appearance and functionality of scroll bars on Windows platforms. Internally, deal with all sizes in pixels, vs. millimeters. Provide conversions between millimeters and pixels as needed. Since FormsVBT scales pixmaps by default, scale the slider and trough to match the size of the button pixmaps. Change scroll behavior to more closely resemble user expectation in Windows. Formerly, scrolling was proportional to distance between mouse click and thumb. Now, clicking up/down buttons scrolls backward/forward by one line, and clicking in the slider trough above/below the thumb scrolls backward/forward by one window at a time. Unfortunately, the interface comments still reference the proportional behavior, but that behavior is still done in the POSIX implementation, so the interface comments are correct for POSIX. Perhaps we should add a comment about the Windows implementation; after all, the interface does say the implementation is free to change the behavior.---Randy Coleburn From jkrell at elego.de Fri Aug 1 08:00:22 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:00:22 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060024.DD2DB10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:00:22 Modified files: cm3/scripts/python/: pylib.py Log message: use another popular (at least with me) guess for where the cm3 executable is -- /cm3/bin/cm3 From jkrell at elego.de Fri Aug 1 08:02:31 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 8:02:31 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801060232.3DFDA10D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 08:02:31 Modified files: cm3/scripts/python/: pylib.py Log message: also the D: drive, probably should check SystemDrive environment variable and iterate a..z or c..z From jkrell at elego.de Fri Aug 1 11:46:18 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 11:46:18 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801094618.94B8710D4DCF@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 11:46:18 Modified files: cm3/m3-ui/ui/src/winvbt/: WinScrnColorMap.m3 Log message: fix warning From jkrell at elego.de Fri Aug 1 15:17:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:17:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801131751.6158010D4DD2@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:17:51 Modified files: cm3/m3-libs/m3core/src/win32/: WinGDI.i3 WinGDI.m3 WinUser.i3 WinUser.m3 Log message: some header updates and repair not all of it compatible -- identifiers not in headers generally removed "max" identifiers that aren't constants, removed The Modula-3 headers appear to be from VERY old Windows headers, possibly NT 3.1 beta since the level of stuff the Modula-3 headers make up that isn't in headers is surprising. only added result of findstr #define on winuser.h did not check if things #if'ed out did not add new functions this triggered by alarm over unrecognized output of windows message tracing, which was merely a message that didn't exist long ago when this stuff was written ie: pre NT 4 -- by a lot I think. always normalize some booleans to true/false the extra check of is_chicago is not worthwhile not worth the cost to initialize and use the global though Modula-3 ought to define boolean as a full 32 bit integer that is 0 or non-zero like C... probably should just do this for all extern functions????? it is not really probably a bug for C to return other values for true note that Modula-3 doesn't have the "#if WINVER >= 0x4000" stuff. everything is unconditionally declared m3core.dll et. al. should probably take care to GetProcAddress anything they use that was added after some agreed upon lowest common denominator all this header duplication could really use some automation/verification none introduced here today.. From jkrell at elego.de Fri Aug 1 15:43:44 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:43:44 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134344.5D47E10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:43:44 Modified files: cm3/m3-libs/m3core/src/win32/: WinUser.i3 Log message: a few more updates, still quite out of date but arguably that is just fine -- only declare what the m3 libs need From jkrell at elego.de Fri Aug 1 15:46:50 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 1 Aug 2008 15:46:50 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080801134650.8A59C10D4BF3@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/01 15:46:50 Modified files: cm3/m3-ui/ui/src/winvbt/: WinMsg.m3 cm3/m3-ui/anim3D/src/win-opengl/: Win_OpenGL_Base.m3 Log message: trace a little more, sure could use some code consolidation here.. From jkrell at elego.de Sat Aug 2 03:36:36 2008 From: jkrell at elego.de (Jay Krell) Date: Sat, 2 Aug 2008 3:36:36 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080802013636.7D26C10D4DD9@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/02 03:36:36 Modified files: cm3/m3-ui/vbtkit/src/vbtkitutils/: VBTKitEnv.m3 Log message: allow getting the emacs text editing model via the environment variable From wagner at elego.de Sun Aug 3 11:44:54 2008 From: wagner at elego.de (Olaf Wagner) Date: Sun, 3 Aug 2008 11:44:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080803094454.764EA10D4E29@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/03 11:44:54 Modified files: cm3/m3-ui/vbtkit/src/: m3overrides cm3/m3-ui/vbtkit/src/etext/: MacModel.m3 TextPort.m3 Log message: tentative fix for text selection and insertion in the Mac TextPortModel: The strange behaviour reported by Randy Coleburn (mouse click, left, left, insert --> first character inserted at mouse click point) should now be gone; cursor up and down at the right end of ragged paragraphs still looks a bit strange, but that seems to be not worse than before. It still should be fixed though. These changes should be reviewed and tested by others; they seem to do the right thing for me, but I haven't done extensive testing, especially in other modes. From jkrell at elego.de Tue Aug 5 12:47:56 2008 From: jkrell at elego.de (Jay Krell) Date: Tue, 5 Aug 2008 12:47:56 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080805104756.C117410D4E5D@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/05 12:47:56 Modified files: cm3/m3-sys/m3cc/src/: m3makefile Log message: dependency on libiconv.so still present, use the previous code From jkrell at elego.de Fri Aug 8 15:14:40 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:14:40 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808131440.33CCF10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:14:40 Modified files: cm3/m3-sys/cminstall/src/config/: NT386.common Log message: fix this: LINK : warning LNK4044: unrecognized option '/cm3\lib\hand.obj'; ignored From jkrell at elego.de Fri Aug 8 15:29:48 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:29:48 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808132948.C7F0F10D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:29:48 Modified files: cm3/m3-libs/m3core/src/win32/: WinDef.i3 WinNT.i3 WinUser.i3 WinUser.m3 m3makefile Added files: cm3/m3-libs/m3core/src/win32/: WinNTc.c Log message: some header file updates WinNT* in particular is used by the upcoming pixmap change (I'd say "fix" but I can't test it, no high dpi system available, nor perhaps apps with noticable deformity) From jkrell at elego.de Fri Aug 8 15:34:51 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:34:51 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133451.B6E0810D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:34:51 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Added files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: make all uses of Image.Raw.xres and yres go through accessor functions in fact, renaming xres/yres so any failure to catch uses will fail to compile On Windows, compute xres and yres from Devicecaps(desktop window) and GetSystemMetrics on X Windows, keep the same old 75 (but this looks not difficult to fix) this has a small dependency on WinNT* change from a minute ago From jkrell at elego.de Fri Aug 8 15:38:08 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:38:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808133808.8766310D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:38:08 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: remove unused import (I was going to use InterlockedIncrement, but that is perhaps overkill, and unsafe apparently) From jkrell at elego.de Fri Aug 8 15:42:29 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:42:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134229.7A65510D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:42:29 Modified files: cm3/m3-libs/m3core/src/win32/: m3makefile Log message: include WinNTc.c in all platforms, I expect it is needed From jkrell at elego.de Fri Aug 8 15:45:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:45:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134534.4E2C410D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:45:33 Modified files: cm3/m3-libs/m3core/src/win32/: WinNT.i3 Log message: small Win64 fix (not relevant to anything at this time) From jkrell at elego.de Fri Aug 8 15:46:33 2008 From: jkrell at elego.de (Jay Krell) Date: Fri, 8 Aug 2008 15:46:33 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080808134633.7700C10D4D00@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/08 15:46:33 Added files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 Log message: forgot to add file From rcoleburn at elego.de Sat Aug 9 07:09:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 7:09:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809050943.8325B10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 07:09:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Compute scroll bar slider width based on screen resolution rather than making it a fixed size in millimeters. Allows scroll bar to adapt to different screen resolutions. Used new procedures in ImageInit provided by Jay Krell.---Randy Coleburn From rcoleburn at elego.de Sat Aug 9 10:02:43 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:02:43 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809080243.B029A10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:02:43 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Ensure millimeters are whole, not fractional.---R.Coleburn From rcoleburn at elego.de Sat Aug 9 10:50:13 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Sat, 9 Aug 2008 10:50:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809085014.538C510D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/09 10:50:13 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: Revert to prior version. My idea about truncating the millimeters was wrong, even though I thought it tested out ok. Sorry. --R.Coleburn From wagner at elego.de Sat Aug 9 11:03:13 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:03:13 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090313.CB2E410D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:03:13 Modified files: cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 Log message: make it compile (fix syntax error) From wagner at elego.de Sat Aug 9 11:08:24 2008 From: wagner at elego.de (Olaf Wagner) Date: Sat, 9 Aug 2008 11:08:24 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080809090824.BD1BE10D4D02@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: wagner at birch. 08/08/09 11:08:24 Modified files: cm3/m3-sys/cm3ide/src/: m3overrides Log message: fix overrides (adapt to cm3 build conventions) From jkrell at elego.de Mon Aug 11 12:43:29 2008 From: jkrell at elego.de (Jay Krell) Date: Mon, 11 Aug 2008 12:43:29 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080811104330.09F1F10D4EF4@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/11 12:43:29 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: When I am copy/pasting code I don't really understand, it helps to look for more nearby code that should also probably be copied. Specifically: Call ReleaseDC after GetDC. From jkrell at elego.de Wed Aug 13 11:41:07 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 11:41:07 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813094107.37C8A10D4DFA@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 11:41:07 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 m3makefile cm3/m3-ui/vbtkit/src/lego/POSIX/: m3makefile cm3/m3-ui/vbtkit/src/lego/WIN32/: m3makefile Removed files: cm3/m3-ui/vbtkit/src/lego/: ImageInit.i3 cm3/m3-ui/vbtkit/src/lego/POSIX/: ImageInit.m3 cm3/m3-ui/vbtkit/src/lego/WIN32/: ImageInit.m3 Log message: go back one version -- back to hard coding Image xres/yres = 75 and leaving them publically accessible From jkrell at elego.de Wed Aug 13 13:10:59 2008 From: jkrell at elego.de (Jay Krell) Date: Wed, 13 Aug 2008 13:10:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080813111059.33D8010D4DFE@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/13 13:10:59 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: fix it to compile, at least and eliminate common subexpressions will see if we can get Windows to draw much more of this using DrawFrameControl From jkrell at elego.de Thu Aug 14 16:26:59 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:26:59 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814142659.91A4A10D4F35@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From jkrell at elego.de Thu Aug 14 16:54:08 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 16:54:08 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814145408.C970510D4D27@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 16:54:08 Modified files: cm3/scripts/: find-packages.sh pkginfo.txt cm3/scripts/python/: make-dist.py pylib.py cm3/scripts/win/: find-packages.cmd Log message: add webcat, deskscape, webscape they might not be particularly useful, but they do build and startup From jkrell at elego.de Thu Aug 14 17:05:54 2008 From: jkrell at elego.de (Jay Krell) Date: Thu, 14 Aug 2008 17:05:54 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080814150554.AE35010D4F2A@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: jkrell at birch. 08/08/14 17:05:54 Modified files: cm3/scripts/: find-packages.sh cm3/scripts/win/: find-packages.cmd Log message: go back a version unless proven to be needed From rcoleburn at scires.com Thu Aug 14 17:13:25 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 11:13:25 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <20080814142659.91A4A10D4F35@birch.elegosoft.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> Message-ID: <48A4134F.1E75.00D7.1@scires.com> Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:05:13 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A4134F.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 00:08:40 2008 From: jay.krell at cornell.edu (Jay) Date: Thu, 14 Aug 2008 22:08:40 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? From rcoleburn at scires.com Fri Aug 15 01:46:38 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 19:46:38 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A48B97.1E75.00D7.1@scires.com> Jay, this comment is indeed disturbing. Since, I didn't have anything to do with the Trestle implementation for Windows, I can't really offer an authoritative opinion here. The phrase "the only reason this *might* work" is especially disturbing as it sounds as if the idea was still being tested and not known to be sound. I've done quite a bit of multi-threaded programming and the complexity gets out of hand quickly without some rigor and the help of a good thread-aware language like Modula-3. When I say rigor, I mean the programmers have to set some ground rules that are enforced throughout the code that will ensure correct multi-threaded operation. Arbitrarily releasing locks to prevent potential recursive locks seems like a recipe for disaster and hints that the necessary rigor was not established. Regards, Randy >>> Jay 8/14/2008 6:05 PM >>> How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM >>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at scires.com Fri Aug 15 02:11:19 2008 From: rcoleburn at scires.com (Randy Coleburn) Date: Thu, 14 Aug 2008 20:11:19 -0400 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> Message-ID: <48A49161.1E75.00D7.1@scires.com> Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy >>> Jay 8/14/2008 6:08 PM >>> Maybe we should have another lock type that allows recursive acquires? To be used sparingly, but used here? I know that is often frowned upon -- the need for it implies a design flaw -- but maybe it is ok sometimes? I don't understand when/why recursive locks are ok or not. - Jay ________________________________ From: jay.krell at cornell.edu To: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.com Date: Thu, 14 Aug 2008 22:05:13 +0000 Subject: Re: [M3commit] CVS Update: cm3 How about this comment in the code: (*-------------------------------------------------- raw seething windows ---*) (* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread. *) ? I also need to check if any state has been modified, or cached in locals, with the locks held that are being released. -Jay ________________________________ Date: Thu, 14 Aug 2008 11:13:25 -0400 From: rcoleburn at scires.com To: jkrell at elego.de; m3commit at elegosoft.com Subject: Re: [M3commit] CVS Update: cm3 Jay: I haven't studied this code in depth, but on the surface I doubt your change is "correct". I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released. Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon. I would suggest reverting these changes and looking to implement the Release procedure. Regards, Randy >>> Jay Krell 8/14/2008 4:26 PM>>> CVSROOT:/usr/cvs Changes by:jkrell at birch.08/08/14 16:26:59 Modified files: cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3 Log message: release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 03:08:23 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 01:08:23 +0000 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <48A49161.1E75.00D7.1@scires.com> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: We should probably add a new type Thread.RecursiveExclusiveLock? And then just use that here? The basic Win32 critical section is such a thing precisely. Not that it is all that great. But it is widely used. Vista adds the "SRWLock" which is a small read/write lock and maybe reduced recursive functionality. It is indeed small, just the size of a pointer, whereas a critical section is fairly large. - Jay Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3 Jay: There is the abstraction of multiple concurrent "readers" but only one "writer". I actually have such a module that I use sometimes in my programs. The idea is one of acquiring/releasing read locks and write locks. The difficulty here is in preventing starvation and usually requires adherence to some rules (again rigor). By starvation I mean that writers must wait for all readers to finish before they get access, so if there is always at least one reader, the writer will "starve" and never gain access. As for recursive locks, again that is usually indicative of a design flaw. I did once implement a system that permitted the same thread to acquire a lock multiple times, provided that it eventually released the lock the same number of times that it acquired it. In other words, after the first lock, subsequent locks by the same thread just increment a counter. Then, unlocks decrement the counter until it is zero with the final unlock actually releasing the mutex. In the code you modified, you could implement such a scheme to handle the recursive locks, provided that at some point the thread releases for each lock. The Thread interface has the ability to note the current thread (i.e., Thread.Self()). So you could make a stack of locks by same thread. Other threads would have to block until the first thread's lock stack was empty. For example: ACQUIRE: If resource available, lock resource mutex and put current thread on granted stack. Otherwise, if current thread already on granted stack, push it on stack again and let it proceed. Otherwise (this is a new thread wanting access), so push this thread onto a waiting stack and wait. RELEASE: Pop granted stack and check to ensure the popped stack entry represents the current thread. If not, must crash due to a programming error. If granted stack is now empty, check to see if the waiting stack is empty. If the waiting stack is empty, release the resource mutex. Otherwise (there is at least one thread waiting), pop waiting stack and push onto granted stack. Repeat pop/push for each occurrence of current thread at top of waiting stack. Allow waiting thread to proceed. I can check into providing my read-write locks modules if needed. Regards, Randy>>> Jay 8/14/2008 6:08 PM >>>Maybe we should have another lock type that allows recursive acquires?To be used sparingly, but used here?I know that is often frowned upon -- the need for it implies a design flaw -- butmaybe it is ok sometimes?I don't understand when/why recursive locks are ok or not.- Jay________________________________From: jay.krell at cornell.eduTo: rcoleburn at scires.com; jkrell at elego.de; m3commit at elegosoft.comDate: Thu, 14 Aug 2008 22:05:13 +0000Subject: Re: [M3commit] CVS Update: cm3How about this comment in the code:(*-------------------------------------------------- raw seething windows ---*)(* NOTE: The helper procedures called by WindowProc lock VBT.mu when calling various Trestle procedures. They do not hold locks while calling Win32 because it knows nothing about Modula-3 locks and it can, on a whim, call WindowProc to do something. The only reason this scheme might work is because we have a single Modula-3 thread that's pulling on the Win32 message queue and calling WindowProc. Similarly, we don't bother locking around updates to Child records. They are updated by the single Modula-3/WindowProc thread.*)?I also need to check if any state has been modified, or cached in locals, withthe locks held that are being released.-Jay________________________________Date: Thu, 14 Aug 2008 11:13:25 -0400From: rcoleburn at scires.comTo: jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS Update: cm3Jay:I haven't studied this code in depth, but on the surface I doubt your change is "correct".I think the more telling problem is that if you look at the body of WinTrestle.Release, it is empty, save for a comment that it is not yet implemented. So, it would seem that if WinTrestle.Acquire is called more than once, you will crash because the locks have not been released.Further, I don't think it would make sense to release the locks before calling out to Windows, and then reacquire them upon return. The locks are known to Modula-3 and not Windows. Releasing them will allow other competing threads to acquire them while you are in the Windows subsystem and would seem to violate whatever protections were in place per holding the locks. Upon return from Windows, the thread will be back in competition with others to reacquire the locks it gave up and upon success, there is no guarantee that the other threads didn't disturb the state the original thread depended upon.I would suggest reverting these changes and looking to implement the Release procedure.Regards,Randy>>> Jay Krell 8/14/2008 4:26 PM>>>CVSROOT:/usr/cvsChanges by:jkrell at birch.08/08/14 16:26:59Modified files:cm3/m3-ui/ui/src/winvbt/: WinTrestle.m3Log message:release locks when calling out to Win32 to prevent recursive lock acquisition. correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 12:41:23 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 11:41:23 +0100 Subject: [M3commit] CVS Update: cm3 In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> Message-ID: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> I am unclear as to why this abstraction is needed for Windows threads when it is not needed for pthreads. Please explain. On Aug 15, 2008, at 2:08 AM, Jay wrote: > > We should probably add a new type Thread.RecursiveExclusiveLock? And > then just use that here? > The basic Win32 critical section is such a thing precisely. Not that > it is all that great. But it is widely used. > Vista adds the "SRWLock" which is a small read/write lock and maybe > reduced recursive functionality. > It is indeed small, just the size of a pointer, whereas a critical > section is fairly large. > > - Jay > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > Update: cm3 > > Jay: > > There is the abstraction of multiple concurrent "readers" but only > one "writer". I actually have such a module that I use sometimes in > my programs. The idea is one of acquiring/releasing read locks and > write locks. The difficulty here is in preventing starvation and > usually requires adherence to some rules (again rigor). By > starvation I mean that writers must wait for all readers to finish > before they get access, so if there is always at least one reader, > the writer will "starve" and never gain access. > > As for recursive locks, again that is usually indicative of a design > flaw. I did once implement a system that permitted the same thread > to acquire a lock multiple times, provided that it eventually > released the lock the same number of times that it acquired it. In > other words, after the first lock, subsequent locks by the same > thread just increment a counter. Then, unlocks decrement the > counter until it is zero with the final unlock actually releasing > the mutex. > > In the code you modified, you could implement such a scheme to > handle the recursive locks, provided that at some point the thread > releases for each lock. The Thread interface has the ability to > note the current thread (i.e., Thread.Self()). So you could make a > stack of locks by same thread. Other threads would have to block > until the first thread's lock stack was empty. > > For example: > > ACQUIRE: If resource available, lock resource mutex and put current > thread on granted stack. Otherwise, if current thread already on > granted stack, push it on stack again and let it proceed. Otherwise > (this is a new thread wanting access), so push this thread onto a > waiting stack and wait From dragisha at m3w.org Fri Aug 15 12:57:36 2008 From: dragisha at m3w.org (=?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 15 Aug 2008 12:57:36 +0200 Subject: [M3commit] CVS Update: cm3 In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <1218797856.6212.4.camel@faramir.m3w.org> Just because we can do something is not a reason to do it. Minimalistic philosophy of Modula-3 API is, as opposed to Java for example, one of it's greatest strenghts. I still can remember a moment when I opened Java book on Thread class. I see it in my nightmares from time to time :). On Fri, 2008-08-15 at 11:41 +0100, Tony Hosking wrote: > I am unclear as to why this abstraction is needed for Windows threads > when it is not needed for pthreads. Please explain. > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? And > > then just use that here? > > The basic Win32 critical section is such a thing precisely. Not that > > it is all that great. But it is widely used. > > Vista adds the "SRWLock" which is a small read/write lock and maybe > > reduced recursive functionality. > > It is indeed small, just the size of a pointer, whereas a critical > > section is fairly large. > > > > - Jay > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS > > Update: cm3 > > > > Jay: > > > > There is the abstraction of multiple concurrent "readers" but only > > one "writer". I actually have such a module that I use sometimes in > > my programs. The idea is one of acquiring/releasing read locks and > > write locks. The difficulty here is in preventing starvation and > > usually requires adherence to some rules (again rigor). By > > starvation I mean that writers must wait for all readers to finish > > before they get access, so if there is always at least one reader, > > the writer will "starve" and never gain access. > > > > As for recursive locks, again that is usually indicative of a design > > flaw. I did once implement a system that permitted the same thread > > to acquire a lock multiple times, provided that it eventually > > released the lock the same number of times that it acquired it. In > > other words, after the first lock, subsequent locks by the same > > thread just increment a counter. Then, unlocks decrement the > > counter until it is zero with the final unlock actually releasing > > the mutex. > > > > In the code you modified, you could implement such a scheme to > > handle the recursive locks, provided that at some point the thread > > releases for each lock. The Thread interface has the ability to > > note the current thread (i.e., Thread.Self()). So you could make a > > stack of locks by same thread. Other threads would have to block > > until the first thread's lock stack was empty. > > > > For example: > > > > ACQUIRE: If resource available, lock resource mutex and put current > > thread on granted stack. Otherwise, if current thread already on > > granted stack, push it on stack again and let it proceed. Otherwise > > (this is a new thread wanting access), so push this thread onto a > > waiting stack and wait > -- Dragi?a Duri? From jay.krell at cornell.edu Fri Aug 15 13:29:29 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 11:29:29 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree. pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 13:51:54 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 12:51:54 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> Message-ID: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Umm, no, we do not need this for pthreads, since the windowing on non- Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 21:37:46 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 19:37:46 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:03:41 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:03:41 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> Message-ID: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Aug 15 22:12:25 2008 From: jay.krell at cornell.edu (Jay) Date: Fri, 15 Aug 2008 20:12:25 +0000 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: My understanding, could be all wrong: The thread is not intended to hold no locks, just not while calling out. The thread obeys the usual Trestle locking rules. I *believe* it is not a special thread, it just happens to be the thread that all gui stuff happens on. I believe other threads can still do some Trestle stuff, just not that result to calls to Windows. It's obvious why this is happening, from the callstack. Less obvious what to do about it. Is reentrance and working on some data while already working on that data on the same thread really always such a terrible thing? If you know where the points you might reenter from are? That is, while data is locked, it can and will likely go through some states in which it is "incoherent", in which it is "in the middle" of being updated. But isn't the incoherence likely to be over short runs of code? Not likely to be at a point where a function is called? Otherwise, it becomes difficult for even the holder of the lock to keep things straight. I mean, you know, data is usually coherent, but must often pass through incoherence between coherent states. I would expect very little code to deal with the incoherence, for the updating code to be as short as possible. - Jay CC: m3commit at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3commit] recursive locking in WinTrestleDate: Fri, 15 Aug 2008 21:03:41 +0100 Then the question is why that thread, which is intended *not* to hold any locks so it *can* call out to Windows, actually does hold a lock. Sounds like a bug to me. The fix is to figure out where it is acquiring the lock and deduce what should actually be happening. On Aug 15, 2008, at 8:37 PM, Jay wrote: The comment is false. It does call out to Windows with lock held. - Jay From: hosking at cs.purdue.eduTo: jay.krell at cornell.eduDate: Fri, 15 Aug 2008 12:51:54 +0100CC: m3commit at elegosoft.comSubject: Re: [M3commit] recursive locking in WinTrestle Umm, no, we do not need this for pthreads, since the windowing on non-Windows system does not have the problem you describe. I think that a deeper understanding of the WinTrestle code is needed here. From reading the WinTrestle comments, it says that calls out to Windows are performed by a special thread that has no locks held. So, how can a call-back from Windows (presumably within the same thread) manage to be in a state where the thread has locks. It sounds like there may be something broken in the design of WinTrestle, but this is NOT a motivation to complicate the Thread interface. I suggest that effort be focused in WinTrestle rather than making a global hack that is not justified. On Aug 15, 2008, at 12:29 PM, Jay wrote: There is a case, trivial to hit, in WinTrestle that calls out to Windows with some locks held, Windows calls back into Trestle, Trestle attempts to acquire the locks again, and it dies.It should be easy to implement...All it takes to hit this in WinTrestle is to copy (as in copy/paste) twice in the same app.My example was from the same control but that isn't necessary.Just copy from one Trestle control and then copy from another and it hits.A careful review might reveal that is safe to release the locks, call out, reacquire.The comments indicate a general non-calling out with locks held, but the implementation does not quite agree.pthread has this: PTHREAD_MUTEX_RECURSIVEIt's just bogus?One lock type -- nonrecursive mutex -- is really all you need?I assume a multi-reader/single-writer is also useful. In fact, a mutex can just be a reader/writer where everyone is a writer, though that might waste space (given that Windows now has a reader/writer lock that is just the size of a pointer, maybe not).- Jay> From: hosking at cs.purdue.edu> To: jay.krell at cornell.edu> Date: Fri, 15 Aug 2008 11:41:23 +0100> CC: m3commit at elegosoft.com> Subject: Re: [M3commit] CVS Update: cm3>> I am unclear as to why this abstraction is needed for Windows threads> when it is not needed for pthreads. Please explain.>> On Aug 15, 2008, at 2:08 AM, Jay wrote:>> >> > We should probably add a new type Thread.RecursiveExclusiveLock? And> > then just use that here?> > The basic Win32 critical section is such a thing precisely. Not that> > it is all that great. But it is widely used.> > Vista adds the "SRWLock" which is a small read/write lock and maybe> > reduced recursive functionality.> > It is indeed small, just the size of a pointer, whereas a critical> > section is fairly large.> >> > - Jay> >> >> >> > Date: Thu, 14 Aug 2008 20:11:19 -0400From: rcoleburn at scires.comTo: jay.krell at cornell.edu> > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] CVS> > Update: cm3> >> > Jay:> >> > There is the abstraction of multiple concurrent "readers" but only> > one "writer". I actually have such a module that I use sometimes in> > my programs. The idea is one of acquiring/releasing read locks and> > write locks. The difficulty here is in preventing starvation and> > usually requires adherence to some rules (again rigor). By> > starvation I mean that writers must wait for all readers to finish> > before they get access, so if there is always at least one reader,> > the writer will "starve" and never gain access.> >> > As for recursive locks, again that is usually indicative of a design> > flaw. I did once implement a system that permitted the same thread> > to acquire a lock multiple times, provided that it eventually> > released the lock the same number of times that it acquired it. In> > other words, after the first lock, subsequent locks by the same> > thread just increment a counter. Then, unlocks decrement the> > counter until it is zero with the final unlock actually releasing> > the mutex.> >> > In the code you modified, you could implement such a scheme to> > handle the recursive locks, provided that at some point the thread> > releases for each lock. The Thread interface has the ability to> > note the current thread (i.e., Thread.Self()). So you could make a> > stack of locks by same thread. Other threads would have to block> > until the first thread's lock stack was empty.> >> > For example:> >> > ACQUIRE: If resource available, lock resource mutex and put current> > thread on granted stack. Otherwise, if current thread already on> > granted stack, push it on stack again and let it proceed. Otherwise> > (this is a new thread wanting access), so push this thread onto a> > waiting stack and wait> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Aug 15 22:28:09 2008 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Aug 2008 21:28:09 +0100 Subject: [M3commit] recursive locking in WinTrestle In-Reply-To: References: <20080814142659.91A4A10D4F35@birch.elegosoft.com> <48A4134F.1E75.00D7.1@scires.com> <48A49161.1E75.00D7.1@scires.com> <8297107C-53F0-4D01-8937-D51E68935E3A@cs.purdue.edu> <82AAD9FB-1CD4-4EE7-8D0A-68318B38DB88@cs.purdue.edu> <6E1A2755-3D60-4F1D-9025-4D796DE8C75A@cs.purdue.edu> Message-ID: <32788D5B-45A7-4D6B-BBDE-758DE5F1C92A@cs.purdue.edu> That thread is supposed to process an event queue, taking events from the Trestle threads, calling out to Windows on their behalf, and sending the result back to the waiting Trestle thread. What lock is it holding? Re-entrance is not a Modula-3 concept, for good reason. I think you may be oversimplifying the need for re-entrance. On Aug 15, 2008, at 9:12 PM, Jay wrote: > My understanding, could be all wrong: > The thread is not intended to hold no locks, just not while calling > out. > The thread obeys the usual Trestle locking rules. > I *believe* it is not a special thread, it just happens to be the > thread that all gui stuff happens on. I believe other threads can > still do some Trestle stuff, just not that result to calls to Windows. > It's obvious why this is happening, from the callstack. > Less obvious what to do about it. > > Is reentrance and working on some data while already working on that > data on the same thread really always such a terrible thing? If you > know where the points you might reenter from are? > That is, while data is locked, it can and will likely go through > some states in which it is "incoherent", in which it is "in the > middle" of being updated. But isn't the incoherence likely to be > over short runs of code? Not likely to be at a point where a > function is called? Otherwise, it becomes difficult for even the > holder of the lock to keep things straight. I mean, you know, data > is usually coherent, but must often pass through incoherence between > coherent states. I would expect very little code to deal with the > incoherence, for the updating code to be as short as possible. > > - Jay > > > CC: m3commit at elegosoft.com > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Subject: Re: [M3commit] recursive locking in WinTrestle > Date: Fri, 15 Aug 2008 21:03:41 +0100 > > > Then the question is why that thread, which is intended *not* to > hold any locks so it *can* call out to Windows, actually does hold a > lock. Sounds like a bug to me. The fix is to figure out where it > is acquiring the lock and deduce what should actually be happening. > > On Aug 15, 2008, at 8:37 PM, Jay wrote: > > The comment is false. It does call out to Windows with lock held. > > - Jay > > > > From: hosking at cs.purdue.edu > To: jay.krell at cornell.edu > Date: Fri, 15 Aug 2008 12:51:54 +0100 > CC: m3commit at elegosoft.com > Subject: Re: [M3commit] recursive locking in WinTrestle > > > Umm, no, we do not need this for pthreads, since the windowing on > non-Windows system does not have the problem you describe. I think > that a deeper understanding of the WinTrestle code is needed here. > From reading the WinTrestle comments, it says that calls out to > Windows are performed by a special thread that has no locks held. > So, how can a call-back from Windows (presumably within the same > thread) manage to be in a state where the thread has locks. It > sounds like there may be something broken in the design of > WinTrestle, but this is NOT a motivation to complicate the Thread > interface. I suggest that effort be focused in WinTrestle rather > than making a global hack that is not justified. > > On Aug 15, 2008, at 12:29 PM, Jay wrote: > > There is a case, trivial to hit, in WinTrestle that calls out to > Windows with some locks held, Windows calls back into Trestle, > Trestle attempts to acquire the locks again, and it dies. > > It should be easy to implement... > > All it takes to hit this in WinTrestle is to copy (as in copy/paste) > twice in the same app. > My example was from the same control but that isn't necessary. > Just copy from one Trestle control and then copy from another and it > hits. > > A careful review might reveal that is safe to release the locks, > call out, reacquire. > The comments indicate a general non-calling out with locks held, but > the implementation does not quite agree. > > pthread has this: PTHREAD_MUTEX_RECURSIVE > It's just bogus? > > One lock type -- nonrecursive mutex -- is really all you need? > > I assume a multi-reader/single-writer is also useful. In fact, a > mutex can just be a reader/writer where everyone is a writer, though > that might waste space (given that Windows now has a reader/writer > lock that is just the size of a pointer, maybe not). > > - Jay > > > From: hosking at cs.purdue.edu > > To: jay.krell at cornell.edu > > Date: Fri, 15 Aug 2008 11:41:23 +0100 > > CC: m3commit at elegosoft.com > > Subject: Re: [M3commit] CVS Update: cm3 > > > > I am unclear as to why this abstraction is needed for Windows > threads > > when it is not needed for pthreads. Please explain. > > > > On Aug 15, 2008, at 2:08 AM, Jay wrote: > > > > > > > > We should probably add a new type Thread.RecursiveExclusiveLock? > And > > > then just use that here? > > > The basic Win32 critical section is such a thing precisely. Not > that > > > it is all that great. But it is widely used. > > > Vista adds the "SRWLock" which is a small read/write lock and > maybe > > > reduced recursive functionality. > > > It is indeed small, just the size of a pointer, whereas a critical > > > section is fairly large. > > > > > > - Jay > > > > > > > > > > > > Date: Thu, 14 Aug 2008 20:11:19 -0400From: > rcoleburn at scires.comTo: jay.krell at cornell.edu > > > ; jkrell at elego.de; m3commit at elegosoft.comSubject: Re: [M3commit] > CVS > > > Update: cm3 > > > > > > Jay: > > > > > > There is the abstraction of multiple concurrent "readers" but only > > > one "writer". I actually have such a module that I use sometimes > in > > > my programs. The idea is one of acquiring/releasing read locks and > > > write locks. The difficulty here is in preventing starvation and > > > usually requires adherence to some rules (again rigor). By > > > starvation I mean that writers must wait for all readers to finish > > > before they get access, so if there is always at least one reader, > > > the writer will "starve" and never gain access. > > > > > > As for recursive locks, again that is usually indicative of a > design > > > flaw. I did once implement a system that permitted the same thread > > > to acquire a lock multiple times, provided that it eventually > > > released the lock the same number of times that it acquired it. In > > > other words, after the first lock, subsequent locks by the same > > > thread just increment a counter. Then, unlocks decrement the > > > counter until it is zero with the final unlock actually releasing > > > the mutex. > > > > > > In the code you modified, you could implement such a scheme to > > > handle the recursive locks, provided that at some point the thread > > > releases for each lock. The Thread interface has the ability to > > > note the current thread (i.e., Thread.Self()). So you could make a > > > stack of locks by same thread. Other threads would have to block > > > until the first thread's lock stack was empty. > > > > > > For example: > > > > > > ACQUIRE: If resource available, lock resource mutex and put > current > > > thread on granted stack. Otherwise, if current thread already on > > > granted stack, push it on stack again and let it proceed. > Otherwise > > > (this is a new thread wanting access), so push this thread onto a > > > waiting stack and wait > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcoleburn at elego.de Fri Aug 22 16:06:53 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:06:53 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822140654.29EBF10D4FE6@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:06:53 Modified files: cm3/m3-ui/vbtkit/src/lego/: Image.i3 Image.m3 Log message: Add comment about Raw.x/yres referring to the resolution at which the original image was constructed. Add procedure ForceUnScaledBehavior(). ---Randy Coleburn From rcoleburn at elego.de Fri Aug 22 16:32:47 2008 From: rcoleburn at elego.de (Randy Coleburn) Date: Fri, 22 Aug 2008 16:32:47 () Subject: [M3commit] CVS Update: cm3 Message-ID: <20080822143247.E873510D4E08@birch.elegosoft.com> CVSROOT: /usr/cvs Changes by: rcoleburn at birch. 08/08/22 16:32:47 Modified files: cm3/m3-ui/vbtkit/src/lego/WIN32/: ScrollerVBTClass.m3 Log message: The following changes greatly improve appearance and functionality of scroll bars on Windows platforms. Internally, deal with all sizes in pixels, vs. millimeters. Provide conversions between millimeters and pixels as needed. Since FormsVBT scales pixmaps by default, scale the slider and trough to match the size of the button pixmaps. Change scroll behavior to more closely resemble user expectation in Windows. Formerly, scrolling was proportional to distance between mouse click and thumb. Now, clicking up/down buttons scrolls backward/forward by one line, and clicking in the slider trough above/below the thumb scrolls backward/forward by one window at a time. Unfortunately, the interface comments still reference the proportional behavior, but that behavior is still done in the POSIX implementation, so the interface comments are correct for POSIX. Perhaps we should add a comment about the Windows implementation; after all, the interface does say the implementation is free to change the behavior.---Randy Coleburn