From jay.krell at cornell.edu Thu May 9 22:22:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Thu, 9 May 2013 20:22:58 +0000 Subject: [M3devel] searches for cm3cg In-Reply-To: <20130509180245.159735DEA96@birch.elegosoft.com> References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 I agree that was probably a mistake and I think I downgraded it to only do that for cross builds.Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. - Jay > Date: Thu, 9 May 2013 20:02:45 +0000 > To: m3commit at elegosoft.com > From: rodney at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: rodney at birch. 13/05/09 20:02:45 > > Modified files: > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > release_branch_cm3_5_8 > cm3cfg.common > > Log message: > Since dinosaurs roamed freely, the cm3 executable is not shipped, > even if you specify ship or buildship, unless you also set environment > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > 1) Undermining an executing executable, which won't work on some OSs, and > > 2) Changing compilers in the middle of group of builds. Everything can be > built with the same, prexisting compiler, including the compiler. This > is sometimes essential to avoid bootstrapping barriers, etc. > > A built but not shipped compiler can be installed later, using > scripts/install-cm3-compiler.sh. > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > while the old cm3 remains. If that is the release cm3, every following M3 > compilation suffers: > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > It makes installing of cm3cg work like cm3. > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > *RELEASE* is the other part. Currently, the release searches all over much > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > one in preference to the installed version, creating the same problem. This > seems very difficult to fix without changing the release branch. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 10 02:41:16 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 09 May 2013 19:41:16 -0500 Subject: [M3devel] searches for cm3cg In-Reply-To: References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: <518C422C.70106@lcwb.coop> Yeah, I saw it is long gone in the head. I am trying to get builds easier for someone who has not been rebuilding as things evolve, and the problem shows up there. It did just occur to me that somebody using the release to build the head will nonetheless be using the scripts from the head, not the release. This should create an opportunity to change install-cm3-compiler.sh in the head, along with m3makefile in m3cc, to get around this without requiring a user to update or modify his release installation. On 05/09/2013 03:22 PM, Jay K wrote: > > > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 > > > I agree that was probably a mistake and I think I downgraded it to only do that for cross builds. > Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. > > > - Jay > > > Date: Thu, 9 May 2013 20:02:45 +0000 > > To: m3commit at elegosoft.com > > From: rodney at elego.de > > Subject: [M3commit] CVS Update: cm3 > > > > CVSROOT: /usr/cvs > > Changes by: rodney at birch. 13/05/09 20:02:45 > > > > Modified files: > > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > > release_branch_cm3_5_8 > > cm3cfg.common > > > > Log message: > > Since dinosaurs roamed freely, the cm3 executable is not shipped, > > even if you specify ship or buildship, unless you also set environment > > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > > > 1) Undermining an executing executable, which won't work on some OSs, and > > > > 2) Changing compilers in the middle of group of builds. Everything can be > > built with the same, prexisting compiler, including the compiler. This > > is sometimes essential to avoid bootstrapping barriers, etc. > > > > A built but not shipped compiler can be installed later, using > > scripts/install-cm3-compiler.sh. > > > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > > while the old cm3 remains. If that is the release cm3, every following M3 > > compilation suffers: > > > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > > It makes installing of cm3cg work like cm3. > > > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > > *RELEASE* is the other part. Currently, the release searches all over much > > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > > one in preference to the installed version, creating the same problem. This > > seems very difficult to fix without changing the release branch. > > From jay.krell at cornell.edu Fri May 10 09:31:55 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 07:31:55 +0000 Subject: [M3devel] Textual backend In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , <8A8BAC72-FA52-4802-9764-6F0E0A6A29AD@cs.purdue.edu>, , , , , , , Message-ID: Dirk wants it to be easy to get the "m3cg text" out.I agree. We should add switches that enable outputting the m3cg binary files, the m3cg text files, either alone, or along with whatever is the "final" backend. I know, when cm3cg is the backend, the binary files are already output. And -keep keeps them. Here is a lame proposal: cm3 -m3cgbin outputs the m3cg bin files and exitscm3 -m3cgtext outputs the m3cg text files and exitscm3 -and-m3cgtext outputs the m3cg text files, and runs whatever other backend it would anywaycm3 -and-m3cgbin outputs the m3cg binary files, and runs whatever other backend it would anyway. -and-m3cgtext and -m3cgbin can be combined-m3cgbin and -m3cgtext cannot be-and-m3cgbin is redundant when using cm3cg and would almost silently be ignored; it would imply -keep My choice of switch names is poor, but I think the functionality is slightly useful, very easy to add, and shouldn't interfere with anything. What will be nice about these switches is -and-m3cgtext will make most of the tracing in M3x86.m3, M3C.m3 and parse.c redundant. Actually, the spec should be more like, user lists a set of output file types.cm3 -output:m3cgtext,m3cgbin,c,o kind of like a list of backends, or constructing a pipeline or multiple pipelines. It might be general enough to express generating "o" via "c" or "o" via cm3cg?Or "c", and then "o" via cm3cg? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 13:54:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 13:54:06 +0200 Subject: [M3devel] Windows issue In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , Message-ID: Most probably, you never used zsh :). On Oct 20, 2012, at 9:09 PM, Jay K wrote: > > And Microsoft's command shell is a pain in the backside to say the least. > > > I live in cmd every day and find it vastly preferable to e.g. Mac OS X Terminal. > It's output is much faster. It's command line editing is much better, including history. > The feature invoked by the F8 key -- command line completion against history -- I use all the time and have yet to find anywhere else. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 14:00:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 14:00:06 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: It probably does? Because our interface to pthread specifics is surely not inlined, and sp hacks most probably are. Still, to access this C code, a procedure call is inserted? If sp hack is more efficient even after that, then we probably need to think about inserting such efficient handling into code we generate, in a way you inserted ChecLoadTraceRef..something :). -- Dragi?a Duri? dragisha at m3w.org On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > I used to use pthread specifics but I think jay changed it to C. Not sure why unless compiler can generate more efficiently via sp hacks. > > Sent from my iPhone > > On 30/04/2013, at 3:42 PM, Dragi?a Duri? wrote: > >> Is there a specific reason why TLS is done through C, and not through pthread.(set|get)specific? >> >> dd >> >> -- >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri May 10 17:22:47 2013 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 10 May 2013 08:22:47 -0700 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: <20130510152247.E44811A207D@async.async.caltech.edu> This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >It probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :). > >-- >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > >On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > >> I used to use pthread specifics but I think jay changed it to C. Not = >sure why unless compiler can generate more efficiently via sp hacks. =20 >>=20 >> Sent from my iPhone >>=20 >> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >wrote: >>=20 >>> Is there a specific reason why TLS is done through C, and not through = >pthread.(set|get)specific? >>>=20 >>> dd >>>=20 >>> -- >>> Dragi=C5=A1a Duri=C4=87 >>> dragisha at m3w.org >>>=20 >>>=20 >>>=20 > > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :).

apple-content-edited=3D"true"> >color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

class=3D"Apple-interchange-newline">
http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
dir=3D"auto">
I used to use pthread specifics but I think jay = >changed it to C. Not sure why unless compiler can generate more = >efficiently via sp hacks.  

Sent from my = >iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = ><dragisha at m3w.org> = >wrote:

Is there a specific = >reason why TLS is done through C, and not through = >pthread.(set|get)specific?

dd

apple-content-edited=3D"true"> >font-family: Candara; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>= >

tml>= > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From dragisha at m3w.org Fri May 10 18:39:33 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 18:39:33 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 18:56:34 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 16:56:34 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. For Linux I do at least use __thread instead of pthread_getspecific.I found __thread support too varying/missing to use otherwise. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c?rev=1.169;content-type=text%2Fplain Look for "M3_COMPILER_THREAD_LOCAL" Oh, only for the builtin thread local "activation". There is no mechanism within the Modula-3 language or libraries to use thread locals.To do this efficiently as you'd like, you need to extend the M3CG interface and the existing backends.Or you can do it strictly through libraries. Or you can avoid thread locals. They are almost as bad as globals.They don't make for reentrant code, only thread safe code.Consider a nested for loop using strtok, for example. > no kernel space is involved in accessing That is probably true of all implementations.Apple's is said to be fast, to that is implied.NT doesn't involve the kernel and is reasonably fast (disassemble kernel32!TlsGetValue).Allocation maybe, but not access. > Also, on Linux since 2004, uncontested lock is user-space only Linux is not alone here or necessarily better than any system.Win32 critical sections have always been user-mode only if uncontended.Modula-3 locks I believe are too. I wouldn't be surprised if every system or every current system is the same in this regard.Everyone knows that kernel calls are expensive and to be minimized. > Linux pioneered lightweight processes/threads in free *nix world, and others followed Long after probably every commercial system got it right, e.g. at least Solaris and NT, probably everything else too. - Jay From: dragisha at m3w.org Date: Fri, 10 May 2013 18:39:33 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote:This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 19:04:12 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 19:04:12 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote: > It is in C for portability. Like everything else. > > > Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 20:03:26 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 18:03:26 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: Posix defines a C source interface, not an ABI.The standard does not say enough to write portable .i3 files.Seriously.Look at the mess that was m3-libs/m3core/unix and compare it to today.Function names at the ABI level sometimes didn't match the source name.Structs had to be defined precisely.It was tedious, error-prone, not statically checked, and "very" unsafe.So many times I ported to a new system, only to find struct stat was wrong and my stack was getting corrupted. That problem is gone. We probably now transparently adapt to 64bit time_t. I'd have to check. Or the stuff we had for signal handling. Another mess. All clean and portable today. Tony fixed some of that, for the same reasons. When we get cooperative suspend, the portability will increase much further.And the setjmp buffer size thing. I should get to that soonish. Things are much better for 64bit systems now, as they have been widely ported to and are in widespread use. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 19:04:12 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote:It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 21:31:20 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 21:31:20 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote: > Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 22:47:37 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 20:47:37 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: > Portable .i3 files would be a bit easier if you > did not insist on pointer alignment of 64 on platform No that is not the problem.Sorry, I didn't mean portable .i3 files in general.I meant portable .i3 file that match the Posix implementation on any systems..i3 files backed by .m3 or backed by one C implementation are "portable".That is how we bridge to Posix now -- .i3 files that describe our own C, that isn't #ifdefed at all. Dragisa, do you mean to say that Posix describes an ABI? It does not.It does not describe a "binary" interface, only a source interface.C code can call "open", but "open" does not necessarily exist in any .o or .a file.It can be #defined to something else, or renamed with a compiler-specific #pragma.I have seen both, not long ago, on mainstream systems. NetBSD comes to mind. And Solaris. And Linux (esp. wrt stat). And Darwin. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 21:31:20 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote:Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 23:20:29 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 23:20:29 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). On May 10, 2013, at 10:47 PM, Jay K wrote: > Dragisa, do you mean to say that Posix describes an ABI? It does not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat May 11 19:58:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 11 May 2013 12:58:19 -0500 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> Message-ID: <518E86BB.9090903@lcwb.coop> It would be so much more elegant and Modula-3-like to just add to interface Thread: PROCEDURE SelfClosure(): Closure; that returns the closure used in creating the executing thread, similar to the existing Self. Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using SelfClosure().Whatever. Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. We can already do thread local stuff now, within the current language, by putting it in a subtype of the Closure and accessing this as the Self formal of the apply override procedure. Or just make it local variable(s) of the apply override. The big inconvenience here is that you have to pass it as a parameter all over the place. Even that can sometimes be circumvented by making called procedures that access it local to the apply override, but that won't work if you need to have thread-local access from something in a separate module. On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: > My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). > > > On May 10, 2013, at 10:47 PM, Jay K wrote: > >> Dragisa, do you mean to say that Posix describes an ABI? It does not. > From dragisha at m3w.org Sun May 12 08:23:11 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 May 2013 08:23:11 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <518E86BB.9090903@lcwb.coop> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> <518E86BB.9090903@lcwb.coop> Message-ID: <1E9BADB9-F21D-4A0F-9440-0A7536DDCF64@m3w.org> Modula-3 does cover semantic of "local storage" by design. Be it "global" to thread, or whatever, we have means for it and use it all the time. Problem we were discussing is related to low-level primitives/applications, going beyond (or under?) Modula-3. We must have a line there, a point where we stop implementing in Modula-3 and start assuming some platform behavior. At least until someone invents more advanced hardware where mutual exclusion, thread local storage etc. are basic building elements done in an elegant way, and replaces all current platforms with this new one :). I have a "liking" for .i3 wrapping, but Jay is right there. Set of platforms I am working on is small (4) and big portability issues are rare. If we want total (or almost total) portability, we must think beyond Modula-3 bottom line and implement some things in a portable way and closer to machine, and C in its "universal assembly" role does fit a bill. On May 11, 2013, at 7:58 PM, Rodney M. Bates wrote: > It would be so much more elegant and Modula-3-like to just add to interface Thread: > > PROCEDURE SelfClosure(): Closure; > > that returns the closure used in creating the executing thread, similar to the existing Self. > Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using > SelfClosure().Whatever. > > Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. > > We can already do thread local stuff now, within the current language, by putting it > in a subtype of the Closure and accessing this as the Self formal of the apply override > procedure. Or just make it local variable(s) of the apply override. The big > inconvenience here is that you have to pass it as a parameter all over the place. > > Even that can sometimes be circumvented by making called procedures that access it > local to the apply override, but that won't work if you need to have thread-local > access from something in a separate module. > > > On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: >> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). >> >> >> On May 10, 2013, at 10:47 PM, Jay K wrote: >> >>> Dragisa, do you mean to say that Posix describes an ABI? It does not. >> > From hosking at cs.purdue.edu Mon May 13 02:48:34 2013 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 May 2013 10:48:34 +1000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <98AA05E9-D267-4EFD-AF49-1BE3DE6201E2@cs.purdue.edu> Yes, it will vary by OS and hardware platform. I don?t know how good they are on modern systems, but I do believe a C compiler will do a better job with whatever intrinsics it supports for thread-locals. See http://en.cppreference.com/w/c/thread for the C11 intrinsics. On May 11, 2013, at 1:22 AM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> It probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :). >> >> -- >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: >> >>> I used to use pthread specifics but I think jay changed it to C. Not = >> sure why unless compiler can generate more efficiently via sp hacks. =20 >>> =20 >>> Sent from my iPhone >>> =20 >>> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> wrote: >>> =20 >>>> Is there a specific reason why TLS is done through C, and not through = >> pthread.(set|get)specific? >>>> =20 >>>> dd >>>> =20 >>>> -- >>>> Dragi=C5=A1a Duri=C4=87 >>>> dragisha at m3w.org >>>> =20 >>>> =20 >>>> =20 >> >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >> probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :).

> apple-content-edited=3D"true"> >> > color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

> class=3D"Apple-interchange-newline">
> http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
> dir=3D"auto">
I used to use pthread specifics but I think jay = >> changed it to C. Not sure why unless compiler can generate more = >> efficiently via sp hacks.  

Sent from my = >> iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> <dragisha at m3w.org> = >> wrote:

Is there a specific = >> reason why TLS is done through C, and not through = >> pthread.(set|get)specific?

dd

> apple-content-edited=3D"true"> >> > font-family: Candara; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>> = >>

> tml>= >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From rodney_bates at lcwb.coop Thu May 23 23:34:02 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 16:34:02 -0500 Subject: [M3devel] New TEXT algorithms checked in Message-ID: <519E8B4A.8030107@lcwb.coop> I have checked in changes to the Text implementation in the head. These make no change to the cm3 data structure or its invariants, only different choices of alternate representations for a given abstract character string. The major changes are in Cat, which, while O(1), created representations inefficient to access. This was particularly bad (O(n)) after building a TEXT value by a linear series of concatenations, either left-to-right or right-to-left. These were also very extravagant in heap space usage. To get the changes built and running on your machine, just build and ship m3core, using cm3 or scripts/do-cm3-min.sh. As you might expect with any purely functional style abstraction, there are gains and losses in space and time performance, depending greatly on the usage pattern. Overall, there are mostly small to large net gains on balanced mixtures of creating and accessing strings, both in time and space. When strings are built linearly, net speed gains around 4-to-1 are common. The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. Beyond native word size, there is little reason to expect platform-dependent bugs. Of course, anything can happen. If you know or suspect there are bugs, you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. This will use all the old algorithms instead. This will suffer small, constant-factor time losses relative to the unmodified implementation because of runtime testing of TextClass.Old and also a good bit of gathering of raw performance statistics. You can turn TextClass.Old on and off at will, whenever no thread is executing a Text operation. The results of old and new algorithms are fully interchangeable as operands of any Text operation. You can use other global variables in TextClass to tune the algorithms. Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. These latter are limits on how much flattening is done. You can approximately simulate the behavior of older pm3 Text implementation by setting these to LAST(INTEGER). This will always fully flatten every concatenated string. Differences from pm3 are that the cm3 data structures require that a separate heap object be in front of the open array, and that this will handle WIDECHAR elements in a TEXT. There is also an extensive test program in m3-libs/m3core/tests/newtext/src. build it and run with -h to see its options. It does large numbers of random string operations, running the old and new algorithms side-by-side and comparing the abstract values of their results It also reports a lot of statistics on time and space usage. Retained heap storage numbers are now running high, apparently due to the inability to force garbage collection to complete. From rodney_bates at lcwb.coop Fri May 24 02:10:10 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:10:10 -0500 Subject: [M3devel] Do we really want to truncate wide chars? Message-ID: <519EAFE2.8070600@lcwb.coop> Right now, the implementations of Text.GetChar and Text.SetChars that we got with cm3, silently truncate any of the requested characters whose value won't fit in the range of CHAR. Do we really want this behavior? Is there any existing code that depends on it? I can't imagine a case where client code would want this. I propose we change it to either give a checked runtime error or raise an explicitly declared exception, if the value won't fit in a CHAR. Any thoughts? From rodney_bates at lcwb.coop Fri May 24 02:24:58 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:24:58 -0500 Subject: [M3devel] New VarArray package Message-ID: <519EB35A.5000208@lcwb.coop> There is now a new generic package named VarArray, that provides heavy weight but very flexible self-expanding arrays. They can be indexed by any ordinal type and have any element type except open arrays. As side-effects of various operations, they keep track of the range of subscripts that have been stored into. Occasionally, they reallocate the underlying array provided by the implementation, also as a side-effect. Subscript values can have ORD values in the entire range of INTEGER, although obviously not this many elements can be occupied at one time. There are also some ways of exerting manual control over space allocation and a less safe but low-level, more efficient method of accessing. A companion generic package Ranges is used by VarArray, but could possibly have other uses on its own. There is a test program for exercising them. All is located in m3-libs/vararray. From wagner at elegosoft.com Fri May 24 11:42:38 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:42:38 +0200 Subject: [M3devel] [M3announce] New TEXT algorithms checked in In-Reply-To: <519E8B4A.8030107@lcwb.coop> References: <519E8B4A.8030107@lcwb.coop> Message-ID: <20130524114238.031ceea23aff4bd74f34c5ab@elegosoft.com> On Thu, 23 May 2013 16:34:02 -0500 "Rodney M. Bates" wrote: > I have checked in changes to the Text implementation in the head. > These make no change to the cm3 data structure or its invariants, only > different choices of alternate representations for a given abstract > character string. The major changes are in Cat, which, while O(1), > created representations inefficient to access. This was particularly > bad (O(n)) after building a TEXT value by a linear series of concatenations, > either left-to-right or right-to-left. These were also very extravagant in > heap space usage. > > To get the changes built and running on your machine, just build and ship > m3core, using cm3 or scripts/do-cm3-min.sh. > > As you might expect with any purely functional style abstraction, there > are gains and losses in space and time performance, depending greatly on > the usage pattern. Overall, there are mostly small to large net gains > on balanced mixtures of creating and accessing strings, both in time > and space. When strings are built linearly, net speed gains around 4-to-1 > are common. > > The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. > Beyond native word size, there is little reason to expect platform-dependent > bugs. Of course, anything can happen. If you know or suspect there are bugs, > you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. > This will use all the old algorithms instead. This will suffer small, > constant-factor time losses relative to the unmodified implementation because > of runtime testing of TextClass.Old and also a good bit of gathering of raw > performance statistics. > > You can turn TextClass.Old on and off at will, whenever no thread is executing > a Text operation. The results of old and new algorithms are fully interchangeable > as operands of any Text operation. > > You can use other global variables in TextClass to tune the algorithms. > Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation > trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the > maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. > These latter are limits on how much flattening is done. > > You can approximately simulate the behavior of older pm3 Text implementation > by setting these to LAST(INTEGER). This will always fully flatten every > concatenated string. Differences from pm3 are that the cm3 data structures > require that a separate heap object be in front of the open array, and that > this will handle WIDECHAR elements in a TEXT. > > There is also an extensive test program in m3-libs/m3core/tests/newtext/src. > build it and run with -h to see its options. It does large numbers of > random string operations, running the old and new algorithms side-by-side > and comparing the abstract values of their results It also reports a lot of > statistics on time and space usage. Retained heap storage numbers are now > running high, apparently due to the inability to force garbage collection > to complete. Great! Thanks for that commit, Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Fri May 24 11:43:45 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:43:45 +0200 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: <519EAFE2.8070600@lcwb.coop> References: <519EAFE2.8070600@lcwb.coop> Message-ID: <20130524114345.39256c1cb323aa40ad616a55@elegosoft.com> On Thu, 23 May 2013 19:10:10 -0500 "Rodney M. Bates" wrote: > Right now, the implementations of Text.GetChar and Text.SetChars that we got with > cm3, silently truncate any of the requested characters whose value won't fit in > the range of CHAR. Do we really want this behavior? Is there any existing code > that depends on it? I can't imagine a case where client code would want this. > > I propose we change it to either give a checked runtime error or raise an > explicitly declared exception, if the value won't fit in a CHAR. > > Any thoughts? I'd vote for a runtime error. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri May 24 20:00:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 24 May 2013 13:00:19 -0500 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: References: <519EAFE2.8070600@lcwb.coop> Message-ID: <519FAAB3.9070001@lcwb.coop> Actually, it's hard for me to imagine what would not be broken if we leave it as is, and it happens. Can lopping off high bits off of a character code be a meaningful mapping? The only thing I can think of is if somebody used the Text mechanism as a handy way to manipulate strings of integers of some kind that represented something other than characters. Even then, it seems a stretch that truncation could accomplish anything wanted. On 05/23/2013 07:30 PM, Antony Hosking wrote: > I?ve not given much thought to it. What might break? > > On May 24, 2013, at 10:10 AM, "Rodney M. Bates" wrote: > >> Right now, the implementations of Text.GetChar and Text.SetChars that we got with >> cm3, silently truncate any of the requested characters whose value won't fit in >> the range of CHAR. Do we really want this behavior? Is there any existing code >> that depends on it? I can't imagine a case where client code would want this. >> >> I propose we change it to either give a checked runtime error or raise an >> explicitly declared exception, if the value won't fit in a CHAR. >> >> Any thoughts? >> >> > > From rcolebur at SCIRES.COM Fri May 24 21:47:28 2013 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Fri, 24 May 2013 19:47:28 +0000 Subject: [M3devel] [M3announce] New TEXT algorithms checked in Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF2F13A854@ATLEX04-SRV.SCIRES.LOCAL> Rodney: I want to publicly thank you for your work on the improved TEXT algorithms and implementation. I look forward to testing the new implementation soon. Thanks! Randy Randy C. Coleburn, CISSP-ISSEP, GCED Corporate Information Security Officer (CISO) Scientific Research Corporation 2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339 "Technology Driven-Customer Focused" Quality Policy: "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements." From rodney_bates at lcwb.coop Sun May 26 20:47:32 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 13:47:32 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST Message-ID: <51A258C4.8000607@lcwb.coop> I know this can be done using m3makefiles and the build system, but I am hoping someone can point to a place where it already exists. I need a Module-3 CONST that tells endianness of the machine we are running on, that is available to code in m3core. From dragisha at m3w.org Sun May 26 21:54:15 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 May 2013 21:54:15 +0200 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A258C4.8000607@lcwb.coop> References: <51A258C4.8000607@lcwb.coop> Message-ID: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> My first idea (very fast&dirty) was CONST BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; But this is not constant expression, so says cm3 :). -- Dragi?a Duri? dragisha at m3w.org On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > I know this can be done using m3makefiles and the build system, but I > am hoping someone can point to a place where it already exists. I need > a Module-3 CONST that tells endianness of the machine we are running > on, that is available to code in m3core. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon May 27 04:02:21 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 21:02:21 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> References: <51A258C4.8000607@lcwb.coop> <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> Message-ID: <51A2BEAD.80704@lcwb.coop> Yes. And so says Modula-3, 2.6.15, Constant Expressions. ADR isn't legal in constant expressions either. On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > My first idea (very fast&dirty) was > > CONST > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > But this is not constant expression, so says cm3 :). > > -- > Dragi?a Duri? > dragisha at m3w.org > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > >> I know this can be done using m3makefiles and the build system, but I >> am hoping someone can point to a place where it already exists. I need >> a Module-3 CONST that tells endianness of the machine we are running >> on, that is available to code in m3core. > From jay.krell at cornell.edu Tue May 28 08:17:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 May 2013 06:17:58 +0000 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A2BEAD.80704@lcwb.coop> References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. See here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain Also a good place to expose this would be here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl => Compiler.ThisEndian or such. Why do you need it? Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. - Jay > Date: Sun, 26 May 2013 21:02:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > ADR isn't legal in constant expressions either. > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > My first idea (very fast&dirty) was > > > > CONST > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > But this is not constant expression, so says cm3 :). > > > > -- > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need > >> a Module-3 CONST that tells endianness of the machine we are running > >> on, that is available to code in m3core. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 31 04:37:07 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 May 2013 21:37:07 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: <51A80CD3.9050304@lcwb.coop> On 05/28/2013 01:17 AM, Jay K wrote: > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > > We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. > See here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain > > > Also a good place to expose this would be here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain > => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl > => Compiler.ThisEndian or such. > > > Why do you need it? > Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. > Yeah, I can do it at runtime, but that means it has to be stored in variables. For a given compile, this will never change, and I have some potentially high- frequency paths that I want to squeeze out as much execution time as possible, especially array subscripts derived from endianness. I want to get it and things derived from it in constants. > > - Jay > > > > > > > Date: Sun, 26 May 2013 21:02:21 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > > ADR isn't legal in constant expressions either. > > > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > > My first idea (very fast&dirty) was > > > > > > CONST > > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > > > But this is not constant expression, so says cm3 :). > > > > > > -- > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > >> a Module-3 CONST that tells endianness of the machine we are running > > >> on, that is available to code in m3core. > > > > > From jay.krell at cornell.edu Thu May 9 22:22:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Thu, 9 May 2013 20:22:58 +0000 Subject: [M3devel] searches for cm3cg In-Reply-To: <20130509180245.159735DEA96@birch.elegosoft.com> References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 I agree that was probably a mistake and I think I downgraded it to only do that for cross builds.Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. - Jay > Date: Thu, 9 May 2013 20:02:45 +0000 > To: m3commit at elegosoft.com > From: rodney at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: rodney at birch. 13/05/09 20:02:45 > > Modified files: > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > release_branch_cm3_5_8 > cm3cfg.common > > Log message: > Since dinosaurs roamed freely, the cm3 executable is not shipped, > even if you specify ship or buildship, unless you also set environment > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > 1) Undermining an executing executable, which won't work on some OSs, and > > 2) Changing compilers in the middle of group of builds. Everything can be > built with the same, prexisting compiler, including the compiler. This > is sometimes essential to avoid bootstrapping barriers, etc. > > A built but not shipped compiler can be installed later, using > scripts/install-cm3-compiler.sh. > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > while the old cm3 remains. If that is the release cm3, every following M3 > compilation suffers: > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > It makes installing of cm3cg work like cm3. > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > *RELEASE* is the other part. Currently, the release searches all over much > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > one in preference to the installed version, creating the same problem. This > seems very difficult to fix without changing the release branch. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 10 02:41:16 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 09 May 2013 19:41:16 -0500 Subject: [M3devel] searches for cm3cg In-Reply-To: References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: <518C422C.70106@lcwb.coop> Yeah, I saw it is long gone in the head. I am trying to get builds easier for someone who has not been rebuilding as things evolve, and the problem shows up there. It did just occur to me that somebody using the release to build the head will nonetheless be using the scripts from the head, not the release. This should create an opportunity to change install-cm3-compiler.sh in the head, along with m3makefile in m3cc, to get around this without requiring a user to update or modify his release installation. On 05/09/2013 03:22 PM, Jay K wrote: > > > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 > > > I agree that was probably a mistake and I think I downgraded it to only do that for cross builds. > Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. > > > - Jay > > > Date: Thu, 9 May 2013 20:02:45 +0000 > > To: m3commit at elegosoft.com > > From: rodney at elego.de > > Subject: [M3commit] CVS Update: cm3 > > > > CVSROOT: /usr/cvs > > Changes by: rodney at birch. 13/05/09 20:02:45 > > > > Modified files: > > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > > release_branch_cm3_5_8 > > cm3cfg.common > > > > Log message: > > Since dinosaurs roamed freely, the cm3 executable is not shipped, > > even if you specify ship or buildship, unless you also set environment > > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > > > 1) Undermining an executing executable, which won't work on some OSs, and > > > > 2) Changing compilers in the middle of group of builds. Everything can be > > built with the same, prexisting compiler, including the compiler. This > > is sometimes essential to avoid bootstrapping barriers, etc. > > > > A built but not shipped compiler can be installed later, using > > scripts/install-cm3-compiler.sh. > > > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > > while the old cm3 remains. If that is the release cm3, every following M3 > > compilation suffers: > > > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > > It makes installing of cm3cg work like cm3. > > > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > > *RELEASE* is the other part. Currently, the release searches all over much > > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > > one in preference to the installed version, creating the same problem. This > > seems very difficult to fix without changing the release branch. > > From jay.krell at cornell.edu Fri May 10 09:31:55 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 07:31:55 +0000 Subject: [M3devel] Textual backend In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , <8A8BAC72-FA52-4802-9764-6F0E0A6A29AD@cs.purdue.edu>, , , , , , , Message-ID: Dirk wants it to be easy to get the "m3cg text" out.I agree. We should add switches that enable outputting the m3cg binary files, the m3cg text files, either alone, or along with whatever is the "final" backend. I know, when cm3cg is the backend, the binary files are already output. And -keep keeps them. Here is a lame proposal: cm3 -m3cgbin outputs the m3cg bin files and exitscm3 -m3cgtext outputs the m3cg text files and exitscm3 -and-m3cgtext outputs the m3cg text files, and runs whatever other backend it would anywaycm3 -and-m3cgbin outputs the m3cg binary files, and runs whatever other backend it would anyway. -and-m3cgtext and -m3cgbin can be combined-m3cgbin and -m3cgtext cannot be-and-m3cgbin is redundant when using cm3cg and would almost silently be ignored; it would imply -keep My choice of switch names is poor, but I think the functionality is slightly useful, very easy to add, and shouldn't interfere with anything. What will be nice about these switches is -and-m3cgtext will make most of the tracing in M3x86.m3, M3C.m3 and parse.c redundant. Actually, the spec should be more like, user lists a set of output file types.cm3 -output:m3cgtext,m3cgbin,c,o kind of like a list of backends, or constructing a pipeline or multiple pipelines. It might be general enough to express generating "o" via "c" or "o" via cm3cg?Or "c", and then "o" via cm3cg? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 13:54:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 13:54:06 +0200 Subject: [M3devel] Windows issue In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , Message-ID: Most probably, you never used zsh :). On Oct 20, 2012, at 9:09 PM, Jay K wrote: > > And Microsoft's command shell is a pain in the backside to say the least. > > > I live in cmd every day and find it vastly preferable to e.g. Mac OS X Terminal. > It's output is much faster. It's command line editing is much better, including history. > The feature invoked by the F8 key -- command line completion against history -- I use all the time and have yet to find anywhere else. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 14:00:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 14:00:06 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: It probably does? Because our interface to pthread specifics is surely not inlined, and sp hacks most probably are. Still, to access this C code, a procedure call is inserted? If sp hack is more efficient even after that, then we probably need to think about inserting such efficient handling into code we generate, in a way you inserted ChecLoadTraceRef..something :). -- Dragi?a Duri? dragisha at m3w.org On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > I used to use pthread specifics but I think jay changed it to C. Not sure why unless compiler can generate more efficiently via sp hacks. > > Sent from my iPhone > > On 30/04/2013, at 3:42 PM, Dragi?a Duri? wrote: > >> Is there a specific reason why TLS is done through C, and not through pthread.(set|get)specific? >> >> dd >> >> -- >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri May 10 17:22:47 2013 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 10 May 2013 08:22:47 -0700 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: <20130510152247.E44811A207D@async.async.caltech.edu> This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >It probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :). > >-- >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > >On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > >> I used to use pthread specifics but I think jay changed it to C. Not = >sure why unless compiler can generate more efficiently via sp hacks. =20 >>=20 >> Sent from my iPhone >>=20 >> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >wrote: >>=20 >>> Is there a specific reason why TLS is done through C, and not through = >pthread.(set|get)specific? >>>=20 >>> dd >>>=20 >>> -- >>> Dragi=C5=A1a Duri=C4=87 >>> dragisha at m3w.org >>>=20 >>>=20 >>>=20 > > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :).

apple-content-edited=3D"true"> >color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

class=3D"Apple-interchange-newline">
http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
dir=3D"auto">
I used to use pthread specifics but I think jay = >changed it to C. Not sure why unless compiler can generate more = >efficiently via sp hacks.  

Sent from my = >iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = ><dragisha at m3w.org> = >wrote:

Is there a specific = >reason why TLS is done through C, and not through = >pthread.(set|get)specific?

dd

apple-content-edited=3D"true"> >font-family: Candara; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>= >

tml>= > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From dragisha at m3w.org Fri May 10 18:39:33 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 18:39:33 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 18:56:34 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 16:56:34 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. For Linux I do at least use __thread instead of pthread_getspecific.I found __thread support too varying/missing to use otherwise. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c?rev=1.169;content-type=text%2Fplain Look for "M3_COMPILER_THREAD_LOCAL" Oh, only for the builtin thread local "activation". There is no mechanism within the Modula-3 language or libraries to use thread locals.To do this efficiently as you'd like, you need to extend the M3CG interface and the existing backends.Or you can do it strictly through libraries. Or you can avoid thread locals. They are almost as bad as globals.They don't make for reentrant code, only thread safe code.Consider a nested for loop using strtok, for example. > no kernel space is involved in accessing That is probably true of all implementations.Apple's is said to be fast, to that is implied.NT doesn't involve the kernel and is reasonably fast (disassemble kernel32!TlsGetValue).Allocation maybe, but not access. > Also, on Linux since 2004, uncontested lock is user-space only Linux is not alone here or necessarily better than any system.Win32 critical sections have always been user-mode only if uncontended.Modula-3 locks I believe are too. I wouldn't be surprised if every system or every current system is the same in this regard.Everyone knows that kernel calls are expensive and to be minimized. > Linux pioneered lightweight processes/threads in free *nix world, and others followed Long after probably every commercial system got it right, e.g. at least Solaris and NT, probably everything else too. - Jay From: dragisha at m3w.org Date: Fri, 10 May 2013 18:39:33 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote:This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 19:04:12 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 19:04:12 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote: > It is in C for portability. Like everything else. > > > Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 20:03:26 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 18:03:26 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: Posix defines a C source interface, not an ABI.The standard does not say enough to write portable .i3 files.Seriously.Look at the mess that was m3-libs/m3core/unix and compare it to today.Function names at the ABI level sometimes didn't match the source name.Structs had to be defined precisely.It was tedious, error-prone, not statically checked, and "very" unsafe.So many times I ported to a new system, only to find struct stat was wrong and my stack was getting corrupted. That problem is gone. We probably now transparently adapt to 64bit time_t. I'd have to check. Or the stuff we had for signal handling. Another mess. All clean and portable today. Tony fixed some of that, for the same reasons. When we get cooperative suspend, the portability will increase much further.And the setjmp buffer size thing. I should get to that soonish. Things are much better for 64bit systems now, as they have been widely ported to and are in widespread use. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 19:04:12 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote:It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 21:31:20 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 21:31:20 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote: > Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 22:47:37 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 20:47:37 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: > Portable .i3 files would be a bit easier if you > did not insist on pointer alignment of 64 on platform No that is not the problem.Sorry, I didn't mean portable .i3 files in general.I meant portable .i3 file that match the Posix implementation on any systems..i3 files backed by .m3 or backed by one C implementation are "portable".That is how we bridge to Posix now -- .i3 files that describe our own C, that isn't #ifdefed at all. Dragisa, do you mean to say that Posix describes an ABI? It does not.It does not describe a "binary" interface, only a source interface.C code can call "open", but "open" does not necessarily exist in any .o or .a file.It can be #defined to something else, or renamed with a compiler-specific #pragma.I have seen both, not long ago, on mainstream systems. NetBSD comes to mind. And Solaris. And Linux (esp. wrt stat). And Darwin. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 21:31:20 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote:Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 23:20:29 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 23:20:29 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). On May 10, 2013, at 10:47 PM, Jay K wrote: > Dragisa, do you mean to say that Posix describes an ABI? It does not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat May 11 19:58:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 11 May 2013 12:58:19 -0500 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> Message-ID: <518E86BB.9090903@lcwb.coop> It would be so much more elegant and Modula-3-like to just add to interface Thread: PROCEDURE SelfClosure(): Closure; that returns the closure used in creating the executing thread, similar to the existing Self. Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using SelfClosure().Whatever. Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. We can already do thread local stuff now, within the current language, by putting it in a subtype of the Closure and accessing this as the Self formal of the apply override procedure. Or just make it local variable(s) of the apply override. The big inconvenience here is that you have to pass it as a parameter all over the place. Even that can sometimes be circumvented by making called procedures that access it local to the apply override, but that won't work if you need to have thread-local access from something in a separate module. On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: > My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). > > > On May 10, 2013, at 10:47 PM, Jay K wrote: > >> Dragisa, do you mean to say that Posix describes an ABI? It does not. > From dragisha at m3w.org Sun May 12 08:23:11 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 May 2013 08:23:11 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <518E86BB.9090903@lcwb.coop> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> <518E86BB.9090903@lcwb.coop> Message-ID: <1E9BADB9-F21D-4A0F-9440-0A7536DDCF64@m3w.org> Modula-3 does cover semantic of "local storage" by design. Be it "global" to thread, or whatever, we have means for it and use it all the time. Problem we were discussing is related to low-level primitives/applications, going beyond (or under?) Modula-3. We must have a line there, a point where we stop implementing in Modula-3 and start assuming some platform behavior. At least until someone invents more advanced hardware where mutual exclusion, thread local storage etc. are basic building elements done in an elegant way, and replaces all current platforms with this new one :). I have a "liking" for .i3 wrapping, but Jay is right there. Set of platforms I am working on is small (4) and big portability issues are rare. If we want total (or almost total) portability, we must think beyond Modula-3 bottom line and implement some things in a portable way and closer to machine, and C in its "universal assembly" role does fit a bill. On May 11, 2013, at 7:58 PM, Rodney M. Bates wrote: > It would be so much more elegant and Modula-3-like to just add to interface Thread: > > PROCEDURE SelfClosure(): Closure; > > that returns the closure used in creating the executing thread, similar to the existing Self. > Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using > SelfClosure().Whatever. > > Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. > > We can already do thread local stuff now, within the current language, by putting it > in a subtype of the Closure and accessing this as the Self formal of the apply override > procedure. Or just make it local variable(s) of the apply override. The big > inconvenience here is that you have to pass it as a parameter all over the place. > > Even that can sometimes be circumvented by making called procedures that access it > local to the apply override, but that won't work if you need to have thread-local > access from something in a separate module. > > > On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: >> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). >> >> >> On May 10, 2013, at 10:47 PM, Jay K wrote: >> >>> Dragisa, do you mean to say that Posix describes an ABI? It does not. >> > From hosking at cs.purdue.edu Mon May 13 02:48:34 2013 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 May 2013 10:48:34 +1000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <98AA05E9-D267-4EFD-AF49-1BE3DE6201E2@cs.purdue.edu> Yes, it will vary by OS and hardware platform. I don?t know how good they are on modern systems, but I do believe a C compiler will do a better job with whatever intrinsics it supports for thread-locals. See http://en.cppreference.com/w/c/thread for the C11 intrinsics. On May 11, 2013, at 1:22 AM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> It probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :). >> >> -- >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: >> >>> I used to use pthread specifics but I think jay changed it to C. Not = >> sure why unless compiler can generate more efficiently via sp hacks. =20 >>> =20 >>> Sent from my iPhone >>> =20 >>> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> wrote: >>> =20 >>>> Is there a specific reason why TLS is done through C, and not through = >> pthread.(set|get)specific? >>>> =20 >>>> dd >>>> =20 >>>> -- >>>> Dragi=C5=A1a Duri=C4=87 >>>> dragisha at m3w.org >>>> =20 >>>> =20 >>>> =20 >> >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >> probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :).

> apple-content-edited=3D"true"> >> > color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

> class=3D"Apple-interchange-newline">
> http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
> dir=3D"auto">
I used to use pthread specifics but I think jay = >> changed it to C. Not sure why unless compiler can generate more = >> efficiently via sp hacks.  

Sent from my = >> iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> <dragisha at m3w.org> = >> wrote:

Is there a specific = >> reason why TLS is done through C, and not through = >> pthread.(set|get)specific?

dd

> apple-content-edited=3D"true"> >> > font-family: Candara; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>> = >>

> tml>= >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From rodney_bates at lcwb.coop Thu May 23 23:34:02 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 16:34:02 -0500 Subject: [M3devel] New TEXT algorithms checked in Message-ID: <519E8B4A.8030107@lcwb.coop> I have checked in changes to the Text implementation in the head. These make no change to the cm3 data structure or its invariants, only different choices of alternate representations for a given abstract character string. The major changes are in Cat, which, while O(1), created representations inefficient to access. This was particularly bad (O(n)) after building a TEXT value by a linear series of concatenations, either left-to-right or right-to-left. These were also very extravagant in heap space usage. To get the changes built and running on your machine, just build and ship m3core, using cm3 or scripts/do-cm3-min.sh. As you might expect with any purely functional style abstraction, there are gains and losses in space and time performance, depending greatly on the usage pattern. Overall, there are mostly small to large net gains on balanced mixtures of creating and accessing strings, both in time and space. When strings are built linearly, net speed gains around 4-to-1 are common. The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. Beyond native word size, there is little reason to expect platform-dependent bugs. Of course, anything can happen. If you know or suspect there are bugs, you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. This will use all the old algorithms instead. This will suffer small, constant-factor time losses relative to the unmodified implementation because of runtime testing of TextClass.Old and also a good bit of gathering of raw performance statistics. You can turn TextClass.Old on and off at will, whenever no thread is executing a Text operation. The results of old and new algorithms are fully interchangeable as operands of any Text operation. You can use other global variables in TextClass to tune the algorithms. Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. These latter are limits on how much flattening is done. You can approximately simulate the behavior of older pm3 Text implementation by setting these to LAST(INTEGER). This will always fully flatten every concatenated string. Differences from pm3 are that the cm3 data structures require that a separate heap object be in front of the open array, and that this will handle WIDECHAR elements in a TEXT. There is also an extensive test program in m3-libs/m3core/tests/newtext/src. build it and run with -h to see its options. It does large numbers of random string operations, running the old and new algorithms side-by-side and comparing the abstract values of their results It also reports a lot of statistics on time and space usage. Retained heap storage numbers are now running high, apparently due to the inability to force garbage collection to complete. From rodney_bates at lcwb.coop Fri May 24 02:10:10 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:10:10 -0500 Subject: [M3devel] Do we really want to truncate wide chars? Message-ID: <519EAFE2.8070600@lcwb.coop> Right now, the implementations of Text.GetChar and Text.SetChars that we got with cm3, silently truncate any of the requested characters whose value won't fit in the range of CHAR. Do we really want this behavior? Is there any existing code that depends on it? I can't imagine a case where client code would want this. I propose we change it to either give a checked runtime error or raise an explicitly declared exception, if the value won't fit in a CHAR. Any thoughts? From rodney_bates at lcwb.coop Fri May 24 02:24:58 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:24:58 -0500 Subject: [M3devel] New VarArray package Message-ID: <519EB35A.5000208@lcwb.coop> There is now a new generic package named VarArray, that provides heavy weight but very flexible self-expanding arrays. They can be indexed by any ordinal type and have any element type except open arrays. As side-effects of various operations, they keep track of the range of subscripts that have been stored into. Occasionally, they reallocate the underlying array provided by the implementation, also as a side-effect. Subscript values can have ORD values in the entire range of INTEGER, although obviously not this many elements can be occupied at one time. There are also some ways of exerting manual control over space allocation and a less safe but low-level, more efficient method of accessing. A companion generic package Ranges is used by VarArray, but could possibly have other uses on its own. There is a test program for exercising them. All is located in m3-libs/vararray. From wagner at elegosoft.com Fri May 24 11:42:38 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:42:38 +0200 Subject: [M3devel] [M3announce] New TEXT algorithms checked in In-Reply-To: <519E8B4A.8030107@lcwb.coop> References: <519E8B4A.8030107@lcwb.coop> Message-ID: <20130524114238.031ceea23aff4bd74f34c5ab@elegosoft.com> On Thu, 23 May 2013 16:34:02 -0500 "Rodney M. Bates" wrote: > I have checked in changes to the Text implementation in the head. > These make no change to the cm3 data structure or its invariants, only > different choices of alternate representations for a given abstract > character string. The major changes are in Cat, which, while O(1), > created representations inefficient to access. This was particularly > bad (O(n)) after building a TEXT value by a linear series of concatenations, > either left-to-right or right-to-left. These were also very extravagant in > heap space usage. > > To get the changes built and running on your machine, just build and ship > m3core, using cm3 or scripts/do-cm3-min.sh. > > As you might expect with any purely functional style abstraction, there > are gains and losses in space and time performance, depending greatly on > the usage pattern. Overall, there are mostly small to large net gains > on balanced mixtures of creating and accessing strings, both in time > and space. When strings are built linearly, net speed gains around 4-to-1 > are common. > > The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. > Beyond native word size, there is little reason to expect platform-dependent > bugs. Of course, anything can happen. If you know or suspect there are bugs, > you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. > This will use all the old algorithms instead. This will suffer small, > constant-factor time losses relative to the unmodified implementation because > of runtime testing of TextClass.Old and also a good bit of gathering of raw > performance statistics. > > You can turn TextClass.Old on and off at will, whenever no thread is executing > a Text operation. The results of old and new algorithms are fully interchangeable > as operands of any Text operation. > > You can use other global variables in TextClass to tune the algorithms. > Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation > trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the > maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. > These latter are limits on how much flattening is done. > > You can approximately simulate the behavior of older pm3 Text implementation > by setting these to LAST(INTEGER). This will always fully flatten every > concatenated string. Differences from pm3 are that the cm3 data structures > require that a separate heap object be in front of the open array, and that > this will handle WIDECHAR elements in a TEXT. > > There is also an extensive test program in m3-libs/m3core/tests/newtext/src. > build it and run with -h to see its options. It does large numbers of > random string operations, running the old and new algorithms side-by-side > and comparing the abstract values of their results It also reports a lot of > statistics on time and space usage. Retained heap storage numbers are now > running high, apparently due to the inability to force garbage collection > to complete. Great! Thanks for that commit, Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Fri May 24 11:43:45 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:43:45 +0200 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: <519EAFE2.8070600@lcwb.coop> References: <519EAFE2.8070600@lcwb.coop> Message-ID: <20130524114345.39256c1cb323aa40ad616a55@elegosoft.com> On Thu, 23 May 2013 19:10:10 -0500 "Rodney M. Bates" wrote: > Right now, the implementations of Text.GetChar and Text.SetChars that we got with > cm3, silently truncate any of the requested characters whose value won't fit in > the range of CHAR. Do we really want this behavior? Is there any existing code > that depends on it? I can't imagine a case where client code would want this. > > I propose we change it to either give a checked runtime error or raise an > explicitly declared exception, if the value won't fit in a CHAR. > > Any thoughts? I'd vote for a runtime error. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri May 24 20:00:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 24 May 2013 13:00:19 -0500 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: References: <519EAFE2.8070600@lcwb.coop> Message-ID: <519FAAB3.9070001@lcwb.coop> Actually, it's hard for me to imagine what would not be broken if we leave it as is, and it happens. Can lopping off high bits off of a character code be a meaningful mapping? The only thing I can think of is if somebody used the Text mechanism as a handy way to manipulate strings of integers of some kind that represented something other than characters. Even then, it seems a stretch that truncation could accomplish anything wanted. On 05/23/2013 07:30 PM, Antony Hosking wrote: > I?ve not given much thought to it. What might break? > > On May 24, 2013, at 10:10 AM, "Rodney M. Bates" wrote: > >> Right now, the implementations of Text.GetChar and Text.SetChars that we got with >> cm3, silently truncate any of the requested characters whose value won't fit in >> the range of CHAR. Do we really want this behavior? Is there any existing code >> that depends on it? I can't imagine a case where client code would want this. >> >> I propose we change it to either give a checked runtime error or raise an >> explicitly declared exception, if the value won't fit in a CHAR. >> >> Any thoughts? >> >> > > From rcolebur at SCIRES.COM Fri May 24 21:47:28 2013 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Fri, 24 May 2013 19:47:28 +0000 Subject: [M3devel] [M3announce] New TEXT algorithms checked in Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF2F13A854@ATLEX04-SRV.SCIRES.LOCAL> Rodney: I want to publicly thank you for your work on the improved TEXT algorithms and implementation. I look forward to testing the new implementation soon. Thanks! Randy Randy C. Coleburn, CISSP-ISSEP, GCED Corporate Information Security Officer (CISO) Scientific Research Corporation 2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339 "Technology Driven-Customer Focused" Quality Policy: "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements." From rodney_bates at lcwb.coop Sun May 26 20:47:32 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 13:47:32 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST Message-ID: <51A258C4.8000607@lcwb.coop> I know this can be done using m3makefiles and the build system, but I am hoping someone can point to a place where it already exists. I need a Module-3 CONST that tells endianness of the machine we are running on, that is available to code in m3core. From dragisha at m3w.org Sun May 26 21:54:15 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 May 2013 21:54:15 +0200 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A258C4.8000607@lcwb.coop> References: <51A258C4.8000607@lcwb.coop> Message-ID: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> My first idea (very fast&dirty) was CONST BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; But this is not constant expression, so says cm3 :). -- Dragi?a Duri? dragisha at m3w.org On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > I know this can be done using m3makefiles and the build system, but I > am hoping someone can point to a place where it already exists. I need > a Module-3 CONST that tells endianness of the machine we are running > on, that is available to code in m3core. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon May 27 04:02:21 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 21:02:21 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> References: <51A258C4.8000607@lcwb.coop> <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> Message-ID: <51A2BEAD.80704@lcwb.coop> Yes. And so says Modula-3, 2.6.15, Constant Expressions. ADR isn't legal in constant expressions either. On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > My first idea (very fast&dirty) was > > CONST > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > But this is not constant expression, so says cm3 :). > > -- > Dragi?a Duri? > dragisha at m3w.org > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > >> I know this can be done using m3makefiles and the build system, but I >> am hoping someone can point to a place where it already exists. I need >> a Module-3 CONST that tells endianness of the machine we are running >> on, that is available to code in m3core. > From jay.krell at cornell.edu Tue May 28 08:17:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 May 2013 06:17:58 +0000 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A2BEAD.80704@lcwb.coop> References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. See here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain Also a good place to expose this would be here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl => Compiler.ThisEndian or such. Why do you need it? Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. - Jay > Date: Sun, 26 May 2013 21:02:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > ADR isn't legal in constant expressions either. > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > My first idea (very fast&dirty) was > > > > CONST > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > But this is not constant expression, so says cm3 :). > > > > -- > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need > >> a Module-3 CONST that tells endianness of the machine we are running > >> on, that is available to code in m3core. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 31 04:37:07 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 May 2013 21:37:07 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: <51A80CD3.9050304@lcwb.coop> On 05/28/2013 01:17 AM, Jay K wrote: > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > > We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. > See here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain > > > Also a good place to expose this would be here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain > => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl > => Compiler.ThisEndian or such. > > > Why do you need it? > Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. > Yeah, I can do it at runtime, but that means it has to be stored in variables. For a given compile, this will never change, and I have some potentially high- frequency paths that I want to squeeze out as much execution time as possible, especially array subscripts derived from endianness. I want to get it and things derived from it in constants. > > - Jay > > > > > > > Date: Sun, 26 May 2013 21:02:21 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > > ADR isn't legal in constant expressions either. > > > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > > My first idea (very fast&dirty) was > > > > > > CONST > > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > > > But this is not constant expression, so says cm3 :). > > > > > > -- > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > >> a Module-3 CONST that tells endianness of the machine we are running > > >> on, that is available to code in m3core. > > > > > From jay.krell at cornell.edu Thu May 9 22:22:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Thu, 9 May 2013 20:22:58 +0000 Subject: [M3devel] searches for cm3cg In-Reply-To: <20130509180245.159735DEA96@birch.elegosoft.com> References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 I agree that was probably a mistake and I think I downgraded it to only do that for cross builds.Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. - Jay > Date: Thu, 9 May 2013 20:02:45 +0000 > To: m3commit at elegosoft.com > From: rodney at elego.de > Subject: [M3commit] CVS Update: cm3 > > CVSROOT: /usr/cvs > Changes by: rodney at birch. 13/05/09 20:02:45 > > Modified files: > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > release_branch_cm3_5_8 > cm3cfg.common > > Log message: > Since dinosaurs roamed freely, the cm3 executable is not shipped, > even if you specify ship or buildship, unless you also set environment > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > 1) Undermining an executing executable, which won't work on some OSs, and > > 2) Changing compilers in the middle of group of builds. Everything can be > built with the same, prexisting compiler, including the compiler. This > is sometimes essential to avoid bootstrapping barriers, etc. > > A built but not shipped compiler can be installed later, using > scripts/install-cm3-compiler.sh. > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > while the old cm3 remains. If that is the release cm3, every following M3 > compilation suffers: > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > It makes installing of cm3cg work like cm3. > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > *RELEASE* is the other part. Currently, the release searches all over much > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > one in preference to the installed version, creating the same problem. This > seems very difficult to fix without changing the release branch. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 10 02:41:16 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 09 May 2013 19:41:16 -0500 Subject: [M3devel] searches for cm3cg In-Reply-To: References: <20130509180245.159735DEA96@birch.elegosoft.com> Message-ID: <518C422C.70106@lcwb.coop> Yeah, I saw it is long gone in the head. I am trying to get builds easier for someone who has not been rebuilding as things evolve, and the problem shows up there. It did just occur to me that somebody using the release to build the head will nonetheless be using the scripts from the head, not the release. This should create an opportunity to change install-cm3-compiler.sh in the head, along with m3makefile in m3cc, to get around this without requiring a user to update or modify his release installation. On 05/09/2013 03:22 PM, Jay K wrote: > > > Currently, the release searches all over much of the known universe for a cm3cg/m3cgc1 > > > I agree that was probably a mistake and I think I downgraded it to only do that for cross builds. > Though cross builds should probably use, like, /usr/local/bin/cm3//cm3cg. > > > - Jay > > > Date: Thu, 9 May 2013 20:02:45 +0000 > > To: m3commit at elegosoft.com > > From: rodney at elego.de > > Subject: [M3commit] CVS Update: cm3 > > > > CVSROOT: /usr/cvs > > Changes by: rodney at birch. 13/05/09 20:02:45 > > > > Modified files: > > cm3/m3-sys/cminstall/src/config-no-install/: Tag: > > release_branch_cm3_5_8 > > cm3cfg.common > > > > Log message: > > Since dinosaurs roamed freely, the cm3 executable is not shipped, > > even if you specify ship or buildship, unless you also set environment > > variable INSTALL_CM3_IN_BIN="yes". This avoids: > > > > 1) Undermining an executing executable, which won't work on some OSs, and > > > > 2) Changing compilers in the middle of group of builds. Everything can be > > built with the same, prexisting compiler, including the compiler. This > > is sometimes essential to avoid bootstrapping barriers, etc. > > > > A built but not shipped compiler can be installed later, using > > scripts/install-cm3-compiler.sh. > > > > Item 2) above is also relevant to cm3cg as well. We currently have an apparent > > bootstrap barrier where both cm3 and cm3cg need to be updated to the head > > atomically. As is, a new cm3cg is built and installed in /usr/local/cm3/bin, > > while the old cm3 remains. If that is the release cm3, every following M3 > > compilation suffers: > > > > m3cgc1: fatal error: *** illegal type: 0x17, at m3cg_lineno 4 > > > > The change to m3-sys/m3cc/src/m3makefile in the *HEAD* is one part of the fix. > > It makes installing of cm3cg work like cm3. > > > > The change to m3-sys/cminstall/src/config-no-install/cm3cfgt.common in the > > *RELEASE* is the other part. Currently, the release searches all over much > > of the known universe for a cm3cg/m3cgc1, and will pick up even an uninstalled > > one in preference to the installed version, creating the same problem. This > > seems very difficult to fix without changing the release branch. > > From jay.krell at cornell.edu Fri May 10 09:31:55 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 07:31:55 +0000 Subject: [M3devel] Textual backend In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , <8A8BAC72-FA52-4802-9764-6F0E0A6A29AD@cs.purdue.edu>, , , , , , , Message-ID: Dirk wants it to be easy to get the "m3cg text" out.I agree. We should add switches that enable outputting the m3cg binary files, the m3cg text files, either alone, or along with whatever is the "final" backend. I know, when cm3cg is the backend, the binary files are already output. And -keep keeps them. Here is a lame proposal: cm3 -m3cgbin outputs the m3cg bin files and exitscm3 -m3cgtext outputs the m3cg text files and exitscm3 -and-m3cgtext outputs the m3cg text files, and runs whatever other backend it would anywaycm3 -and-m3cgbin outputs the m3cg binary files, and runs whatever other backend it would anyway. -and-m3cgtext and -m3cgbin can be combined-m3cgbin and -m3cgtext cannot be-and-m3cgbin is redundant when using cm3cg and would almost silently be ignored; it would imply -keep My choice of switch names is poor, but I think the functionality is slightly useful, very easy to add, and shouldn't interfere with anything. What will be nice about these switches is -and-m3cgtext will make most of the tracing in M3x86.m3, M3C.m3 and parse.c redundant. Actually, the spec should be more like, user lists a set of output file types.cm3 -output:m3cgtext,m3cgbin,c,o kind of like a list of backends, or constructing a pipeline or multiple pipelines. It might be general enough to express generating "o" via "c" or "o" via cm3cg?Or "c", and then "o" via cm3cg? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 13:54:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 13:54:06 +0200 Subject: [M3devel] Windows issue In-Reply-To: References: <266A382C-BDB8-43CF-9688-512D0ECB2AB1@cs.purdue.edu>, , Message-ID: Most probably, you never used zsh :). On Oct 20, 2012, at 9:09 PM, Jay K wrote: > > And Microsoft's command shell is a pain in the backside to say the least. > > > I live in cmd every day and find it vastly preferable to e.g. Mac OS X Terminal. > It's output is much faster. It's command line editing is much better, including history. > The feature invoked by the F8 key -- command line completion against history -- I use all the time and have yet to find anywhere else. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 14:00:06 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 14:00:06 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: It probably does? Because our interface to pthread specifics is surely not inlined, and sp hacks most probably are. Still, to access this C code, a procedure call is inserted? If sp hack is more efficient even after that, then we probably need to think about inserting such efficient handling into code we generate, in a way you inserted ChecLoadTraceRef..something :). -- Dragi?a Duri? dragisha at m3w.org On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > I used to use pthread specifics but I think jay changed it to C. Not sure why unless compiler can generate more efficiently via sp hacks. > > Sent from my iPhone > > On 30/04/2013, at 3:42 PM, Dragi?a Duri? wrote: > >> Is there a specific reason why TLS is done through C, and not through pthread.(set|get)specific? >> >> dd >> >> -- >> Dragi?a Duri? >> dragisha at m3w.org >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri May 10 17:22:47 2013 From: mika at async.caltech.edu (mika at async.caltech.edu) Date: Fri, 10 May 2013 08:22:47 -0700 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> Message-ID: <20130510152247.E44811A207D@async.async.caltech.edu> This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/plain; > charset=utf-8 > >It probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :). > >-- >Dragi=C5=A1a Duri=C4=87 >dragisha at m3w.org > > > >On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: > >> I used to use pthread specifics but I think jay changed it to C. Not = >sure why unless compiler can generate more efficiently via sp hacks. =20 >>=20 >> Sent from my iPhone >>=20 >> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >wrote: >>=20 >>> Is there a specific reason why TLS is done through C, and not through = >pthread.(set|get)specific? >>>=20 >>> dd >>>=20 >>> -- >>> Dragi=C5=A1a Duri=C4=87 >>> dragisha at m3w.org >>>=20 >>>=20 >>>=20 > > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >Content-Transfer-Encoding: quoted-printable >Content-Type: text/html; > charset=utf-8 > >-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >probably does=E2=80=A6 Because our interface to pthread specifics is = >surely not inlined, and sp hacks most probably are. Still, to access = >this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >efficient even after that, then we probably need to think about = >inserting such efficient handling into code we generate, in a way you = >inserted ChecLoadTraceRef..something :).

apple-content-edited=3D"true"> >color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >font-variant: normal; font-weight: normal; letter-spacing: normal; = >line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >normal; font-weight: normal; letter-spacing: normal; line-height: = >normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >Helvetica; font-style: normal; font-variant: normal; font-weight: = >normal; letter-spacing: normal; line-height: normal; orphans: 2; = >text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

class=3D"Apple-interchange-newline">
http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
dir=3D"auto">
I used to use pthread specifics but I think jay = >changed it to C. Not sure why unless compiler can generate more = >efficiently via sp hacks.  

Sent from my = >iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = ><dragisha at m3w.org> = >wrote:

Is there a specific = >reason why TLS is done through C, and not through = >pthread.(set|get)specific?

dd

apple-content-edited=3D"true"> >font-family: Candara; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; = >">
--
style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87class=3D"Apple-style-span" style=3D"border-collapse: separate; = >font-family: Helvetica; font-style: normal; font-variant: normal; = >font-weight: normal; letter-spacing: normal; line-height: normal; = >orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >none; white-space: normal; widows: 2; word-spacing: 0px; = >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >normal; font-variant: normal; font-weight: normal; letter-spacing: = >normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >-webkit-border-vertical-spacing: 0px; = >-webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >-webkit-line-break: after-white-space; ">

= >

>
>= >

tml>= > >--Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From dragisha at m3w.org Fri May 10 18:39:33 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 18:39:33 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 18:56:34 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 16:56:34 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. For Linux I do at least use __thread instead of pthread_getspecific.I found __thread support too varying/missing to use otherwise. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c?rev=1.169;content-type=text%2Fplain Look for "M3_COMPILER_THREAD_LOCAL" Oh, only for the builtin thread local "activation". There is no mechanism within the Modula-3 language or libraries to use thread locals.To do this efficiently as you'd like, you need to extend the M3CG interface and the existing backends.Or you can do it strictly through libraries. Or you can avoid thread locals. They are almost as bad as globals.They don't make for reentrant code, only thread safe code.Consider a nested for loop using strtok, for example. > no kernel space is involved in accessing That is probably true of all implementations.Apple's is said to be fast, to that is implied.NT doesn't involve the kernel and is reasonably fast (disassemble kernel32!TlsGetValue).Allocation maybe, but not access. > Also, on Linux since 2004, uncontested lock is user-space only Linux is not alone here or necessarily better than any system.Win32 critical sections have always been user-mode only if uncontended.Modula-3 locks I believe are too. I wouldn't be surprised if every system or every current system is the same in this regard.Everyone knows that kernel calls are expensive and to be minimized. > Linux pioneered lightweight processes/threads in free *nix world, and others followed Long after probably every commercial system got it right, e.g. at least Solaris and NT, probably everything else too. - Jay From: dragisha at m3w.org Date: Fri, 10 May 2013 18:39:33 +0200 To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) AFAIK, Linux ABI uses dedicated register (one of segment registers) to access TLS. It's setting is part of context switch, and one is only supposed to use it read-only. No kernel space is involved in accessing, and was not since NPTL was introduced. Old and FreeBSD does ring bells :). Linux pioneered lightweight processes/threads in free *nix world, and others followed. If your test was near enough in time to 2004-5-6, then probably it was a time when FreeBSD played catch-up. Also, on Linux since 2004, uncontested lock is user-space only. It probably involves a barrier of some kind, but it is under 0.5microsecond in worst case, as opposed to 20-50microsecond for kernel space operation. So, it is efficient :). On May 10, 2013, at 5:22 PM, mika at async.caltech.edu wrote:This is just one data point, but I remember testing the performance of thread locals on pthreads, probably on an old FreeBSD system, and the performance was horrendously bad. There was a system call involved in obtaining the pointer to the thread-local area. I was had some clever algorithms I wanted to use thread locals for (since you can do all sorts of things without locks then), but I gave up on it since it was way more expensive than a lock on that system... Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 19:04:12 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 19:04:12 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> Message-ID: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote: > It is in C for portability. Like everything else. > > > Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 20:03:26 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 18:03:26 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: Posix defines a C source interface, not an ABI.The standard does not say enough to write portable .i3 files.Seriously.Look at the mess that was m3-libs/m3core/unix and compare it to today.Function names at the ABI level sometimes didn't match the source name.Structs had to be defined precisely.It was tedious, error-prone, not statically checked, and "very" unsafe.So many times I ported to a new system, only to find struct stat was wrong and my stack was getting corrupted. That problem is gone. We probably now transparently adapt to 64bit time_t. I'd have to check. Or the stuff we had for signal handling. Another mess. All clean and portable today. Tony fixed some of that, for the same reasons. When we get cooperative suspend, the portability will increase much further.And the setjmp buffer size thing. I should get to that soonish. Things are much better for 64bit systems now, as they have been widely ported to and are in widespread use. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 19:04:12 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu We are talking POSIX here. There are same chances for #define to change behavior of your C as are to change binding through EXTERNAL. In 90s, when I did port of pm3 to LINUX_ALPHA, most problems I met were in C code, due to it's non-portability to RISC, 64bit, big-endian machine. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 6:56 PM, Jay K wrote:It is in C for portability. Like everything else. Otherwise you have to duplicate the C headers in Modula-3, and they end up being system specific because there are too many ways to turn the portable C source interface into non-portable ABI -- #defines, #pragmas, etc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 21:31:20 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 21:31:20 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> Message-ID: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. -- Dragi?a Duri? dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote: > Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri May 10 22:47:37 2013 From: jay.krell at cornell.edu (Jay K) Date: Fri, 10 May 2013 20:47:37 +0000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: > Portable .i3 files would be a bit easier if you > did not insist on pointer alignment of 64 on platform No that is not the problem.Sorry, I didn't mean portable .i3 files in general.I meant portable .i3 file that match the Posix implementation on any systems..i3 files backed by .m3 or backed by one C implementation are "portable".That is how we bridge to Posix now -- .i3 files that describe our own C, that isn't #ifdefed at all. Dragisa, do you mean to say that Posix describes an ABI? It does not.It does not describe a "binary" interface, only a source interface.C code can call "open", but "open" does not necessarily exist in any .o or .a file.It can be #defined to something else, or renamed with a compiler-specific #pragma.I have seen both, not long ago, on mainstream systems. NetBSD comes to mind. And Solaris. And Linux (esp. wrt stat). And Darwin. - Jay Subject: Re: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) From: dragisha at m3w.org Date: Fri, 10 May 2013 21:31:20 +0200 CC: mika at async.caltech.edu; m3devel at elegosoft.com To: jay.krell at cornell.edu Yes, POSIX defines it, and then usually does not change it every now and then. Portable .i3 files would be a bit easier if you did not insist on pointer alignment of 64 on platform where 32bit is a standard :). Bit packing gives enough control over how struct is being made with only problem there being insistence on 64bit alignment for pointers, although it can be controlled with BITS .. FOR. As for m3core mess, and it was lots of it, you are right and work you two did is great. --Dragi?a Duri?dragisha at m3w.org On May 10, 2013, at 8:03 PM, Jay K wrote:Posix defines a C source interface, not an ABI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dragisha at m3w.org Fri May 10 23:20:29 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Fri, 10 May 2013 23:20:29 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> Message-ID: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). On May 10, 2013, at 10:47 PM, Jay K wrote: > Dragisa, do you mean to say that Posix describes an ABI? It does not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Sat May 11 19:58:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sat, 11 May 2013 12:58:19 -0500 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> Message-ID: <518E86BB.9090903@lcwb.coop> It would be so much more elegant and Modula-3-like to just add to interface Thread: PROCEDURE SelfClosure(): Closure; that returns the closure used in creating the executing thread, similar to the existing Self. Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using SelfClosure().Whatever. Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. We can already do thread local stuff now, within the current language, by putting it in a subtype of the Closure and accessing this as the Self formal of the apply override procedure. Or just make it local variable(s) of the apply override. The big inconvenience here is that you have to pass it as a parameter all over the place. Even that can sometimes be circumvented by making called procedures that access it local to the apply override, but that won't work if you need to have thread-local access from something in a separate module. On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: > My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). > > > On May 10, 2013, at 10:47 PM, Jay K wrote: > >> Dragisa, do you mean to say that Posix describes an ABI? It does not. > From dragisha at m3w.org Sun May 12 08:23:11 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 12 May 2013 08:23:11 +0200 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <518E86BB.9090903@lcwb.coop> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org>, <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu>, , <20130510152247.E44811A207D@async.async.caltech.edu>, <039D51CE-3B22-4420-AF17-DE645682D295@m3w.org> , <80DA8B47-EE97-4660-9D59-EB6087C4DE65@m3w.org> , <0E023568-B043-4DBC-99C6-79D97372798A@m3w.org> <2CFD09F5-E8F9-484A-90CC-E483328D7CE3@m3w.org> <518E86BB.9090903@lcwb.coop> Message-ID: <1E9BADB9-F21D-4A0F-9440-0A7536DDCF64@m3w.org> Modula-3 does cover semantic of "local storage" by design. Be it "global" to thread, or whatever, we have means for it and use it all the time. Problem we were discussing is related to low-level primitives/applications, going beyond (or under?) Modula-3. We must have a line there, a point where we stop implementing in Modula-3 and start assuming some platform behavior. At least until someone invents more advanced hardware where mutual exclusion, thread local storage etc. are basic building elements done in an elegant way, and replaces all current platforms with this new one :). I have a "liking" for .i3 wrapping, but Jay is right there. Set of platforms I am working on is small (4) and big portability issues are rare. If we want total (or almost total) portability, we must think beyond Modula-3 bottom line and implement some things in a portable way and closer to machine, and C in its "universal assembly" role does fit a bill. On May 11, 2013, at 7:58 PM, Rodney M. Bates wrote: > It would be so much more elegant and Modula-3-like to just add to interface Thread: > > PROCEDURE SelfClosure(): Closure; > > that returns the closure used in creating the executing thread, similar to the existing Self. > Then you can put your thread-local data in a subtype of Closure, and access it anywhere, using > SelfClosure().Whatever. > > Or, maybe just put a reference to the Closure in the Thread.T, and use Self().closure. > > We can already do thread local stuff now, within the current language, by putting it > in a subtype of the Closure and accessing this as the Self formal of the apply override > procedure. Or just make it local variable(s) of the apply override. The big > inconvenience here is that you have to pass it as a parameter all over the place. > > Even that can sometimes be circumvented by making called procedures that access it > local to the apply override, but that won't work if you need to have thread-local > access from something in a separate module. > > > On 05/10/2013 04:20 PM, Dragi?a Duri? wrote: >> My only mention of ABI was related to TLS implementation, and maybe for pointer alignment on platform? No POSIX there :). >> >> >> On May 10, 2013, at 10:47 PM, Jay K wrote: >> >>> Dragisa, do you mean to say that Posix describes an ABI? It does not. >> > From hosking at cs.purdue.edu Mon May 13 02:48:34 2013 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 13 May 2013 10:48:34 +1000 Subject: [M3devel] thread local storage, but in C pieces of runtime... (jay, tony?) In-Reply-To: <20130510152247.E44811A207D@async.async.caltech.edu> References: <8489B8D9-9442-4EE1-940F-4C69B96AF61B@m3w.org> <80EA00C3-AACC-472D-A43B-E2121CBD4AAB@cs.purdue.edu> <20130510152247.E44811A207D@async.async.caltech.edu> Message-ID: <98AA05E9-D267-4EFD-AF49-1BE3DE6201E2@cs.purdue.edu> Yes, it will vary by OS and hardware platform. I don?t know how good they are on modern systems, but I do believe a C compiler will do a better job with whatever intrinsics it supports for thread-locals. See http://en.cppreference.com/w/c/thread for the C11 intrinsics. On May 11, 2013, at 1:22 AM, mika at async.caltech.edu wrote: > This is just one data point, but I remember testing the performance of thread locals > on pthreads, probably on an old FreeBSD system, and the performance was horrendously > bad. There was a system call involved in obtaining the pointer to the thread-local > area. I was had some clever algorithms I wanted to use thread locals for (since you > can do all sorts of things without locks then), but I gave up on it since it was way > more expensive than a lock on that system... > > Mika > > =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/plain; >> charset=utf-8 >> >> It probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :). >> >> -- >> Dragi=C5=A1a Duri=C4=87 >> dragisha at m3w.org >> >> >> >> On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote: >> >>> I used to use pthread specifics but I think jay changed it to C. Not = >> sure why unless compiler can generate more efficiently via sp hacks. =20 >>> =20 >>> Sent from my iPhone >>> =20 >>> On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> wrote: >>> =20 >>>> Is there a specific reason why TLS is done through C, and not through = >> pthread.(set|get)specific? >>>> =20 >>>> dd >>>> =20 >>>> -- >>>> Dragi=C5=A1a Duri=C4=87 >>>> dragisha at m3w.org >>>> =20 >>>> =20 >>>> =20 >> >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954 >> Content-Transfer-Encoding: quoted-printable >> Content-Type: text/html; >> charset=utf-8 >> >> > -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">It = >> probably does=E2=80=A6 Because our interface to pthread specifics is = >> surely not inlined, and sp hacks most probably are. Still, to access = >> this C code, a procedure call is inserted=E2=80=A6 If sp hack is more = >> efficient even after that, then we probably need to think about = >> inserting such efficient handling into code we generate, in a way you = >> inserted ChecLoadTraceRef..something :).

> apple-content-edited=3D"true"> >> > color: rgb(0, 0, 0); font-family: Candara; font-style: normal; = >> font-variant: normal; font-weight: normal; letter-spacing: normal; = >> line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: = >> 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: = >> 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; color: = >> rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: = >> normal; font-weight: normal; letter-spacing: normal; line-height: = >> normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; = >> text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; color: rgb(0, 0, 0); font-family: = >> Helvetica; font-style: normal; font-variant: normal; font-weight: = >> normal; letter-spacing: normal; line-height: normal; orphans: 2; = >> text-align: -webkit-auto; text-indent: 0px; text-transform: none; = >> white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>>
On Apr 30, 2013, at 9:21 AM, Tony Hosking wrote:

> class=3D"Apple-interchange-newline">
> http-equiv=3D"content-type" content=3D"text/html; charset=3Dutf-8">
> dir=3D"auto">
I used to use pthread specifics but I think jay = >> changed it to C. Not sure why unless compiler can generate more = >> efficiently via sp hacks.  

Sent from my = >> iPhone

On 30/04/2013, at 3:42 PM, Dragi=C5=A1a Duri=C4=87 = >> <dragisha at m3w.org> = >> wrote:

Is there a specific = >> reason why TLS is done through C, and not through = >> pthread.(set|get)specific?

dd

> apple-content-edited=3D"true"> >> > font-family: Candara; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; = >> ">
--
> style=3D"font-family: Helvetica; ">Dragi=C5=A1a Duri=C4=87> class=3D"Apple-style-span" style=3D"border-collapse: separate; = >> font-family: Helvetica; font-style: normal; font-variant: normal; = >> font-weight: normal; letter-spacing: normal; line-height: normal; = >> orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = >> none; white-space: normal; widows: 2; word-spacing: 0px; = >> -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = >> 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">> style=3D"border-collapse: separate; font-family: Helvetica; font-style: = >> normal; font-variant: normal; font-weight: normal; letter-spacing: = >> normal; line-height: normal; orphans: 2; text-align: -webkit-auto; = >> text-indent: 0px; text-transform: none; white-space: normal; widows: 2; = >> word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; = >> -webkit-border-vertical-spacing: 0px; = >> -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = >> auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
> style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; = >> -webkit-line-break: after-white-space; ">

= >>

>>
>> = >>

> tml>= >> >> --Apple-Mail=_EEF8E283-B126-4B87-BA54-81432C085954-- From rodney_bates at lcwb.coop Thu May 23 23:34:02 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 16:34:02 -0500 Subject: [M3devel] New TEXT algorithms checked in Message-ID: <519E8B4A.8030107@lcwb.coop> I have checked in changes to the Text implementation in the head. These make no change to the cm3 data structure or its invariants, only different choices of alternate representations for a given abstract character string. The major changes are in Cat, which, while O(1), created representations inefficient to access. This was particularly bad (O(n)) after building a TEXT value by a linear series of concatenations, either left-to-right or right-to-left. These were also very extravagant in heap space usage. To get the changes built and running on your machine, just build and ship m3core, using cm3 or scripts/do-cm3-min.sh. As you might expect with any purely functional style abstraction, there are gains and losses in space and time performance, depending greatly on the usage pattern. Overall, there are mostly small to large net gains on balanced mixtures of creating and accessing strings, both in time and space. When strings are built linearly, net speed gains around 4-to-1 are common. The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. Beyond native word size, there is little reason to expect platform-dependent bugs. Of course, anything can happen. If you know or suspect there are bugs, you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. This will use all the old algorithms instead. This will suffer small, constant-factor time losses relative to the unmodified implementation because of runtime testing of TextClass.Old and also a good bit of gathering of raw performance statistics. You can turn TextClass.Old on and off at will, whenever no thread is executing a Text operation. The results of old and new algorithms are fully interchangeable as operands of any Text operation. You can use other global variables in TextClass to tune the algorithms. Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. These latter are limits on how much flattening is done. You can approximately simulate the behavior of older pm3 Text implementation by setting these to LAST(INTEGER). This will always fully flatten every concatenated string. Differences from pm3 are that the cm3 data structures require that a separate heap object be in front of the open array, and that this will handle WIDECHAR elements in a TEXT. There is also an extensive test program in m3-libs/m3core/tests/newtext/src. build it and run with -h to see its options. It does large numbers of random string operations, running the old and new algorithms side-by-side and comparing the abstract values of their results It also reports a lot of statistics on time and space usage. Retained heap storage numbers are now running high, apparently due to the inability to force garbage collection to complete. From rodney_bates at lcwb.coop Fri May 24 02:10:10 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:10:10 -0500 Subject: [M3devel] Do we really want to truncate wide chars? Message-ID: <519EAFE2.8070600@lcwb.coop> Right now, the implementations of Text.GetChar and Text.SetChars that we got with cm3, silently truncate any of the requested characters whose value won't fit in the range of CHAR. Do we really want this behavior? Is there any existing code that depends on it? I can't imagine a case where client code would want this. I propose we change it to either give a checked runtime error or raise an explicitly declared exception, if the value won't fit in a CHAR. Any thoughts? From rodney_bates at lcwb.coop Fri May 24 02:24:58 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 23 May 2013 19:24:58 -0500 Subject: [M3devel] New VarArray package Message-ID: <519EB35A.5000208@lcwb.coop> There is now a new generic package named VarArray, that provides heavy weight but very flexible self-expanding arrays. They can be indexed by any ordinal type and have any element type except open arrays. As side-effects of various operations, they keep track of the range of subscripts that have been stored into. Occasionally, they reallocate the underlying array provided by the implementation, also as a side-effect. Subscript values can have ORD values in the entire range of INTEGER, although obviously not this many elements can be occupied at one time. There are also some ways of exerting manual control over space allocation and a less safe but low-level, more efficient method of accessing. A companion generic package Ranges is used by VarArray, but could possibly have other uses on its own. There is a test program for exercising them. All is located in m3-libs/vararray. From wagner at elegosoft.com Fri May 24 11:42:38 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:42:38 +0200 Subject: [M3devel] [M3announce] New TEXT algorithms checked in In-Reply-To: <519E8B4A.8030107@lcwb.coop> References: <519E8B4A.8030107@lcwb.coop> Message-ID: <20130524114238.031ceea23aff4bd74f34c5ab@elegosoft.com> On Thu, 23 May 2013 16:34:02 -0500 "Rodney M. Bates" wrote: > I have checked in changes to the Text implementation in the head. > These make no change to the cm3 data structure or its invariants, only > different choices of alternate representations for a given abstract > character string. The major changes are in Cat, which, while O(1), > created representations inefficient to access. This was particularly > bad (O(n)) after building a TEXT value by a linear series of concatenations, > either left-to-right or right-to-left. These were also very extravagant in > heap space usage. > > To get the changes built and running on your machine, just build and ship > m3core, using cm3 or scripts/do-cm3-min.sh. > > As you might expect with any purely functional style abstraction, there > are gains and losses in space and time performance, depending greatly on > the usage pattern. Overall, there are mostly small to large net gains > on balanced mixtures of creating and accessing strings, both in time > and space. When strings are built linearly, net speed gains around 4-to-1 > are common. > > The new algorithms are extensively tested on LINUXLIBC6 and AMD64_LINUX. > Beyond native word size, there is little reason to expect platform-dependent > bugs. Of course, anything can happen. If you know or suspect there are bugs, > you can disable the new algorithms by importing and setting TextClass.Old:=TRUE. > This will use all the old algorithms instead. This will suffer small, > constant-factor time losses relative to the unmodified implementation because > of runtime testing of TextClass.Old and also a good bit of gathering of raw > performance statistics. > > You can turn TextClass.Old on and off at will, whenever no thread is executing > a Text operation. The results of old and new algorithms are fully interchangeable > as operands of any Text operation. > > You can use other global variables in TextClass to tune the algorithms. > Setting TextClass.Flatten:=FALSE disables partial flattening of concatenation > trees. Otherwise, TextClass.MaxFlat8, and TextClass.MaxFlatWide set the > maximum lengths of internal open arrays of CHAR and WIDECHAR, respectively. > These latter are limits on how much flattening is done. > > You can approximately simulate the behavior of older pm3 Text implementation > by setting these to LAST(INTEGER). This will always fully flatten every > concatenated string. Differences from pm3 are that the cm3 data structures > require that a separate heap object be in front of the open array, and that > this will handle WIDECHAR elements in a TEXT. > > There is also an extensive test program in m3-libs/m3core/tests/newtext/src. > build it and run with -h to see its options. It does large numbers of > random string operations, running the old and new algorithms side-by-side > and comparing the abstract values of their results It also reports a lot of > statistics on time and space usage. Retained heap storage numbers are now > running high, apparently due to the inability to force garbage collection > to complete. Great! Thanks for that commit, Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From wagner at elegosoft.com Fri May 24 11:43:45 2013 From: wagner at elegosoft.com (Olaf Wagner) Date: Fri, 24 May 2013 11:43:45 +0200 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: <519EAFE2.8070600@lcwb.coop> References: <519EAFE2.8070600@lcwb.coop> Message-ID: <20130524114345.39256c1cb323aa40ad616a55@elegosoft.com> On Thu, 23 May 2013 19:10:10 -0500 "Rodney M. Bates" wrote: > Right now, the implementations of Text.GetChar and Text.SetChars that we got with > cm3, silently truncate any of the requested characters whose value won't fit in > the range of CHAR. Do we really want this behavior? Is there any existing code > that depends on it? I can't imagine a case where client code would want this. > > I propose we change it to either give a checked runtime error or raise an > explicitly declared exception, if the value won't fit in a CHAR. > > Any thoughts? I'd vote for a runtime error. Olaf -- Olaf Wagner -- elego Software Solutions GmbH -- http://www.elegosoft.com Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 Gesch?ftsf?hrer: Michael Diers, Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From rodney_bates at lcwb.coop Fri May 24 20:00:19 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 24 May 2013 13:00:19 -0500 Subject: [M3devel] Do we really want to truncate wide chars? In-Reply-To: References: <519EAFE2.8070600@lcwb.coop> Message-ID: <519FAAB3.9070001@lcwb.coop> Actually, it's hard for me to imagine what would not be broken if we leave it as is, and it happens. Can lopping off high bits off of a character code be a meaningful mapping? The only thing I can think of is if somebody used the Text mechanism as a handy way to manipulate strings of integers of some kind that represented something other than characters. Even then, it seems a stretch that truncation could accomplish anything wanted. On 05/23/2013 07:30 PM, Antony Hosking wrote: > I?ve not given much thought to it. What might break? > > On May 24, 2013, at 10:10 AM, "Rodney M. Bates" wrote: > >> Right now, the implementations of Text.GetChar and Text.SetChars that we got with >> cm3, silently truncate any of the requested characters whose value won't fit in >> the range of CHAR. Do we really want this behavior? Is there any existing code >> that depends on it? I can't imagine a case where client code would want this. >> >> I propose we change it to either give a checked runtime error or raise an >> explicitly declared exception, if the value won't fit in a CHAR. >> >> Any thoughts? >> >> > > From rcolebur at SCIRES.COM Fri May 24 21:47:28 2013 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Fri, 24 May 2013 19:47:28 +0000 Subject: [M3devel] [M3announce] New TEXT algorithms checked in Message-ID: <0BB8FA59C2932741A3A2941A8B9D8BFF2F13A854@ATLEX04-SRV.SCIRES.LOCAL> Rodney: I want to publicly thank you for your work on the improved TEXT algorithms and implementation. I look forward to testing the new implementation soon. Thanks! Randy Randy C. Coleburn, CISSP-ISSEP, GCED Corporate Information Security Officer (CISO) Scientific Research Corporation 2300 Windy Ridge Parkway, Suite 400 South, Atlanta, Georgia 30339 "Technology Driven-Customer Focused" Quality Policy: "SRC is committed to delivering continually improving research & engineering excellence that meets or exceeds customer requirements." From rodney_bates at lcwb.coop Sun May 26 20:47:32 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 13:47:32 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST Message-ID: <51A258C4.8000607@lcwb.coop> I know this can be done using m3makefiles and the build system, but I am hoping someone can point to a place where it already exists. I need a Module-3 CONST that tells endianness of the machine we are running on, that is available to code in m3core. From dragisha at m3w.org Sun May 26 21:54:15 2013 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 26 May 2013 21:54:15 +0200 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A258C4.8000607@lcwb.coop> References: <51A258C4.8000607@lcwb.coop> Message-ID: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> My first idea (very fast&dirty) was CONST BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; But this is not constant expression, so says cm3 :). -- Dragi?a Duri? dragisha at m3w.org On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > I know this can be done using m3makefiles and the build system, but I > am hoping someone can point to a place where it already exists. I need > a Module-3 CONST that tells endianness of the machine we are running > on, that is available to code in m3core. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Mon May 27 04:02:21 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 26 May 2013 21:02:21 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> References: <51A258C4.8000607@lcwb.coop> <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org> Message-ID: <51A2BEAD.80704@lcwb.coop> Yes. And so says Modula-3, 2.6.15, Constant Expressions. ADR isn't legal in constant expressions either. On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > My first idea (very fast&dirty) was > > CONST > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > But this is not constant expression, so says cm3 :). > > -- > Dragi?a Duri? > dragisha at m3w.org > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > >> I know this can be done using m3makefiles and the build system, but I >> am hoping someone can point to a place where it already exists. I need >> a Module-3 CONST that tells endianness of the machine we are running >> on, that is available to code in m3core. > From jay.krell at cornell.edu Tue May 28 08:17:58 2013 From: jay.krell at cornell.edu (Jay K) Date: Tue, 28 May 2013 06:17:58 +0000 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: <51A2BEAD.80704@lcwb.coop> References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. See here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain Also a good place to expose this would be here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl => Compiler.ThisEndian or such. Why do you need it? Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. - Jay > Date: Sun, 26 May 2013 21:02:21 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > ADR isn't legal in constant expressions either. > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > My first idea (very fast&dirty) was > > > > CONST > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > But this is not constant expression, so says cm3 :). > > > > -- > > Dragi?a Duri? > > dragisha at m3w.org > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > >> I know this can be done using m3makefiles and the build system, but I > >> am hoping someone can point to a place where it already exists. I need > >> a Module-3 CONST that tells endianness of the machine we are running > >> on, that is available to code in m3core. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Fri May 31 04:37:07 2013 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 30 May 2013 21:37:07 -0500 Subject: [M3devel] Getting IsLittleEndian as a Modula-3 CONST In-Reply-To: References: <51A258C4.8000607@lcwb.coop>, <78E95894-36F9-4308-A26B-98F07519CDE1@m3w.org>, <51A2BEAD.80704@lcwb.coop> Message-ID: <51A80CD3.9050304@lcwb.coop> On 05/28/2013 01:17 AM, Jay K wrote: > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > > We already do expose this, TARGET_ENDIAN, via m3makefile as you allude. > See here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/float/m3makefile?rev=1.38;content-type=text%2Fplain > > > Also a good place to expose this would be here: > http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/m3makefile?rev=1.34;content-type=text%2Fplain > => http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/runtime/common/Compiler.tmpl > => Compiler.ThisEndian or such. > > > Why do you need it? > Very little code cares about this sort of thing, and you can usually figure it out easily enough portably at runtime. > Yeah, I can do it at runtime, but that means it has to be stored in variables. For a given compile, this will never change, and I have some potentially high- frequency paths that I want to squeeze out as much execution time as possible, especially array subscripts derived from endianness. I want to get it and things derived from it in constants. > > - Jay > > > > > > > Date: Sun, 26 May 2013 21:02:21 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] Getting IsLittleEndian as a Modula-3 CONST > > > > Yes. And so says Modula-3, 2.6.15, Constant Expressions. > > ADR isn't legal in constant expressions either. > > > > On 05/26/2013 02:54 PM, Dragi?a Duri? wrote: > > > My first idea (very fast&dirty) was > > > > > > CONST > > > BigEndian = LOOPHOLE(1, ARRAY[0..7] OF BITS 8 FOR [0..255])[0] = 0; > > > > > > But this is not constant expression, so says cm3 :). > > > > > > -- > > > Dragi?a Duri? > > > dragisha at m3w.org > > > > > > > > > > > > On May 26, 2013, at 8:47 PM, Rodney M. Bates wrote: > > > > > >> I know this can be done using m3makefiles and the build system, but I > > >> am hoping someone can point to a place where it already exists. I need > > >> a Module-3 CONST that tells endianness of the machine we are running > > >> on, that is available to code in m3core. > > > > >