From rodney_bates at lcwb.coop Fri Apr 1 19:43:25 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 01 Apr 2011 12:43:25 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: <20110305180558.451421A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu> Message-ID: <4D960EBD.2090206@lcwb.coop> I have looked at this a bit. I do not get this symptom on LINUXLIBC6, using the Cartesian product of release/head for m3gdb and the compiler. However, I see a lot of broken stuff that used to work. In all the cases I have seen, it correlates to debugging code compiled by the head compiler, while the release and head versions of m3gdb make no difference to the symptoms in anything I have tried. I believe Jay had started updating the m3gdb in head to use a newer gdb. I am not sure how far that got. Its source code is vastly different, and most of the other language support in gdb has been ripped out. There is quite a lot of change in the stabs info generated by the two compiler versions. The existing way for a long time has been that this is constructed during the first pass over the input to cm3cg, before anything other than initial tree building is done. This is a very limited strategy, because cm3cg can change things later, and this will not be reflected in the stabs info. Debugging anything involving static links has been broken for a long time, because tree-nested.c in cm3cg does some very convoluted stuff with the way static ancestor activation records are accessed, and I have not been able to see an easy way to get stabs info to reflect it. I have thought for a long time that we really need to go to a version of dwarf for debug info. stabs is very limited, and there is a lot of info in the stabs output that is not really stabs, it's just kludgily coded values crammed inside various fields of stabs that the compiler and m3gdb both agree on (well, they used to.) Maybe the current head cm3cg can be fixed, but I know Jay has made big changes in where type information is actually developed, so it may not be worth it. After all the work I have done on this to get m3gdb to where it was, I am unenthusiastic about a bunch more work just to get back to where we were. I have long lists of improvements to make to m3gdb, and it's hard to go back that way. And, It's all in C :-(. If I put much effort into fixing m3gdb and its support in cm3cg at this point, it will probably be a rework to use dwarf. That is likely a big job too, but at least more rewarding and likely to lead to better m3gdb function in the end. For now, it you want a working m3gdb, you really have to use the release compiler. On 03/05/2011 12:05 PM, Mika Nystrom wrote: > Hi m3devel, > > I don't know what's happened here. m3gdb used to work, and work fairly > well (never perfectly, but...). Now I'm trying to debug a program and > I get the following: > > (m3gdb) break Main.m3:65 > Debug info for file "Main.mc" not in stabs format > (m3gdb) > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > systems too. > > Everything is from the CVS head. > > Main was compiled as follows, says cm3 -commands: > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ ci > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > I'm not even sure where to begin looking. Is it a configuration issue, > something broken in the compiler, or what? > > I fear m3gdb is essentially useless at the moment. > > Mika > From jay.krell at cornell.edu Sat Apr 2 03:26:39 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 01:26:39 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <4D960EBD.2090206@lcwb.coop> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: I really tried not to break anything in the wierdo stubs/m3gdb stuff. The work I have been doing to properly declare types is I believe disabled currently. As types become more correct, things start breaking. Either because the optimizer is more aggressive, or because more sanity checks can be done by the compiler, and we don't give it particularly good trees. In most things I've looked it, it's really not any mystery what is wrong with our trees btw. It's not really generally the fault of gcc being obscure or changing. Though perhaps to some extent that is true. Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not accessible from C source, and therefore bugs in them go undetected. It's more generally clearly not great stuff in cm3cg. The main bad thing is that field access is done by just adding to base pointers and casting away. Note that stabs is not supported on Darwin. Nor as I recall HP-UX/HPPA64. All the more reason to no do things this way. Note furthermore that discussion here is often confused. Really no particular debug-info generation code should be cm3cg. In our part -- m3cg/parse.c Neither stabs nor dwarf nor coff nor anything else. The right approach is to build an intermediate form with reasonably good types. The backend handles it from there, whatever debug-info format is requested on the command line. This also enables using stock/current gdb, instead of hacked up/old gdb. I made no visible progress upgrading our fork of gdb. I'd rather just have it go away. I understand, that custom information is being tunneled through to m3gdb. But that "custom information" is largely just basic type information, that any decent C compiler/debug-info/debugger can handle. Someone should investigate if we really pass stuff through that can't be well represented in gcc's type information. For example: sub ranges? sets? enums? packed? Some higher level description of "methods" (really -- whatever C++ does should suffice; if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) At some point the new type information was really breaking stuff so I think I disabled it. The system is kind of incomplete and fragile and the way forward is not always easy or possible in small steps. :( - Jay > Date: Fri, 1 Apr 2011 12:43:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > using the Cartesian product of release/head for m3gdb and the compiler. > > However, I see a lot of broken stuff that used to work. In all the cases > I have seen, it correlates to debugging code compiled by the head compiler, > while the release and head versions of m3gdb make no difference to the > symptoms in anything I have tried. > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > I am not sure how far that got. Its source code is vastly different, and > most of the other language support in gdb has been ripped out. > > There is quite a lot of change in the stabs info generated by the two > compiler versions. The existing way for a long time has been that this > is constructed during the first pass over the input to cm3cg, before > anything other than initial tree building is done. This is a very > limited strategy, because cm3cg can change things later, and this will > not be reflected in the stabs info. > > Debugging anything involving static links has been broken for a long time, > because tree-nested.c in cm3cg does some very convoluted stuff with the > way static ancestor activation records are accessed, and I have not been > able to see an easy way to get stabs info to reflect it. > > I have thought for a long time that we really need to go to a version > of dwarf for debug info. stabs is very limited, and there is a lot of > info in the stabs output that is not really stabs, it's just kludgily > coded values crammed inside various fields of stabs that the compiler > and m3gdb both agree on (well, they used to.) > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > changes in where type information is actually developed, so it may not > be worth it. After all the work I have done on this to get m3gdb to where > it was, I am unenthusiastic about a bunch more work just to get back to > where we were. I have long lists of improvements to make to m3gdb, and > it's hard to go back that way. And, It's all in C :-(. > > If I put much effort into fixing m3gdb and its support in cm3cg at this > point, it will probably be a rework to use dwarf. That is likely a big > job too, but at least more rewarding and likely to lead to better m3gdb > function in the end. > > For now, it you want a working m3gdb, you really have to use the release > compiler. > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > Hi m3devel, > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > well (never perfectly, but...). Now I'm trying to debug a program and > > I get the following: > > > > (m3gdb) break Main.m3:65 > > Debug info for file "Main.mc" not in stabs format > > (m3gdb) > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > systems too. > > > > Everything is from the CVS head. > > > > Main was compiled as follows, says cm3 -commands: > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ > ci > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > something broken in the compiler, or what? > > > > I fear m3gdb is essentially useless at the moment. > > > > Mika > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Apr 2 04:42:33 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 01 Apr 2011 19:42:33 -0700 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <20110402024233.0BEC11A2078@async.async.caltech.edu> Jay K writes: ... >But that "custom information" is largely just basic type information=2C tha= >t any decent >C compiler/debug-info/debugger can handle. >Someone should investigate if we really pass stuff through that can't be we= >ll represented >in gcc's type information. >For example: sub ranges? sets? enums? packed? Subranges, sets, and enums certainly used to be shown in a "friendly" way by m3gdb. Mika From jay.krell at cornell.edu Sat Apr 2 05:24:02 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 03:24:02 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <20110402024233.0BEC11A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Apr 2 17:53:19 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 2 Apr 2011 16:53:19 +0100 (BST) Subject: [M3devel] m3gdb broken In-Reply-To: Message-ID: <196546.21262.qm@web29708.mail.ird.yahoo.com> Hi all: well, all that said, is it possible, but how many checks would be needed to all those errors, I perhaps think the best tool at hand is but to run undependently of the platform ESC, at least to catch the most common RT errors for now, but gcc as a backend is enough optimizable and also enough conservative to catch at compile time warnings you see as possible M3CG errors, in Modula-3 safe code there are not explicity needed in ESC terms but in UNSAFE ones, so the most RTError you want to catch as m3gdb exceptions would be advisable in foreseeable future, perhaps making some of them implementable at ESC level, but to check for those that are that in the RTCollector, RTException and the RTThread specially perhaps simpler ones in ESC terms but important, and in the M3CG memory handling, exception, and threads too, obviously in the RTType, even if those are not safely checked RTError should have some correspondent there and that is it must be caught in the M3CG before it can also generate bad codes, if so, also possible improved UNSAFE RTTypeSRC information to accomplish that perhaps a better but not limited to type checking and exceptions handled of both safe and UNSAFE code. Thinking in what's on the market there are recent reports in Logic Analyzer of embedded software-hardware verification environments for circuit analysis, such kind of automation tests are deployable even at the try and so we may get that, although this Embedded software Arena, but even at that level, cost effective knowing of the logic synthesis theories errors they want to be rid of, that is, is more important to shape some circuit level checks before product is deployed since may things depended on that too. We could seemingly adopt that strategic objective too, detect most of the common errors in those areas, also a nice RTTypeSRC interface for unsafe type code patterns. Perhaps some of this work is already done implemented in Modula-3 VM backend Quest abstract machine, since? it has nice constructs and basically same stack architecture and machine but with additional ability to support for separating sound and unsound modules. I think in this way the margin of RT errors will be diminished by at least by some volume bounded by the amount of each type inference in the checker, specially some of the ones in RTTypeSRC information portion plus desired good to watch at least some code modularity level sound handled in such code, so some of them that must be catched and, something it isn't right now for M3CG will not, but just because of correspondent MODULEs so it can be translated sound and caught in better CG compiler time in other representation and then generate code to its RT binary structures perhaps some may want for that too. Also that could be generous in terms of debugging both Quest level and m3gdb source level. Thanks in advance --- El vie, 1/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] m3gdb broken Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 1 de abril, 2011 22:24 The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. ?- Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Apr 3 03:28:18 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:28:18 -0700 Subject: [M3devel] floating point : problem with Extended.m3 Message-ID: <20110403012818.3560A1A2078@async.async.caltech.edu> Hello m3devel (mainly Tony), I believe I've found a problem with float/Common/Extended.m3. Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? That does not seem to me to be a safe assumption... in fact I don't think it's even safe to round up. EXTENDED could (and should) be 10 bytes wide on most machines. (I am aware that it is currently the same as LONGREAL, but...) Mika > UNSAFE MODULE Extended; > > IMPORT Word, Int32; > > PROCEDURE Equal(a, b: T): BOOLEAN = > BEGIN RETURN a = b END Equal; > > > CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); > > TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; > > PROCEDURE Hash(a: T): Word.T = > VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN > FOR i := 0 TO LAST(arr) DO > res := Word.Xor(res, Int32.Hash(arr[i])) > END (* FOR *); > RETURN res > END Hash; > > PROCEDURE Compare(a, b: T): [-1..1] = > BEGIN > IF a < b THEN RETURN -1 > ELSIF a > b THEN RETURN 1 > ELSE RETURN 0 > END (* IF *) > END Compare; From hosking at cs.purdue.edu Sun Apr 3 03:40:02 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 2 Apr 2011 21:40:02 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403012818.3560A1A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: Are you saying it is broken with EXTENDED=LONGREAL? That is the current implementation. This code is heritage. Not sure I understand all the implications. On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > Hello m3devel (mainly Tony), > > I believe I've found a problem with float/Common/Extended.m3. > > Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? > > That does not seem to me to be a safe assumption... in fact I don't > think it's even safe to round up. EXTENDED could (and should) be > 10 bytes wide on most machines. (I am aware that it is currently > the same as LONGREAL, but...) > > Mika > >> UNSAFE MODULE Extended; >> >> IMPORT Word, Int32; >> >> PROCEDURE Equal(a, b: T): BOOLEAN = >> BEGIN RETURN a = b END Equal; >> >> >> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >> >> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >> >> PROCEDURE Hash(a: T): Word.T = >> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >> FOR i := 0 TO LAST(arr) DO >> res := Word.Xor(res, Int32.Hash(arr[i])) >> END (* FOR *); >> RETURN res >> END Hash; >> >> PROCEDURE Compare(a, b: T): [-1..1] = >> BEGIN >> IF a < b THEN RETURN -1 >> ELSIF a > b THEN RETURN 1 >> ELSE RETURN 0 >> END (* IF *) >> END Compare; From mika at async.caltech.edu Sun Apr 3 03:52:08 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:52:08 -0700 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: <20110403015208.6B1511A2078@async.async.caltech.edu> No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 bits). I'm really just curious what has to be changed to make EXTENDED not be LONGREAL... probably many more things in the compiler? Mika Tony Hosking writes: >Are you saying it is broken with EXTENDED=LONGREAL? >That is the current implementation. >This code is heritage. Not sure I understand all the implications. > >On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > >> Hello m3devel (mainly Tony), >> >> I believe I've found a problem with float/Common/Extended.m3. >> >> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >> >> That does not seem to me to be a safe assumption... in fact I don't >> think it's even safe to round up. EXTENDED could (and should) be >> 10 bytes wide on most machines. (I am aware that it is currently >> the same as LONGREAL, but...) >> >> Mika >> >>> UNSAFE MODULE Extended; >>> >>> IMPORT Word, Int32; >>> >>> PROCEDURE Equal(a, b: T): BOOLEAN = >>> BEGIN RETURN a = b END Equal; >>> >>> >>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>> >>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>> >>> PROCEDURE Hash(a: T): Word.T = >>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>> FOR i := 0 TO LAST(arr) DO >>> res := Word.Xor(res, Int32.Hash(arr[i])) >>> END (* FOR *); >>> RETURN res >>> END Hash; >>> >>> PROCEDURE Compare(a, b: T): [-1..1] = >>> BEGIN >>> IF a < b THEN RETURN -1 >>> ELSIF a > b THEN RETURN 1 >>> ELSE RETURN 0 >>> END (* IF *) >>> END Compare; From hosking at cs.purdue.edu Sun Apr 3 22:15:00 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 3 Apr 2011 16:15:00 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403015208.6B1511A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> <20110403015208.6B1511A2078@async.async.caltech.edu> Message-ID: <3523B817-7987-4CA1-8230-0DF99EE76CCE@cs.purdue.edu> On Apr 2, 2011, at 9:52 PM, Mika Nystrom wrote: > No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 > bits). > > I'm really just curious what has to be changed to make EXTENDED not be > LONGREAL... probably many more things in the compiler? Yes, definitely. Compiler and run-time. > > Mika > > Tony Hosking writes: >> Are you saying it is broken with EXTENDED=LONGREAL? >> That is the current implementation. >> This code is heritage. Not sure I understand all the implications. >> >> On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: >> >>> Hello m3devel (mainly Tony), >>> >>> I believe I've found a problem with float/Common/Extended.m3. >>> >>> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >>> >>> That does not seem to me to be a safe assumption... in fact I don't >>> think it's even safe to round up. EXTENDED could (and should) be >>> 10 bytes wide on most machines. (I am aware that it is currently >>> the same as LONGREAL, but...) >>> >>> Mika >>> >>>> UNSAFE MODULE Extended; >>>> >>>> IMPORT Word, Int32; >>>> >>>> PROCEDURE Equal(a, b: T): BOOLEAN = >>>> BEGIN RETURN a = b END Equal; >>>> >>>> >>>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>>> >>>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>>> >>>> PROCEDURE Hash(a: T): Word.T = >>>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>>> FOR i := 0 TO LAST(arr) DO >>>> res := Word.Xor(res, Int32.Hash(arr[i])) >>>> END (* FOR *); >>>> RETURN res >>>> END Hash; >>>> >>>> PROCEDURE Compare(a, b: T): [-1..1] = >>>> BEGIN >>>> IF a < b THEN RETURN -1 >>>> ELSIF a > b THEN RETURN 1 >>>> ELSE RETURN 0 >>>> END (* IF *) >>>> END Compare; From rodney_bates at lcwb.coop Sun Apr 3 22:50:35 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:50:35 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <4D98DD9B.9080003@lcwb.coop> On 04/01/2011 08:26 PM, Jay K wrote: > I really tried not to break anything in the wierdo stubs/m3gdb stuff. > > > The work I have been doing to properly declare types is I believe disabled currently. > > Hmm, I looked at a sample of stabs output for the same program, produced by the release and head compilers. In the head, all of the stabs type information is missing, including the type definitions and the references to them. This may be enough to explain all the breakage in m3gdb I have seen. I see another small change, but it is something that, I think, m3gdb already adapts to, and, as in the release version, was weird anyway. > As types become more correct, things start breaking. > Either because the optimizer is more aggressive, or because more sanity checks > can be done by the compiler, and we don't give it particularly good trees. > > > In most things I've looked it, it's really not any mystery what is wrong with our trees btw. > It's not really generally the fault of gcc being obscure or changing. > Though perhaps to some extent that is true. > Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not > accessible from C source, and therefore bugs in them go undetected. > It's more generally clearly not great stuff in cm3cg. > The main bad thing is that field access is done by just adding to base pointers and casting away. > > > Note that stabs is not supported on Darwin. > Nor as I recall HP-UX/HPPA64. > All the more reason to no do things this way. > > > Note furthermore that discussion here is often confused. > Really no particular debug-info generation code should be cm3cg. > In our part -- m3cg/parse.c > Neither stabs nor dwarf nor coff nor anything else. > > > The right approach is to build an intermediate form with reasonably good types. > The backend handles it from there, whatever debug-info format is requested on the command line. > > > This also enables using stock/current gdb, instead of hacked up/old gdb. > > > I made no visible progress upgrading our fork of gdb. > I'd rather just have it go away. > > > I understand, that custom information is being tunneled through to m3gdb. > But that "custom information" is largely just basic type information, that any decent > C compiler/debug-info/debugger can handle. > Someone should investigate if we really pass stuff through that can't be well represented > in gcc's type information. > For example: sub ranges? sets? enums? packed? > Some higher level description of "methods" (really -- whatever C++ does should suffice; > if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) > > > At some point the new type information was really breaking stuff so I think I disabled it. > The system is kind of incomplete and fragile and the way forward is not always easy > or possible in small steps. :( > > > - Jay > > > > Date: Fri, 1 Apr 2011 12:43:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > > using the Cartesian product of release/head for m3gdb and the compiler. > > > > However, I see a lot of broken stuff that used to work. In all the cases > > I have seen, it correlates to debugging code compiled by the head compiler, > > while the release and head versions of m3gdb make no difference to the > > symptoms in anything I have tried. > > > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > > I am not sure how far that got. Its source code is vastly different, and > > most of the other language support in gdb has been ripped out. > > > > There is quite a lot of change in the stabs info generated by the two > > compiler versions. The existing way for a long time has been that this > > is constructed during the first pass over the input to cm3cg, before > > anything other than initial tree building is done. This is a very > > limited strategy, because cm3cg can change things later, and this will > > not be reflected in the stabs info. > > > > Debugging anything involving static links has been broken for a long time, > > because tree-nested.c in cm3cg does some very convoluted stuff with the > > way static ancestor activation records are accessed, and I have not been > > able to see an easy way to get stabs info to reflect it. > > > > I have thought for a long time that we really need to go to a version > > of dwarf for debug info. stabs is very limited, and there is a lot of > > info in the stabs output that is not really stabs, it's just kludgily > > coded values crammed inside various fields of stabs that the compiler > > and m3gdb both agree on (well, they used to.) > > > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > > changes in where type information is actually developed, so it may not > > be worth it. After all the work I have done on this to get m3gdb to where > > it was, I am unenthusiastic about a bunch more work just to get back to > > where we were. I have long lists of improvements to make to m3gdb, and > > it's hard to go back that way. And, It's all in C :-(. > > > > If I put much effort into fixing m3gdb and its support in cm3cg at this > > point, it will probably be a rework to use dwarf. That is likely a big > > job too, but at least more rewarding and likely to lead to better m3gdb > > function in the end. > > > > For now, it you want a working m3gdb, you really have to use the release > > compiler. > > > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > > Hi m3devel, > > > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > > well (never perfectly, but...). Now I'm trying to debug a program and > > > I get the following: > > > > > > (m3gdb) break Main.m3:65 > > > Debug info for file "Main.mc" not in stabs format > > > (m3gdb) > > > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > > systems too. > > > > > > Everything is from the CVS head. > > > > > > Main was compiled as follows, says cm3 -commands: > > > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika /t/ > > ci > > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > > something broken in the compiler, or what? > > > > > > I fear m3gdb is essentially useless at the moment. > > > > > > Mika > > > From rodney_bates at lcwb.coop Sun Apr 3 22:46:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:46:08 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: <4D98DC90.3090900@lcwb.coop> On 04/01/2011 10:24 PM, Jay K wrote: > The question is, can they be well represented within gcc's usual trees/types/debug-info > for good experience in stock gdb. > Enums for example -- yes, I had them working. > > Subranges -- what should the experience be? > For example, when I use debuggers, I use expression evaluation involving assignment > to set values. Should the debugger catch attempts to assign out of range values? > Or are subranges just integers with a certain number of bits and that is it and that suffices? > I understand the runtime does more precise checks than just bit-width. > > sets -- I can see this being....are they just an integer or an array of integers? > Or higher level? I can see this being poor...unless unless.. gcc maybe already > had good support for Pascal? Maybe Ada has a similar feature? > Ada does actually look like Modula-3 to a large extent! > > Consider also that on Darwin, none of this stuff works at all. > So even sets as integers or arrays of integers would be much better than current. > > One has to read some of the gcc documentation, browse tree.h, tree.def, etc., > learn what is well representable. > > One shouldn't just completely ignore the significant infrastructure and just > tunnel all the information through in a custom way that stock gdb is completely ignorant of. > There is no way stock gdb can possibly provide anything remotely similar to the support it gives other languages, just by providing it with better debug info. Every one of the languages already supported by stock gdb (C, C++, Objective C, Pascal, Ada, Java, Fortran, and Scheme come to mind) have significant amounts of language-specific code, comparable to what m3gdb adds for Modula-3 support. Some categories of language-specific code include: name demanglers, expression parsers, expression evaluators, identifier lookups, characteristics of builtin types, and value printers. You can find anecdotal cases of types, operators, and their syntax that are equivalent to another language, but this is highly incomplete. Value notations also differ. Semantics of same-looking operators can be significantly different. The dot is a good example. Not even Pascal nor Ada are the same as Modula-3. Interpretation of memory is also highly language dependent. There are significant differences in runtime representation of seemingly equivalent types and other stuff. Modula-3's TEXT, for example, is represented unlike any of the other languages. Heap objects have distinct ways of showing allocation status, GC info, open array sizes for differing dimensionalities, and just the plain allocated object type. The method dispatching mechanism is distinct. Actually, this stuff is really compiler-dependent, not just language-dependent. There are significant differences between pm3/friends and cm3, and occasional smaller differences between versions thereof. m3gdb recognizes many of these and adapts. Ditto for some of the differences in thread and GC implementations. Some procedures have hidden parameters that show up in the type info, but do not appear in source code calls. The rules for these are fairly conditional, on both the procedure signature and on the target OS. It is difficult for a debugger user to even know when, where, and of what type these need to be to supply them explicitly to make such a call, and as I recall, in some cases impossible to supply some of the needed values, even with full knowledge. In contrast, m3gdb allows typing procedure and method calls as they would appear in source code, and supplies the hidden parameters as needed. Assigning procedures to variables and parameters and calling through them also have unique runtime representations. As of the gdb the current m3gdb is based on, no stock gdb language handles up-level addressing. Much of this is working in the current m3gdb, or was until recently. Most of what isn't is documented as work to do. > > - Jay > > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > Date: Fri, 1 Apr 2011 19:42:33 -0700 > > From: mika at async.caltech.edu > > > > Jay K writes: > > ... > > >But that "custom information" is largely just basic type information=2C tha= > > >t any decent > > >C compiler/debug-info/debugger can handle. > > >Someone should investigate if we really pass stuff through that can't be we= > > >ll represented > > >in gcc's type information. > > >For example: sub ranges? sets? enums? packed? > > > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > > by m3gdb. > > > > Mika From dragisha at m3w.org Thu Apr 7 16:34:20 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 7 Apr 2011 16:34:20 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Message-ID: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> I am getting this, when my program does a Thread.Fork. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 *** Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... TIA, dd From hosking at cs.purdue.edu Thu Apr 7 17:41:07 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 11:41:07 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> Message-ID: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> OK, sounds like the stability of the system is going downhill. I'm not sure when I'll have time to look at this. Maybe soon, maybe in the summer. I am discouraged at all the reports we have been having about these sorts of problems. On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > I am getting this, when my program does a Thread.Fork. > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > *** > > Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... > > TIA, > dd From dabenavidesd at yahoo.es Thu Apr 7 18:30:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 7 Apr 2011 17:30:54 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <785020.50972.qm@web29715.mail.ird.yahoo.com> Hi all: it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. Thanks in advance PS there is Elego's product wiki article http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System Also related: http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf And also: http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf Perhaps is good to see this issue: http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). --- El jue, 7/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Dragi?a Duri?" > CC: "m3devel" > Fecha: jueves, 7 de abril, 2011 10:41 > OK, sounds like the stability of the > system is going downhill. I'm not sure when I'll have > time to look at this. Maybe soon, maybe in the > summer. I am discouraged at all the reports we have > been having about these sorts of problems. > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > I am getting this, when my program does a > Thread.Fork. > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > *** > > > > Any ideas? It worked... And stopped working... Some > changes were made to openssl libraries and similar... > > > > TIA, > > dd > > From mika at async.caltech.edu Thu Apr 7 21:02:19 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 07 Apr 2011 12:02:19 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <20110407190219.C00E31A2078@async.async.caltech.edu> Remember that for mission-critical applications there's always user threading.. I find that the system (up to and including the CVS head and on both amd64 and i386 flavors of Linux as well as every other OS I have tried) is very very stable with user threading. I'm eager to use pthreads again of course, for all sorts of reasons... Mika Tony Hosking writes: >OK, sounds like the stability of the system is going downhill. I'm not = >sure when I'll have time to look at this. Maybe soon, maybe in the = >summer. I am discouraged at all the reports we have been having about = >these sorts of problems. > >On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: > >> I am getting this, when my program does a Thread.Fork. >>=20 >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >> *** >>=20 >> Any ideas? It worked... And stopped working... Some changes were made = >to openssl libraries and similar... >>=20 >> TIA, >> dd From hosking at cs.purdue.edu Thu Apr 7 22:07:51 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 16:07:51 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110407190219.C00E31A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> Message-ID: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Well, that is reassuring news. If the problems are isolated to threading then they can be resolved easily. On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > > Remember that for mission-critical applications there's always user > threading.. I find that the system (up to and including the CVS head > and on both amd64 and i386 flavors of Linux as well as every other OS I > have tried) is very very stable with user threading. I'm eager to use > pthreads again of course, for all sorts of reasons... > > Mika > > Tony Hosking writes: >> OK, sounds like the stability of the system is going downhill. I'm not = >> sure when I'll have time to look at this. Maybe soon, maybe in the = >> summer. I am discouraged at all the reports we have been having about = >> these sorts of problems. >> >> On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: >> >>> I am getting this, when my program does a Thread.Fork. >>> =20 >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >>> *** >>> =20 >>> Any ideas? It worked... And stopped working... Some changes were made = >> to openssl libraries and similar... >>> =20 >>> TIA, >>> dd From jay.krell at cornell.edu Thu Apr 7 23:25:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 7 Apr 2011 21:25:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <785020.50972.qm@web29715.mail.ird.yahoo.com> References: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <785020.50972.qm@web29715.mail.ird.yahoo.com> Message-ID: Problems with cvsupd are understood and believed fixed. It depended on all threads surviving fork, instead of just the one forking thread. Which is what userthreads did and which is what pthreads never did and never will do. No threads survive fork+exec, which is what most users of fork do. But cvsupd uses the "fork and do more work" pattern. Userthreads have been changed to be like pthreads. Past problems with cvsupd on pthreads should NOT be used to think about problems anywhere else in the system. If cvsupd still doesn't work, then that is possibly relevant to the rest of the system. - Jay > Date: Thu, 7 Apr 2011 17:30:54 +0100 > From: dabenavidesd at yahoo.es > To: dragisha at m3w.org; hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Hi all: > it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? > > After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I > possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). > For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. > > Thanks in advance > > PS there is Elego's product wiki article > http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System > Also related: > http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf > And also: > http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf > > Perhaps is good to see this issue: > http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded > > This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). > > --- El jue, 7/4/11, Tony Hosking escribi?: > > > De: Tony Hosking > > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Para: "Dragi?a Duri?" > > CC: "m3devel" > > Fecha: jueves, 7 de abril, 2011 10:41 > > OK, sounds like the stability of the > > system is going downhill. I'm not sure when I'll have > > time to look at this. Maybe soon, maybe in the > > summer. I am discouraged at all the reports we have > > been having about these sorts of problems. > > > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > > > I am getting this, when my program does a > > Thread.Fork. > > > > > > *** > > > *** runtime error: > > > *** <*ASSERT*> failed. > > > *** file > > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > > *** > > > > > > Any ideas? It worked... And stopped working... Some > > changes were made to openssl libraries and similar... > > > > > > TIA, > > > dd > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 00:45:16 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 10 Apr 2011 15:45:16 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX Message-ID: <20110410224516.E72F41A207A@async.async.caltech.edu> Hello m3devel (especially Jay), I'm having some time conversion problems. I *believe* (a bit hard to verify because m3gdb isn't working!!) that Date.ToTime called on Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. I note the following (fairly recent) code in DatePosixC.c: #ifdef _TIME64_T t = mktime64(&tm); #else t = mktime(&tm); #endif #ifdef DATE_BSD if (t == -1) goto Exit; /* adjust result to reflect "date->offset" */ #ifdef _TIME64_T time64(&now); local_now = localtime64(&now); #else time(&now); local_now = localtime(&now); #endif assert(local_now != NULL); if (local_now->tm_isdst > 0) /* decrement the local time zone by one hour if DST is in effect */ local_now->m3_tm_gmtoff -= SecsPerHour; /* As above, we must negate "date->offset" to account for the opposite sense of that field compared to Unix. */ t -= ((-date->offset) - local_now->m3_tm_gmtoff); Exit: #endif return t; What I don't understand is the subtracting off of SecsPerHour if "now" is DST. Yeah, now is DST (it's April), but the time I'm converting was in the winter, and besides GMT and UTC don't even have DST. I don't understand the intent of the code.. can someone explain why we're subtracting off an hour? And why is it based on the value of tm_isdst of the *current* time? Some additional documentation within the file would be helpful too. Note my conversion returns the correct 0 on various other M3 implementations. Note the system time on my system is in a timezone that has DST (namely, BST) but I have changed the TZ environment variable to UTC or GMT before calling this code. Mika From jay.krell at cornell.edu Mon Apr 11 04:00:35 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 11 Apr 2011 02:00:35 +0000 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110410224516.E72F41A207A@async.async.caltech.edu> References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: This stuff is tricky. Maybe impossible. And requires a fair amount of time and attention. Which I might not have right now, but which I definitely tried to apply at the time. Can you compare with some historical versions? e.g. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=1.1.1.2;content-type=text%2Fplain http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=1.1.1.1;content-type=text%2Fplain and notice: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3 Revision 1.3: download - view: text, markup, annotated - select for diffs Note: This also fixes the UTC timezone on Solaris, Interix, HP-UX, Cygwin, which appear to have been historically broken. I'm usually a bit nervous when I make accusations of long-term breakage. And yeah there is a general problem of determining when daylight savings is in effect. The rules very often change. The operating system needs a lot of data to know this. This stuff should be easy to debug without m3gdb. Just print stuff to stdout liberally. Aha, see here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/Attic/DateBsd.m3?rev=1.1.1.2;content-type=text%2Fplain (* adjust result to reflect "d.offset" *) EVAL Utime.time(ADR(now)); local_now := Utime.localtime(ADR(now)); IF local_now.tm_isdst > 0 THEN (* decrement the local time zone by one hour if DST is in effect *) DEC(local_now.tm_gmtoff, 1 * SecsPerHour) END; - Jay > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Date: Sun, 10 Apr 2011 15:45:16 -0700 > From: mika at async.caltech.edu > Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > > Hello m3devel (especially Jay), > > I'm having some time conversion problems. I *believe* (a bit hard to > verify because m3gdb isn't working!!) that Date.ToTime called on > Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. > > I note the following (fairly recent) code in DatePosixC.c: > > #ifdef _TIME64_T > t = mktime64(&tm); > #else > t = mktime(&tm); > #endif > #ifdef DATE_BSD > if (t == -1) > goto Exit; > > /* adjust result to reflect "date->offset" */ > #ifdef _TIME64_T > time64(&now); > local_now = localtime64(&now); > #else > time(&now); > local_now = localtime(&now); > #endif > assert(local_now != NULL); > if (local_now->tm_isdst > 0) > /* decrement the local time zone by one hour if DST is in effect */ > local_now->m3_tm_gmtoff -= SecsPerHour; > > /* As above, we must negate "date->offset" to account for the > opposite sense of that field compared to Unix. */ > t -= ((-date->offset) - local_now->m3_tm_gmtoff); > Exit: > #endif > return t; > > > What I don't understand is the subtracting off of SecsPerHour if > "now" is DST. Yeah, now is DST (it's April), but the time I'm > converting was in the winter, and besides GMT and UTC don't even > have DST. > > I don't understand the intent of the code.. can someone explain why > we're subtracting off an hour? And why is it based on the value of > tm_isdst of the *current* time? Some additional documentation within > the file would be helpful too. > > Note my conversion returns the correct 0 on various other M3 > implementations. > > Note the system time on my system is in a timezone that has DST > (namely, BST) but I have changed the TZ environment variable to UTC or GMT > before calling this code. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 11:01:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 11 Apr 2011 02:01:37 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: <20110411090137.BEF721A207A@async.async.caltech.edu> I'm not sure about impossible. But maybe... Here's the situation. I'm using unix time in my code. Sometimes it needs to run on Windows. Some versions of Modula-3 on Windows use the Windows epoch for timekeeping. So I first request the time at midnight UTC Jan 1 1970 and set that to "UnixEpoch". On Unix systems I expect it to be zero, on some other systems it might be another number. Now I've found what the problem is, and it's not anything you have done. In fact PM3 exhibits the same problem if you set everything up precisely the same. The issue is that the server I am using is located in the United Kingdom and it has its time zone set to Europe/London. Nowadays the Brits use the following definitions for time: in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 in summer "British Summer Time" abbrev. "BST" = UTC + 3600 However, between October 26, 1968 and October 31, 1971, they used something else, to wit: all year "British Standard Time" abbrev. "BST" = UTC + 3600 Note the confusing abbreviations. British Standard Time (which no longer exists) and British Summer Time have the same abbreviation. You might think this is sensible, because British Summer Time has the same value as British Standard Time (when the former is defined). There is just one problem. British Standard Time is not a "daylight savings time" whereas British Summer Time *is*. Witness the output of zdump for London: Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar 19 03:00:00 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct 29 02:59:59 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct 29 02:00:00 1967 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb 18 01:59:59 1968 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb 18 03:00:00 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct 31 02:00:00 1971 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar 19 01:59:59 1972 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar 19 03:00:00 1972 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct 29 02:59:59 1972 BST isdst=1 gmtoff=3600 The following lines are the special ones: Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Note how the gmtoff stays the same but isdst changes. And this of course straddles the definition of Unix time at Jan 1 1970 00:00:00 UTC. Clearly there is a bug in M3's date code here. I just don't understand what invariant the original author appealed to when he inserted the dst-changing code. It is hardly unusual for a given locale to have changed time zone rules... it is completely wrong for that to affect the conversion of times for a *different* locale (here I am just trying to do a time conversion for UTC, it shouldn't matter what country my computer is in when it does that!) Mika Jay K writes: >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention. >Which I might not have right now=2C but which I definitely tried to apply a= >t the time. > >Can you compare with some historical versions? >e.g. > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > >and notice: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3 >Revision 1.3: download - view: text=2C markup=2C annotated - select for dif= >fs >Note: This also fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cy= >gwin=2C which appear to have been historically broken. > > >I'm usually a bit nervous when I make accusations of long-term breakage. > > >And yeah there is a general problem of determining when daylight savings is= > in effect. >The rules very often change. The operating system needs a lot of data to kn= >ow this. > > >This stuff should be easy to debug without m3gdb. >Just print stuff to stdout liberally. > >Aha=2C see here: >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > (* adjust result to reflect "d.offset" *) > EVAL Utime.time(ADR(now))=3B > local_now :=3D Utime.localtime(ADR(now))=3B > IF local_now.tm_isdst > 0 THEN > (* decrement the local time zone by one hour if DST is in effect *) > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour) > END=3B > > - Jay > >> To: m3devel at elegosoft.com=3B jay.krell at cornell.edu >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 >> From: mika at async.caltech.edu >> Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX >>=20 >> Hello m3devel (especially Jay)=2C >>=20 >> I'm having some time conversion problems. I *believe* (a bit hard to >> verify because m3gdb isn't working!!) that Date.ToTime called on >> Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. >>=20 >> I note the following (fairly recent) code in DatePosixC.c: >>=20 >> #ifdef _TIME64_T >> t =3D mktime64(&tm)=3B >> #else >> t =3D mktime(&tm)=3B >> #endif >> #ifdef DATE_BSD >> if (t =3D=3D -1) >> goto Exit=3B >>=20 >> /* adjust result to reflect "date->offset" */ >> #ifdef _TIME64_T >> time64(&now)=3B >> local_now =3D localtime64(&now)=3B >> #else >> time(&now)=3B >> local_now =3D localtime(&now)=3B >> #endif >> assert(local_now !=3D NULL)=3B >> if (local_now->tm_isdst > 0) >> /* decrement the local time zone by one hour if DST is in effect */ >> local_now->m3_tm_gmtoff -=3D SecsPerHour=3B >>=20 >> /* As above=2C we must negate "date->offset" to account for the >> opposite sense of that field compared to Unix. */ >> t -=3D ((-date->offset) - local_now->m3_tm_gmtoff)=3B >> Exit: >> #endif >> return t=3B >>=20 >>=20 >> What I don't understand is the subtracting off of SecsPerHour if >> "now" is DST. Yeah=2C now is DST (it's April)=2C but the time I'm >> converting was in the winter=2C and besides GMT and UTC don't even >> have DST.=20 >>=20 >> I don't understand the intent of the code.. can someone explain why >> we're subtracting off an hour? And why is it based on the value of=20 >> tm_isdst of the *current* time? Some additional documentation within >> the file would be helpful too. >>=20 >> Note my conversion returns the correct 0 on various other M3 >> implementations. >>=20 >> Note the system time on my system is in a timezone that has DST=20 >> (namely=2C BST) but I have changed the TZ environment variable to UTC or = >GMT >> before calling this code. >>=20 >> Mika > = > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention.
Which I might not have right now=2C but which I definitel= >y tried to apply at the time.

Can you compare with some historical v= >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= >lain


and notice:

http://modula3.elegosoft.com/cgi-bin/cvs= >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision 1.3b>: 7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fplain" class=3D"download-link">download - view: tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" class=3D"display= >-link">text=2C gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fx-cvsweb-markup" class=3D"display-link">markup=2C http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?annotate=3D1.3">annotated - a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: This a= >lso fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cygwin=2C whic= >h appear to have been historically broken.


I'm usually a bit ner= >vous when I make accusations of long-term breakage.


And yeah the= >re is a general problem of determining when daylight savings is in effect.<= >br>The rules very often change. The operating system needs a lot of data to= > know this.


This stuff should be easy to debug without m3gdb.>Just print stuff to stdout liberally.

Aha=2C see here:
http://mo= >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

  (*=
> adjust result to reflect "d.offset" *)
    EVAL Utime.time(ADR(now))=3B=
>
    local_now :=3D Utime.localtime(ADR(now))=3B
    IF local_now.tm_=
>isdst >=3B 0 THEN
      (* decrement the local time zone by one hour i=
>f DST is in effect *)
      DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
>br>    END=3B

 =3B- Jay

>=3B To: m3devel at elegosof= >t.com=3B jay.krell at cornell.edu
>=3B Date: Sun=2C 10 Apr 2011 15:45:16 = >-0700
>=3B From: mika at async.caltech.edu
>=3B Subject: [M3devel] t= >ime conversion problems : Date.ToTime on AMD64_LINUX
>=3B
>=3B H= >ello m3devel (especially Jay)=2C
>=3B
>=3B I'm having some time = >conversion problems. I *believe* (a bit hard to
>=3B verify because m= >3gdb isn't working!!) that Date.ToTime called on
>=3B Jan 1 1970 in ti= >mezone "UTC" or "GMT" is returning -3600 for me.
>=3B
>=3B I not= >e the following (fairly recent) code in DatePosixC.c:
>=3B
>=3B = >#ifdef _TIME64_T
>=3B t =3D mktime64(&=3Btm)=3B
>=3B #else= >
>=3B t =3D mktime(&=3Btm)=3B
>=3B #endif
>=3B #ifdef= > DATE_BSD
>=3B if (t =3D=3D -1)
>=3B goto Exit=3B
= >>=3B
>=3B /* adjust result to reflect "date->=3Boffset" */>>=3B #ifdef _TIME64_T
>=3B time64(&=3Bnow)=3B
>=3B = >local_now =3D localtime64(&=3Bnow)=3B
>=3B #else
>=3B time= >(&=3Bnow)=3B
>=3B local_now =3D localtime(&=3Bnow)=3B
>= >=3B #endif
>=3B assert(local_now !=3D NULL)=3B
>=3B if (l= >ocal_now->=3Btm_isdst >=3B 0)
>=3B /* decrement the local ti= >me zone by one hour if DST is in effect */
>=3B local_now->=3B= >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B
>=3B /* As above=2C we= > must negate "date->=3Boffset" to account for the
>=3B opposi= >te sense of that field compared to Unix. */
>=3B t -=3D ((-date-&g= >t=3Boffset) - local_now->=3Bm3_tm_gmtoff)=3B
>=3B Exit:
>=3B #e= >ndif
>=3B return t=3B
>=3B
>=3B
>=3B What I don't= > understand is the subtracting off of SecsPerHour if
>=3B "now" is DST= >. Yeah=2C now is DST (it's April)=2C but the time I'm
>=3B converting= > was in the winter=2C and besides GMT and UTC don't even
>=3B have DST= >.
>=3B
>=3B I don't understand the intent of the code.. can som= >eone explain why
>=3B we're subtracting off an hour? And why is it ba= >sed on the value of
>=3B tm_isdst of the *current* time? Some additi= >onal documentation within
>=3B the file would be helpful too.
>= >=3B
>=3B Note my conversion returns the correct 0 on various other M3= >
>=3B implementations.
>=3B
>=3B Note the system time on my= > system is in a timezone that has DST
>=3B (namely=2C BST) but I have= > changed the TZ environment variable to UTC or GMT
>=3B before calling= > this code.
>=3B
>=3B Mika
> >= > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- From dabenavidesd at yahoo.es Mon Apr 11 18:28:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 11 Apr 2011 17:28:45 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110411090137.BEF721A207A@async.async.caltech.edu> Message-ID: <926161.28913.qm@web29713.mail.ird.yahoo.com> Hi all: although my understanding of the problem is not that good, I think there are two routes for this abstract way one could try to prove the correctness of some implementation by setting directly instrumention code to try to see for your self dynamically if you may that or what you want to see: http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s This could would run with just 3 missing modules from this sources besides their examples, maybe another time the HP guys allow us to see their works on that: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ This package is missing from public repositories Elego too as I know, though they were in (see bottom links dead as of now, but, ... maybe you can look for yourselves): http://www.cs.arizona.edu/~collberg/Research/Modula-3/ I know some ESC/Modula-3 tries to make some inferencing of invariants, but as I said it dones't mean this is totally computable, im fact in general its isn't, but it would be good to try. Essentially where are the codes for showing that, or are you implementing this as private code? Thanks in advance --- El lun, 11/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 04:01 > I'm not sure about impossible. > But maybe... > > Here's the situation. > > I'm using unix time in my code. Sometimes it needs to > run on Windows. > Some versions of Modula-3 on Windows use the Windows epoch > for timekeeping. > So I first request the time at midnight UTC Jan 1 1970 and > set that to > "UnixEpoch". On Unix systems I expect it to be zero, > on some other systems > it might be another number. > > Now I've found what the problem is, and it's not anything > you have done. > In fact PM3 exhibits the same problem if you set everything > up precisely > the same. > > The issue is that the server I am using is located in the > United Kingdom > and it has its time zone set to Europe/London. > Nowadays the Brits use the > following definitions for time: > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 > in summer "British Summer Time" abbrev. "BST" = UTC + 3600 > > However, between October 26, 1968 and October 31, 1971, > they used something > else, to wit: > > all year "British Standard Time" abbrev. "BST" = UTC + > 3600 > > Note the confusing abbreviations. British Standard > Time (which no longer > exists) and British Summer Time have the same > abbreviation. You might think > this is sensible, because British Summer Time has the same > value as British > Standard Time (when the former is defined). > > There is just one problem. British Standard Time is > not a "daylight > savings time" whereas British Summer Time *is*. > Witness the output of > zdump for London: > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > The following lines are the special ones: > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Note how the gmtoff stays the same but isdst changes. > And this of course > straddles the definition of Unix time at Jan 1 1970 > 00:00:00 UTC. > > Clearly there is a bug in M3's date code here. I just > don't understand what > invariant the original author appealed to when he inserted > the dst-changing code. > It is hardly unusual for a given locale to have changed > time zone rules... > it is completely wrong for that to affect the conversion of > times for a *different* > locale (here I am just trying to do a time conversion for > UTC, it shouldn't > matter what country my computer is in when it does that!) > > Mika > > Jay K writes: > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention. > >Which I might not have right now=2C but which I > definitely tried to apply a= > >t the time. > > > >Can you compare with some historical versions? > >e.g. > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > >and notice: > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3 > >Revision 1.3: download - view: text=2C markup=2C > annotated - select for dif= > >fs > >Note: This also fixes the UTC timezone on Solaris=2C > Interix=2C HP-UX=2C Cy= > >gwin=2C which appear to have been historically broken. > > > > > >I'm usually a bit nervous when I make accusations of > long-term breakage. > > > > > >And yeah there is a general problem of determining when > daylight savings is= > > in effect. > >The rules very often change. The operating system needs > a lot of data to kn= > >ow this. > > > > > >This stuff should be easy to debug without m3gdb. > >Just print stuff to stdout liberally. > > > >Aha=2C see here: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > (* adjust result to reflect "d.offset" *) > > EVAL Utime.time(ADR(now))=3B > > local_now :=3D > Utime.localtime(ADR(now))=3B > > IF local_now.tm_isdst > 0 THEN > > (* decrement the local time zone > by one hour if DST is in effect *) > > DEC(local_now.tm_gmtoff=2C 1 * > SecsPerHour) > > END=3B > > > > - Jay > > > >> To: m3devel at elegosoft.com=3B > jay.krell at cornell.edu > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > >> From: mika at async.caltech.edu > >> Subject: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > >>=20 > >> Hello m3devel (especially Jay)=2C > >>=20 > >> I'm having some time conversion problems. I > *believe* (a bit hard to > >> verify because m3gdb isn't working!!) that > Date.ToTime called on > >> Jan 1 1970 in timezone "UTC" or "GMT" is returning > -3600 for me. > >>=20 > >> I note the following (fairly recent) code in > DatePosixC.c: > >>=20 > >> #ifdef _TIME64_T > >> t =3D > mktime64(&tm)=3B > >> #else > >> t =3D mktime(&tm)=3B > >> #endif > >> #ifdef DATE_BSD > >> if (t =3D=3D -1) > >> goto > Exit=3B > >>=20 > >> /* adjust result to > reflect "date->offset" */ > >> #ifdef _TIME64_T > >> time64(&now)=3B > >> local_now =3D > localtime64(&now)=3B > >> #else > >> time(&now)=3B > >> local_now =3D > localtime(&now)=3B > >> #endif > >> assert(local_now !=3D > NULL)=3B > >> if (local_now->tm_isdst > > 0) > >> /* decrement the > local time zone by one hour if DST is in effect */ > >> > local_now->m3_tm_gmtoff -=3D > SecsPerHour=3B > >>=20 > >> /* As above=2C we must > negate "date->offset" to account for the > >> opposite sense of that > field compared to Unix. */ > >> t -=3D ((-date->offset) > - local_now->m3_tm_gmtoff)=3B > >> Exit: > >> #endif > >> return t=3B > >>=20 > >>=20 > >> What I don't understand is the subtracting off of > SecsPerHour if > >> "now" is DST. Yeah=2C now is DST (it's > April)=2C but the time I'm > >> converting was in the winter=2C and besides GMT > and UTC don't even > >> have DST.=20 > >>=20 > >> I don't understand the intent of the code.. can > someone explain why > >> we're subtracting off an hour? And why is it > based on the value of=20 > >> tm_isdst of the *current* time? Some > additional documentation within > >> the file would be helpful too. > >>=20 > >> Note my conversion returns the correct 0 on > various other M3 > >> implementations. > >>=20 > >> Note the system time on my system is in a timezone > that has DST=20 > >> (namely=2C BST) but I have changed the TZ > environment variable to UTC or = > >GMT > >> before calling this code. > >>=20 > >> Mika > > > > > = > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention.
Which I might not have right > now=2C but which I definitel= > >y tried to apply at the time.

Can > you compare with some historical v= > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > >lain


and > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > 1.3 >b>: >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fplain" > class=3D"download-link">download - view: href=3D"ht= > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > class=3D"display= > >-link">text=2C >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fx-cvsweb-markup" > class=3D"display-link">markup=2C href=3D"= > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > - >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > This a= > >lso fixes the UTC timezone on Solaris=2C Interix=2C > HP-UX=2C Cygwin=2C whic= > >h appear to have been historically > broken.


I'm usually a bit ner= > >vous when I make accusations of long-term > breakage.


And yeah the= > >re is a general problem of determining when daylight > savings is in effect.<= > >br>The rules very often change. The operating system > needs a lot of data to= > > know this.


This stuff > should be easy to debug without m3gdb. >>Just print stuff to stdout > liberally.

Aha=2C see here:
http://mo= > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

 
> (*=
> > adjust result to reflect "d.offset" *)
 
>   EVAL Utime.time(ADR(now))=3B=
> >
    local_now :=3D
> Utime.localtime(ADR(now))=3B
    IF
> local_now.tm_=
> >isdst >=3B 0 THEN
     
> (* decrement the local time zone by one hour i=
> >f DST is in effect *)
     
> DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> >br>   
> END=3B

 =3B- > Jay

>=3B To: m3devel at elegosof= > >t.com=3B jay.krell at cornell.edu
>=3B > Date: Sun=2C 10 Apr 2011 15:45:16 = > >-0700
>=3B From: mika at async.caltech.edu
>=3B > Subject: [M3devel] t= > >ime conversion problems : Date.ToTime on > AMD64_LINUX
>=3B
>=3B H= > >ello m3devel (especially Jay)=2C
>=3B >
>=3B I'm having some time = > >conversion problems. I *believe* (a bit hard > to
>=3B verify because m= > >3gdb isn't working!!) that Date.ToTime called > on
>=3B Jan 1 1970 in ti= > >mezone "UTC" or "GMT" is returning -3600 for > me.
>=3B
>=3B I not= > >e the following (fairly recent) code in > DatePosixC.c:
>=3B
>=3B = > >#ifdef _TIME64_T
>=3B > t =3D > mktime64(&=3Btm)=3B
>=3B #else= > >
>=3B t =3D > mktime(&=3Btm)=3B
>=3B > #endif
>=3B #ifdef= > > DATE_BSD
>=3B > if (t =3D=3D -1)
>=3B > goto Exit=3B
= > >>=3B
>=3B > /* adjust result to reflect > "date->=3Boffset" */ >>>=3B #ifdef > _TIME64_T
>=3B > time64(&=3Bnow)=3B
>=3B > = > >local_now =3D > localtime64(&=3Bnow)=3B
>=3B > #else
>=3B time= > >(&=3Bnow)=3B
>=3B > local_now =3D > localtime(&=3Bnow)=3B
>= > >=3B #endif
>=3B > assert(local_now !=3D > NULL)=3B
>=3B if (l= > >ocal_now->=3Btm_isdst >=3B > 0)
>=3B /* > decrement the local ti= > >me zone by one hour if DST is in effect > */
>=3B > local_now->=3B= > >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B >
>=3B /* As above=2C > we= > > must negate "date->=3Boffset" to account for > the
>=3B opposi= > >te sense of that field compared to Unix. > */
>=3B t -=3D > ((-date-&g= > >t=3Boffset) - > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > Exit:
>=3B #e= > >ndif
>=3B return > t=3B
>=3B
>=3B >
>=3B What I don't= > > understand is the subtracting off of SecsPerHour > if
>=3B "now" is DST= > >. Yeah=2C now is DST (it's April)=2C but the time > I'm
>=3B converting= > > was in the winter=2C and besides GMT and UTC don't > even
>=3B have DST= > >.
>=3B
>=3B I don't > understand the intent of the code.. can som= > >eone explain why
>=3B we're subtracting > off an hour? And why is it ba= > >sed on the value of
>=3B tm_isdst of > the *current* time? Some additi= > >onal documentation within
>=3B the file > would be helpful too.
>= > >=3B
>=3B Note my conversion returns the > correct 0 on various other M3= > >
>=3B > implementations.
>=3B
>=3B > Note the system time on my= > > system is in a timezone that has DST >
>=3B (namely=2C BST) but I have= > > changed the TZ environment variable to UTC or > GMT
>=3B before calling= > > this code.
>=3B >
>=3B Mika
> > > > > > >= > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > From dabenavidesd at yahoo.es Wed Apr 13 17:14:51 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 13 Apr 2011 16:14:51 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <171917.14906.qm@web29716.mail.ird.yahoo.com> Hi all: it would nice to ask HP whether they have decommissioned algorithm animation examples in Zeus3D, to get their sources, since there hasn't been much advances in this area even without Modula-3 on the Scene (let's say of my knowledge there has not been too much attention to hold animation festivals regularly, etc), I just wonder what is about their example implementations: ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-110a.pdf The ones seen in first video report at least, the heap sort: http://replay.waybackmachine.org/20051217123443/http://www.research.compaq.com/SRC/zeus/heapsort3D.html http://replay.waybackmachine.org/20051101010009/http://research.compaq.com/SRC/zeus/sorting3D.html I think 234 trees are missing screenshots (binary three search) though redblack trees appears: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/redblack3D.html The same example for computational geometry kd-tree: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/2dtree3D.html And the last shortest path algorithm example: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/shortestpath3D.html The ones, decommissioned ones I read in initial paragraph here: http://www.internotesstrategy.com/AlgorithMain.htm The most interesting ones in Modula2+ if so are the parallel sorting algorithms, but who might know where are they? Thanks in advance --- El lun, 11/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" , "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 11:28 > Hi all: > although my understanding of the problem is not that good, > I think there are two routes for this abstract way one could > try to prove the correctness of some implementation by > setting directly instrumention code to try to see for your > self dynamically if you may that or what you want to see: > http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s > > This could would run with just 3 missing modules from this > sources besides their examples, maybe another time the HP > guys allow us to see their works on that: > http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ > > This package is missing from public repositories Elego too > as I know, though they were in (see bottom links dead as of > now, but, ... maybe you can look for yourselves): > http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > > I know some ESC/Modula-3 tries to make some inferencing of > invariants, but as I said it dones't mean this is totally > computable, im fact in general its isn't, but it would be > good to try. > > Essentially where are the codes for showing that, or are > you implementing this as private code? > > Thanks in advance > > --- El lun, 11/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > > Para: "Jay K" > > CC: m3devel at elegosoft.com > > Fecha: lunes, 11 de abril, 2011 04:01 > > I'm not sure about impossible. > > But maybe... > > > > Here's the situation. > > > > I'm using unix time in my code. Sometimes it > needs to > > run on Windows. > > Some versions of Modula-3 on Windows use the Windows > epoch > > for timekeeping. > > So I first request the time at midnight UTC Jan 1 1970 > and > > set that to > > "UnixEpoch". On Unix systems I expect it to be > zero, > > on some other systems > > it might be another number. > > > > Now I've found what the problem is, and it's not > anything > > you have done. > > In fact PM3 exhibits the same problem if you set > everything > > up precisely > > the same. > > > > The issue is that the server I am using is located in > the > > United Kingdom > > and it has its time zone set to Europe/London. > > Nowadays the Brits use the > > following definitions for time: > > > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + > 0 > > in summer "British Summer Time" abbrev. "BST" = UTC + > 3600 > > > > However, between October 26, 1968 and October 31, > 1971, > > they used something > > else, to wit: > > > > all year "British Standard Time" abbrev. "BST" = UTC > + > > 3600 > > > > Note the confusing abbreviations. British > Standard > > Time (which no longer > > exists) and British Summer Time have the same > > abbreviation. You might think > > this is sensible, because British Summer Time has the > same > > value as British > > Standard Time (when the former is defined). > > > > There is just one problem. British Standard Time > is > > not a "daylight > > savings time" whereas British Summer Time *is*. > > Witness the output of > > zdump for London: > > > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun > Mar > > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun > Oct > > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun > Oct > > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun > Feb > > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun > Feb > > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun > Oct > > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun > Mar > > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun > Mar > > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun > Oct > > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > > > The following lines are the special ones: > > > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > > > Note how the gmtoff stays the same but isdst changes. > > > And this of course > > straddles the definition of Unix time at Jan 1 1970 > > 00:00:00 UTC. > > > > Clearly there is a bug in M3's date code here. I > just > > don't understand what > > invariant the original author appealed to when he > inserted > > the dst-changing code. > > It is hardly unusual for a given locale to have > changed > > time zone rules... > > it is completely wrong for that to affect the > conversion of > > times for a *different* > > locale (here I am just trying to do a time conversion > for > > UTC, it shouldn't > > matter what country my computer is in when it does > that!) > > > > Mika > > > > Jay K writes: > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/plain; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention. > > >Which I might not have right now=2C but which I > > definitely tried to apply a= > > >t the time. > > > > > >Can you compare with some historical versions? > > >e.g. > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > > > > >and notice: > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > >/POSIX/TimePosix.m3 > > >Revision 1.3: download - view: text=2C markup=2C > > annotated - select for dif= > > >fs > > >Note: This also fixes the UTC timezone on > Solaris=2C > > Interix=2C HP-UX=2C Cy= > > >gwin=2C which appear to have been historically > broken. > > > > > > > > >I'm usually a bit nervous when I make accusations > of > > long-term breakage. > > > > > > > > >And yeah there is a general problem of determining > when > > daylight savings is= > > > in effect. > > >The rules very often change. The operating system > needs > > a lot of data to kn= > > >ow this. > > > > > > > > >This stuff should be easy to debug without m3gdb. > > >Just print stuff to stdout liberally. > > > > > >Aha=2C see here: > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > > > (* adjust result to reflect "d.offset" *) > > > EVAL Utime.time(ADR(now))=3B > > > local_now :=3D > > Utime.localtime(ADR(now))=3B > > > IF local_now.tm_isdst > 0 THEN > > > (* decrement the local time > zone > > by one hour if DST is in effect *) > > > DEC(local_now.tm_gmtoff=2C 1 > * > > SecsPerHour) > > > END=3B > > > > > > - Jay > > > > > >> To: m3devel at elegosoft.com=3B > > jay.krell at cornell.edu > > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > > >> From: mika at async.caltech.edu > > >> Subject: [M3devel] time conversion problems > : > > Date.ToTime on AMD64_LINUX > > >>=20 > > >> Hello m3devel (especially Jay)=2C > > >>=20 > > >> I'm having some time conversion > problems. I > > *believe* (a bit hard to > > >> verify because m3gdb isn't working!!) that > > Date.ToTime called on > > >> Jan 1 1970 in timezone "UTC" or "GMT" is > returning > > -3600 for me. > > >>=20 > > >> I note the following (fairly recent) code in > > DatePosixC.c: > > >>=20 > > >> #ifdef _TIME64_T > > >> t =3D > > mktime64(&tm)=3B > > >> #else > > >> t =3D > mktime(&tm)=3B > > >> #endif > > >> #ifdef DATE_BSD > > >> if (t =3D=3D -1) > > >> goto > > Exit=3B > > >>=20 > > >> /* adjust result to > > reflect "date->offset" */ > > >> #ifdef _TIME64_T > > >> time64(&now)=3B > > >> local_now =3D > > localtime64(&now)=3B > > >> #else > > >> time(&now)=3B > > >> local_now =3D > > localtime(&now)=3B > > >> #endif > > >> assert(local_now > !=3D > > NULL)=3B > > >> if > (local_now->tm_isdst > > > 0) > > >> /* decrement > the > > local time zone by one hour if DST is in effect */ > > >> > > local_now->m3_tm_gmtoff -=3D > > SecsPerHour=3B > > >>=20 > > >> /* As above=2C we > must > > negate "date->offset" to account for the > > >> opposite sense of > that > > field compared to Unix. */ > > >> t -=3D > ((-date->offset) > > - local_now->m3_tm_gmtoff)=3B > > >> Exit: > > >> #endif > > >> return t=3B > > >>=20 > > >>=20 > > >> What I don't understand is the subtracting > off of > > SecsPerHour if > > >> "now" is DST. Yeah=2C now is DST (it's > > April)=2C but the time I'm > > >> converting was in the winter=2C and besides > GMT > > and UTC don't even > > >> have DST.=20 > > >>=20 > > >> I don't understand the intent of the code.. > can > > someone explain why > > >> we're subtracting off an hour? And why > is it > > based on the value of=20 > > >> tm_isdst of the *current* time? Some > > additional documentation within > > >> the file would be helpful too. > > >>=20 > > >> Note my conversion returns the correct 0 on > > various other M3 > > >> implementations. > > >>=20 > > >> Note the system time on my system is in a > timezone > > that has DST=20 > > >> (namely=2C BST) but I have changed the TZ > > environment variable to UTC or = > > >GMT > > >> before calling this code. > > >>=20 > > >> Mika > > > > > > > > > = > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/html; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention.
Which I might not have > right > > now=2C but which I definitel= > > >y tried to apply at the > time.

Can > > you compare with some historical v= > > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > > > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > > > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > > >lain


and > > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > > > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > > 1.3 > >b>: > > >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fplain" > > class=3D"download-link">download - view: > > href=3D"ht= > > > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > > > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > > class=3D"display= > > >-link">text=2C > > >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fx-cvsweb-markup" > > class=3D"display-link">markup=2C > href=3D"= > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > > - > > >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > > > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > > This a= > > >lso fixes the UTC timezone on Solaris=2C > Interix=2C > > HP-UX=2C Cygwin=2C whic= > > >h appear to have been historically > > broken.


I'm usually a bit > ner= > > >vous when I make accusations of long-term > > breakage.


And yeah the= > > >re is a general problem of determining when > daylight > > savings is in effect.<= > > >br>The rules very often change. The operating > system > > needs a lot of data to= > > > know this.


This > stuff > > should be easy to debug without m3gdb. > >>Just print stuff to stdout > > liberally.

Aha=2C see > here:
http://mo= > > > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > > > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

> 
> > (*=
> > > adjust result to reflect "d.offset" *)

> 
> >   EVAL Utime.time(ADR(now))=3B=
> > >
    local_now :=3D
> > Utime.localtime(ADR(now))=3B
   
> IF
> > local_now.tm_=
> > >isdst >=3B 0 THEN
 
>    
> > (* decrement the local time zone by one hour i=
> > >f DST is in effect *)
 
>    
> > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> > >br>   
> > END=3B

 =3B- > > Jay

>=3B To: > m3devel at elegosof= > > >t.com=3B jay.krell at cornell.edu
>=3B > > Date: Sun=2C 10 Apr 2011 15:45:16 = > > >-0700
>=3B From: mika at async.caltech.edu
>=3B > > Subject: [M3devel] t= > > >ime conversion problems : Date.ToTime on > > AMD64_LINUX
>=3B
>=3B > H= > > >ello m3devel (especially > Jay)=2C
>=3B > >
>=3B I'm having some time = > > >conversion problems. I *believe* (a bit > hard > > to
>=3B verify because m= > > >3gdb isn't working!!) that Date.ToTime called > > on
>=3B Jan 1 1970 in ti= > > >mezone "UTC" or "GMT" is returning -3600 for > > me.
>=3B
>=3B I not= > > >e the following (fairly recent) code in > > DatePosixC.c:
>=3B
>=3B > = > > >#ifdef _TIME64_T
>=3B > > t =3D > > mktime64(&=3Btm)=3B
>=3B #else= > > >
>=3B t > =3D > > mktime(&=3Btm)=3B
>=3B > > #endif
>=3B #ifdef= > > > DATE_BSD
>=3B > > if (t =3D=3D -1)
>=3B > > goto Exit=3B
= > > >>=3B
>=3B > > /* adjust result to reflect > > "date->=3Boffset" */ > >>>=3B #ifdef > > _TIME64_T
>=3B > > > time64(&=3Bnow)=3B
>=3B > > = > > >local_now =3D > > localtime64(&=3Bnow)=3B
>=3B > > #else
>=3B > time= > > >(&=3Bnow)=3B
>=3B > > local_now =3D > > localtime(&=3Bnow)=3B
>= > > >=3B #endif
>=3B > > assert(local_now !=3D > > NULL)=3B
>=3B > if (l= > > >ocal_now->=3Btm_isdst >=3B > > 0)
>=3B > /* > > decrement the local ti= > > >me zone by one hour if DST is in effect > > */
>=3B > > local_now->=3B= > > >m3_tm_gmtoff -=3D > SecsPerHour=3B
>=3B > >
>=3B /* As > above=2C > > we= > > > must negate "date->=3Boffset" to account > for > > the
>=3B > opposi= > > >te sense of that field compared to Unix. > > */
>=3B t -=3D > > ((-date-&g= > > >t=3Boffset) - > > > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > > Exit:
>=3B #e= > > >ndif
>=3B > return > > t=3B
>=3B
>=3B > >
>=3B What I don't= > > > understand is the subtracting off of SecsPerHour > > if
>=3B "now" is DST= > > >. Yeah=2C now is DST (it's April)=2C but the > time > > I'm
>=3B converting= > > > was in the winter=2C and besides GMT and UTC > don't > > even
>=3B have DST= > > >.
>=3B
>=3B I > don't > > understand the intent of the code.. can som= > > >eone explain why
>=3B we're > subtracting > > off an hour? And why is it ba= > > >sed on the value of
>=3B tm_isdst > of > > the *current* time? Some additi= > > >onal documentation within
>=3B the > file > > would be helpful too.
>= > > >=3B
>=3B Note my conversion > returns the > > correct 0 on various other M3= > > >
>=3B > > implementations.
>=3B >
>=3B > > Note the system time on my= > > > system is in a timezone that has DST > >
>=3B (namely=2C BST) but I have= > > > changed the TZ environment variable to UTC or > > GMT
>=3B before calling= > > > this code.
>=3B > >
>=3B > Mika
> > > > > > > > > > > >= > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > > > From rodney_bates at lcwb.coop Thu Apr 14 17:45:06 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 14 Apr 2011 10:45:06 -0500 Subject: [M3devel] Thread local storage? Message-ID: <4DA71682.9040709@lcwb.coop> I have never understood what "thread local storage" is all about, in particular, what need it addresses. All local variables and parameters are already private to a thread, as consequence of semantics going back as far as Algol-60. They are allocated space when and only when a call is executed. That can't happen until there is a thread to execute the call and it has to have its own private stack where the space is allocated, or it wouldn't work as a thread, (OK leaving out languages like very old Fortrans that don't support recursion either.) That leaves global variables. At least some of these have to be shared by all the threads of a process, or the threads could have no shared memory interaction, and thus no function that would not be served by just separate processes running separate main programs. If you need variables that are private to a thread but in some sense global, just put them in the Thread.Closure. They would hardly make sense as declared lexically global variables anyway, since, except for the initial thread provided by the OS/RTS, their lifetime does not cover the entire execution of the main program. So what are they really, and what are they for? And do we even have or use them in Modula-3? It is looking to me like some sort of accommodation for them in cm3cg may be what has undermined m3gdb in the CVS head. Rodney Bates From hosking at cs.purdue.edu Thu Apr 14 23:05:27 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 17:05:27 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: <4DA71682.9040709@lcwb.coop> References: <4DA71682.9040709@lcwb.coop> Message-ID: I think that these were introduced into the C libraries by Jay. On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > I have never understood what "thread local storage" is all about, in particular, > what need it addresses. > > All local variables and parameters are already private to a thread, as consequence > of semantics going back as far as Algol-60. They are allocated space when and > only when a call is executed. That can't happen until there is a thread to > execute the call and it has to have its own private stack where the space is > allocated, or it wouldn't work as a thread, (OK leaving out languages like > very old Fortrans that don't support recursion either.) > > That leaves global variables. At least some of these have to be shared by all > the threads of a process, or the threads could have no shared memory interaction, > and thus no function that would not be served by just separate processes running > separate main programs. > > If you need variables that are private to a thread but in some sense global, > just put them in the Thread.Closure. They would hardly make sense as declared > lexically global variables anyway, since, except for the initial thread provided > by the OS/RTS, their lifetime does not cover the entire execution of the main > program. > > So what are they really, and what are they for? And do we even have or use them in > Modula-3? > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > undermined m3gdb in the CVS head. > > Rodney Bates From jay.krell at cornell.edu Thu Apr 14 23:27:10 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:27:10 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, Message-ID: [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 14 23:32:22 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:32:22 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Apr 15 02:28:12 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 20:28:12 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend. I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > > We've always had thread locals > > ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? > > - Jay > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > Date: Thu, 14 Apr 2011 21:27:10 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > [I wrote a long answer here but it seemed to have gotten lost.] > > We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. > On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. > There is nothing in cm3cg about this. > > http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c > Revision 1.155: download - view: text, markup, annotated - select for diffs > Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell > Branches: MAIN > Diff to: previous 1.154: preferred, unified > Changes since revision 1.154: +2 -1 lines > Don't use __thread on Solaris. > It failed to link. > See: > http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console > Revision 1.151: download - view: text, markup, annotated - select for diffs > Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell > Branches: MAIN > Diff to: previous 1.150: preferred, unified > Changes since revision 1.150: +30 -0 lines > Use __thread on Linux and Solaris. > Doesn't compile on Apple. > Segfaults on OpenBSD and NetBSD. > Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. > In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker > or C++ exceptions (C++ backend). > > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Thu, 14 Apr 2011 17:05:27 -0400 > > To: rodney_bates at lcwb.coop > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Thread local storage? > > > > I think that these were introduced into the C libraries by Jay. > > > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > > > I have never understood what "thread local storage" is all about, in particular, > > > what need it addresses. > > > > > > All local variables and parameters are already private to a thread, as consequence > > > of semantics going back as far as Algol-60. They are allocated space when and > > > only when a call is executed. That can't happen until there is a thread to > > > execute the call and it has to have its own private stack where the space is > > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > > very old Fortrans that don't support recursion either.) > > > > > > That leaves global variables. At least some of these have to be shared by all > > > the threads of a process, or the threads could have no shared memory interaction, > > > and thus no function that would not be served by just separate processes running > > > separate main programs. > > > > > > If you need variables that are private to a thread but in some sense global, > > > just put them in the Thread.Closure. They would hardly make sense as declared > > > lexically global variables anyway, since, except for the initial thread provided > > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > > program. > > > > > > So what are they really, and what are they for? And do we even have or use them in > > > Modula-3? > > > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > > undermined m3gdb in the CVS head. > > > > > > Rodney Bates > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 15 04:46:46 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 15 Apr 2011 02:46:46 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , , , , , Message-ID: __thread is only used in C code. The difference is in the gcc backend within the gcc C/C++ compiler. Not cm3cg. I put in using __thread for Linux. There is some potential for using it elsewhere, but that's all I did. e.g. Solaris sometimes, AIX. All other non-Windows targets use pthread_getspecific/setspecific. It is hidden from any Modula-3 code. It is all in C. m3gdb might might might need to know about it. It could also be that m3gdb gets confused by any C code linked in. Or that m3gdb is old (it is) and is confused by __thread (but again, m3gdb might or might not have a builtin need to often look at these, or maybe just the human debugger wants to see them). Currently no targets support stack walking. I couldn't get SOLgnu/SOLsun's stack walker to work with the gcc 4.5 backend. I really really tried. I tried not optimizing at all. And/or putting volatile everywhere. Tru64 also didn't work despite a bit of an attempt. I probably also tried not optimizing there. I probably didn't try as much as for SOLgnu/SOLsun due to the machine being slower and the platform being in less use. - Jay From: hosking at cs.purdue.edu Date: Thu, 14 Apr 2011 20:28:12 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend.I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAOffice +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 15 12:37:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:37:37 -0700 Subject: [M3devel] malloc/free from Modula-3? Message-ID: <20110415103737.E73B31A2078@async.async.caltech.edu> Hi m3devel, I'm debugging a problem with some code I have that calls out to C code. It's calling pqlib (PostgreSQL library) and you might expect things to go wrong there, but as it turns out it's dying in CM3's code. The system is AMD64_LINUX (user threading because of the problems with pthreads) and the program is just freezing up hard. No segfault, bus error or anything. It locks up a couple of times a day. What are the rules on calling malloc/free from Modula-3? Stack trace: (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) Here is the code that causes the problem: VAR s := CopyTtoS(query); BEGIN TRY t.res := t.myExec(s,busyWait) FINALLY FreeCopiedS(s) (* stuck here *) END END; The query is a SQL query---t.myExec eventually calls the libpq SQL exec routine (PQexec I think it's called). Since this is already an unsafe module I suppose I could just make an array of CHARs on the stack and pass a pointer to that... Regards, Mika From mika at async.caltech.edu Fri Apr 15 12:46:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:46:47 -0700 Subject: [M3devel] malloc/free from Modula-3 (more details) Message-ID: <20110415104647.6AFF81A2078@async.async.caltech.edu> Here is some more info. Note that all my access to the pq library is inside a single giant lock (I've run this code for years and years but not under CM3 and not under AMD64_LINUX or for that matter any kind of Linux). The locking is not likely to be obviously wrong, but of course there could be something subtly wrong. In gdb the program is completely frozen: (gdb) stepi ^C Program received signal SIGINT, Interrupt. 0x00002ac519a9202e in ?? () from /lib/libc.so.6 (gdb) info registers rax 0xfffffffffffffe00 -512 rbx 0x2ac519d059e0 47026030008800 rcx 0xffffffffffffffff -1 rdx 0x2 2 rsi 0x80 128 rdi 0x2ac519d059e0 47026030008800 rbp 0x2ac63b0ef0e0 0x2ac63b0ef0e0 rsp 0x2ac63b0ef028 0x2ac63b0ef028 r8 0x2fe2620 50210336 r9 0x2ac6400008ef 47030965635311 r10 0x0 0 r11 0x302 770 r12 0x411858 4266072 r13 0x1 1 r14 0x2f5b6d0 49657552 r15 0x0 0 rip 0x2ac519a9202e 0x2ac519a9202e eflags 0x202 [ IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 fctrl 0x37f 895 fstat 0x0 0 ftag 0xffff 65535 fiseg 0x0 0 fioff 0x0 0 foseg 0x0 0 fooff 0x0 0 fop 0x0 0 mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ] (gdb) (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) disassemble 0x00002ac519a9202e No function contains specified address. (gdb) disassemble 0x00002ac519a2caab Dump of assembler code for function free: 0x00002ac519a2ca40 : mov 0x2d7519(%rip),%rax # 0x2ac519d03f60 0x00002ac519a2ca47 : push %rbx 0x00002ac519a2ca48 : mov %rdi,%r8 0x00002ac519a2ca4b : mov (%rax),%rax 0x00002ac519a2ca4e : test %rax,%rax 0x00002ac519a2ca51 : je 0x2ac519a2ca60 0x00002ac519a2ca53 : mov 0x8(%rsp),%rsi 0x00002ac519a2ca58 : mov %rax,%r11 0x00002ac519a2ca5b : pop %rbx 0x00002ac519a2ca5c : jmpq *%r11 0x00002ac519a2ca5f : nop 0x00002ac519a2ca60 : test %rdi,%rdi 0x00002ac519a2ca63 : je 0x2ac519a2cad2 0x00002ac519a2ca65 : mov -0x8(%rdi),%rax 0x00002ac519a2ca69 : lea -0x10(%rdi),%rdx 0x00002ac519a2ca6d : test $0x2,%al 0x00002ac519a2ca6f : jne 0x2ac519a2cad8 0x00002ac519a2ca71 : test $0x4,%al 0x00002ac519a2ca73 : lea 0x2d8f66(%rip),%rbx # 0x2ac519d059e0 0x00002ac519a2ca7a : je 0x2ac519a2ca86 0x00002ac519a2ca7c : and $0xfffffffffc000000,%rdx 0x00002ac519a2ca83 : mov (%rdx),%rbx 0x00002ac519a2ca86 : mov $0x1,%esi 0x00002ac519a2ca8b : xor %eax,%eax 0x00002ac519a2ca8d : cmpl $0x0,0x2dc594(%rip) # 0x2ac519d09028 0x00002ac519a2ca94 : je 0x2ac519a2caa2 0x00002ac519a2ca96 : lock cmpxchg %esi,(%rbx) 0x00002ac519a2ca9a : jne 0x2ac519a2fb34 0x00002ac519a2caa0 : jmp 0x2ac519a2caab 0x00002ac519a2caa2 : cmpxchg %esi,(%rbx) 0x00002ac519a2caa5 : jne 0x2ac519a2fb34 0x00002ac519a2caab : mov %r8,%rsi 0x00002ac519a2caae : mov %rbx,%rdi 0x00002ac519a2cab1 : callq 0x2ac519a2c2b0 0x00002ac519a2cab6 : cmpl $0x0,0x2dc56b(%rip) # 0x2ac519d09028 0x00002ac519a2cabd : je 0x2ac519a2caca 0x00002ac519a2cabf : lock decl (%rbx) 0x00002ac519a2cac2 : jne 0x2ac519a2fb4f 0x00002ac519a2cac8 : jmp 0x2ac519a2cad2 0x00002ac519a2caca : decl (%rbx) 0x00002ac519a2cacc : jne 0x2ac519a2fb4f 0x00002ac519a2cad2 : pop %rbx 0x00002ac519a2cad3 : retq 0x00002ac519a2cad4 : nopl 0x0(%rax) 0x00002ac519a2cad8 : mov 0x2d97c6(%rip),%ecx # 0x2ac519d062a4 0x00002ac519a2cade : test %ecx,%ecx 0x00002ac519a2cae0 : jne 0x2ac519a2cb10 0x00002ac519a2cae2 : cmp 0x2d97a7(%rip),%rax # 0x2ac519d06290 0x00002ac519a2cae9 : jbe 0x2ac519a2cb10 0x00002ac519a2caeb : cmp $0x2000000,%rax 0x00002ac519a2caf1 : ja 0x2ac519a2cb10 0x00002ac519a2caf3 : and $0xfffffffffffffff8,%rax 0x00002ac519a2caf7 : mov %rax,0x2d9792(%rip) # 0x2ac519d06290 0x00002ac519a2cafe : add %rax,%rax 0x00002ac519a2cb01 : mov %rax,0x2d9778(%rip) # 0x2ac519d06280 0x00002ac519a2cb08 : nopl 0x0(%rax,%rax,1) ---Type to continue, or q to quit--- 0x00002ac519a2cb10 : pop %rbx 0x00002ac519a2cb11 : mov %rdx,%rdi 0x00002ac519a2cb14 : jmpq 0x2ac519a2bc40 End of assembler dump. Mika From hosking at cs.purdue.edu Fri Apr 15 17:26:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Apr 2011 11:26:15 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110415103737.E73B31A2078@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > Hi m3devel, > > I'm debugging a problem with some code I have that calls out to C > code. It's calling pqlib (PostgreSQL library) and you might expect > things to go wrong there, but as it turns out it's dying in CM3's code. > > The system is AMD64_LINUX (user threading because of the problems with > pthreads) and the program is just freezing up hard. No segfault, > bus error or anything. It locks up a couple of times a day. > > What are the rules on calling malloc/free from Modula-3? > > Stack trace: > > (gdb) where > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > ) at ../src/C/Common/CstdlibC.c:26 > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > ) at ../src/C/Common/M3toC.m3:46 > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > ) at ../src/UnsafeDatabase.m3:230 > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > ) at ../src/UnsafeDatabase.m3:228 > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > ) at ../src/UnsafeDatabase.m3:258 > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > ) at ../src/DesynchronizedDB.m3:126 > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > ) at ../src/DesynchronizedDB.m3:293 > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > ) at ../src/GCOMSOrderManager.m3:596 > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > ) at ../src/GCOMSOrderManager.m3:310 > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > ) at ../src/EuroDriver.m3:1108 > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > ) at ../src/EuroDriver.m3:1545 > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > #17 0x0000000000000000 in ?? () > (gdb) > > Here is the code that causes the problem: > > VAR > s := CopyTtoS(query); > BEGIN > TRY > t.res := t.myExec(s,busyWait) > FINALLY > FreeCopiedS(s) (* stuck here *) > END > END; > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > routine (PQexec I think it's called). > > Since this is already an unsafe module I suppose I could just make an > array of CHARs on the stack and pass a pointer to that... > > > Regards, > Mika From mika at async.caltech.edu Fri Apr 15 21:03:49 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 12:03:49 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> References: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <20110415190349.E4F531A2078@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >although my understanding of the problem is not that good, I think there ar= >e two routes for this abstract way one could try to prove the correctness o= >f some implementation by setting directly instrumention code to try to see = >for your self dynamically if you may that or what you want to see: >http://www.youtube.com/watch?v=3DzIgu9q0vVc0#t=3D2m06s > >This could would run with just 3 missing modules from this sources besides = >their examples, maybe another time the HP guys allow us to see their works = >on that: >http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D= >/ > >This package is missing from public repositories Elego too as I know, thoug= >h they were in (see bottom links dead as of now, but, ... maybe you can loo= >k for yourselves): >http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > >I know some ESC/Modula-3 tries to make some inferencing of invariants, but = >as I said it dones't mean this is totally computable, im fact in general it= >s isn't, but it would be good to try. > >Essentially where are the codes for showing that, or are you implementing t= >his as private code? The problem is in DatePosix.m3 . To reproduce, set your TZ environment variable to Europe/London and try to convert a Date of 1970-01-01 @ 00:00:00 in the time zone UTC to a Time.T. You should get 0; you will get -3600. Mika > >Thanks in advance > >--- El lun, 11/4/11, Mika Nystrom escribi=F3: > From dragisha at m3w.org Sun Apr 17 16:05:58 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 16:05:58 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? Message-ID: I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? TIA, dd From hosking at cs.purdue.edu Sun Apr 17 17:25:04 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 11:25:04 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: Message-ID: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? > > TIA, > dd > From dabenavidesd at yahoo.es Sun Apr 17 17:31:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 16:31:54 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: Message-ID: <274332.77724.qm@web29720.mail.ird.yahoo.com> Hi all: it's possible to get yet another implementation of it, perhaps when we have the ESC we will be able to check it for Lock level annotations, earlier there was an implementation: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 Is important to keep in mind this await construct annotation may be verified as the one in Locking Level, it would be interesting to see whether this means one can execute commands without locking, is in some way specifying locks underneath. If you want to ask, if their statements is sound it would be good to ask if they might have a copy of the implementation If this is better to check for checks under Pthreads module would be a good idea to use directly their semantics. Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" > Fecha: domingo, 17 de abril, 2011 09:05 > I am still puzzled with this... What > exactly do we have now? Do we have a version where things > work, or something? Or we don't have threads for cm3 > anymore? > > TIA, > dd > > From dragisha at m3w.org Sun Apr 17 18:10:33 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 18:10:33 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. > > On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > >> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >> >> TIA, >> dd >> > From dabenavidesd at yahoo.es Sun Apr 17 18:04:08 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 17:04:08 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <274332.77724.qm@web29720.mail.ird.yahoo.com> Message-ID: <617521.89071.qm@web29714.mail.ird.yahoo.com> Hi all: And yet to more on this, the first on the implementation of mutex abstraction theory based on await construct: http://www.cs.rug.nl/~wim/pub/whh233.ps.gz And better than that I guess, this with Useful Guarded Commands of Dijkstra's language, this is really a good idea to use it inside the ESC checker since it's its internal AST rep: http://www.cs.rug.nl/~wim/onderwijs/concurrency/dikt.pdf.gz Thanks in advance: --- El dom, 17/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" , "Dragi?a Duri?" > Fecha: domingo, 17 de abril, 2011 10:31 > Hi all: > it's possible to get yet another implementation of it, > perhaps when we have the ESC we will be able to check it for > Lock level annotations, earlier there was an > implementation: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 > > Is important to keep in mind this await construct > annotation may be verified as the one in Locking Level, it > would be interesting to see whether this means one can > execute commands without locking, is in some way specifying > locks underneath. > > If you want to ask, if their statements is sound it would > be good to ask if they might have a copy of the > implementation > > If this is better to check for checks under Pthreads module > would be a good idea to use directly their semantics. > > Thanks in advance > > --- El dom, 17/4/11, Dragi?a Duri? > escribi?: > > > De: Dragi?a Duri? > > Asunto: [M3devel] State of threads? pthreads not safe > to use on some platforms? > > Para: "m3devel" > > Fecha: domingo, 17 de abril, 2011 09:05 > > I am still puzzled with this... What > > exactly do we have now? Do we have a version where > things > > work, or something? Or we don't have threads for cm3 > > anymore? > > > > TIA, > > dd > > > > > From hosking at cs.purdue.edu Sun Apr 17 18:43:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 12:43:16 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> So the question is what has changed. I know threads were functional and stable at one point. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 17, 2011, at 12:10 PM, Dragi?a Duri? wrote: > Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... > > On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > >> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. >> >> On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: >> >>> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >>> >>> TIA, >>> dd >>> >> > From hendrik at topoi.pooq.com Sun Apr 17 19:34:55 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 17 Apr 2011 13:34:55 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> Message-ID: <20110417173455.GB11082@topoi.pooq.com> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: > So the question is what has changed. > I know threads were functional and stable at one point. Was that back when they were green threads? Has our use of posix threads ever been stable? Are green threads still available? -- hendrik From hosking at cs.purdue.edu Sun Apr 17 20:13:01 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 14:13:01 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <20110417173455.GB11082@topoi.pooq.com> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >> So the question is what has changed. >> I know threads were functional and stable at one point. > > Was that back when they were green threads? User threads have historically been pretty stable, and I believe still are. Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > Has our use of posix threads ever been stable? > Are green threads still available? > > -- hendrik From dragisha at m3w.org Sun Apr 17 20:18:04 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 20:18:04 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> But, what recent change when my now problematic platform is on 5.8.6....? On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From dabenavidesd at yahoo.es Sun Apr 17 23:21:47 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 22:21:47 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <946881.5570.qm@web29717.mail.ird.yahoo.com> Hi all: it's good to check if at all the platform per target and per threading too, SMP/HT, x86 or AMD64. I'm suspecting more than anything else this really affects specially SMP/HT, but specially not missing that information, perhaps even historically we can't assure that, at least that's how I see it without it, what do you think? Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: domingo, 17 de abril, 2011 13:18 > But, what recent change when my now > problematic platform is on 5.8.6....? > > On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > > > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > > > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony > Hosking wrote: > >>> So the question is what has changed. > >>> I know threads were functional and stable at > one point. > >> > >> Was that back when they were green threads? > > > > User threads have historically been pretty stable, and > I believe still are. > > Similarly, our use of pthreads was stable at one point > but there have been recent changes that appear to have > degraded them. > > > >> Has our use of posix threads ever been stable? > >> Are green threads still available? > >> > >> -- hendrik > > > > From dragisha at m3w.org Mon Apr 18 14:07:40 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 18 Apr 2011 14:07:40 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <7D82E5E5-EC36-4254-A4A7-782A230320FD@m3w.org> User threads were stable, but they had problems of their own... I met some of them when I implemented first pthread support - user threads switching is in order, and pthread switching is out of order. m3pp was failing because of that. On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From mika at async.caltech.edu Mon Apr 18 18:10:28 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:10:28 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Tony Hosking writes: >I suspect the problem is re-entrance to the library. Should you be = >disabling user thread switching somewhere? I suspect the same. That's why I asked what the "rules" are for calling malloc and free. I notice that the m3core code has Scheduler.DisableSwitching around all calls to malloc but not around calls to free. Why would one be unsafe and the other safe? It is always free that my program hangs up on by the way. Surrounding all "suspicious" calls in my code with Scheduler.DisableSwitching doesn't solve the problem, as far as I can tell. I am further suspecting that my problem is related to my having linked the program with -pthread. I am also guessing that this is the same reason you can't link FreeBSD user threads programs with -pthread: namely that if you go reentrantly into the pthread malloc library some code assumes the two invocations are from separate threads and locks up if they are from the same thread. That's my guess anyhow. But that doesn't explain why I've never seen any malloc/free problems on FreeBSD without -pthread. (I don't think I have, anyhow.) Mika From hosking at cs.purdue.edu Mon Apr 18 18:30:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 12:30:45 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Yes, I think you should avoid -pthread. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > Tony Hosking writes: >> I suspect the problem is re-entrance to the library. Should you be = >> disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the "rules" are > for calling malloc and free. I notice that the m3core code has > Scheduler.DisableSwitching around all calls to malloc but not around calls > to free. Why would one be unsafe and the other safe? It is always free > that my program hangs up on by the way. Surrounding all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my having linked the > program with -pthread. I am also guessing that this is the same reason > you can't link FreeBSD user threads programs with -pthread: namely that if > you go reentrantly into the pthread malloc library some code assumes > the two invocations are from separate threads and locks up if they are from > the same thread. That's my guess anyhow. But that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without -pthread. > (I don't think I have, anyhow.) > > Mika From mika at async.caltech.edu Mon Apr 18 18:42:39 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:42:39 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Message-ID: <20110418164239.84CEB1A207A@async.async.caltech.edu> Tony Hosking writes: >Well, that is reassuring news. >If the problems are isolated to threading then they can be resolved easily. > >On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > >> >> Remember that for mission-critical applications there's always user >> threading.. I find that the system (up to and including the CVS head >> and on both amd64 and i386 flavors of Linux as well as every other OS I >> have tried) is very very stable with user threading. I'm eager to use >> pthreads again of course, for all sorts of reasons... >> >> Mika Actually I'm afraid I have to take this back. After digging a bit here, I *think* user threads are OK with the proviso that one not link with -pthread. But I'm not sure. Unfortunately, user threads require -pthread on most systems because of pthread_atfork. I asked Jay what pthread_atfork is used for *under user threads* but he never answered my question. I had understood that pthread_atfork is used in the pthreads implementation to simulate the old behavior of user threads and that it wasn't needed under user threads... I'm also not sure how to change RTProcessC.c so that it picks up pthread_atfork exactly when user threading has been specified. (Programs won't link with pthread_atfork in them without -pthread.) In any case I am pretty sure that linking with -pthread picks up versions of malloc and free that really don't like being called reentrantly---and by "don't like" I mean the worst kind of crash: the application locks up hard in some sort of spin lock. A segfault would be preferable by far. And m3core has a lot of unprotected calls to free in it. There may be some really nasty bugs here. They are nasty because they kill reliability, kill liveness, and can't really be programmed around because they are in such a vital subsystem (namely threading). I would go back to PM3 if I could but I think I'm stuck with CM3 now. Mika P.S. as far as the pthreads bugs go I think it's still a possibility the bugs are in the garbage collector? The user threads m3core disables the garbage collector at different points from where the pthreads m3core does, no? From hosking at cs.purdue.edu Mon Apr 18 19:05:46 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 13:05:46 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418164239.84CEB1A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> Message-ID: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> On Apr 18, 2011, at 12:42 PM, Mika Nystrom wrote: > Tony Hosking writes: >> Well, that is reassuring news. >> If the problems are isolated to threading then they can be resolved easily. >> >> On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: >> >>> >>> Remember that for mission-critical applications there's always user >>> threading.. I find that the system (up to and including the CVS head >>> and on both amd64 and i386 flavors of Linux as well as every other OS I >>> have tried) is very very stable with user threading. I'm eager to use >>> pthreads again of course, for all sorts of reasons... >>> >>> Mika > > > Actually I'm afraid I have to take this back. After digging a bit here, > I *think* user threads are OK with the proviso that one not link with > -pthread. But I'm not sure. > > Unfortunately, user threads require -pthread on most systems because > of pthread_atfork. I asked Jay what pthread_atfork is used for *under > user threads* but he never answered my question. I had understood > that pthread_atfork is used in the pthreads implementation to simulate > the old behavior of user threads and that it wasn't needed under user > threads... I'm also not sure how to change RTProcessC.c so that it > picks up pthread_atfork exactly when user threading has been specified. > (Programs won't link with pthread_atfork in them without -pthread.) pthread_atfork should not be needed under user threads. > > In any case I am pretty sure that linking with -pthread picks up versions > of malloc and free that really don't like being called reentrantly---and > by "don't like" I mean the worst kind of crash: the application locks up > hard in some sort of spin lock. A segfault would be preferable by far. > And m3core has a lot of unprotected calls to free in it. > > There may be some really nasty bugs here. They are nasty because they > kill reliability, kill liveness, and can't really be programmed around > because they are in such a vital subsystem (namely threading). I would > go back to PM3 if I could but I think I'm stuck with CM3 now. > > Mika > > P.S. as far as the pthreads bugs go I think it's still a possibility the > bugs are in the garbage collector? Not so much bugs in GC as bugs in thread interactions with GC perhaps. The GC itself has been stable for a long time. > The user threads m3core disables > the garbage collector at different points from where the pthreads m3core > does, no? From dabenavidesd at yahoo.es Mon Apr 18 19:17:35 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 18:17:35 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <886452.94715.qm@web29705.mail.ird.yahoo.com> Hi all: the only thing you need to keep control as of CM3 is concerned is the modula-3 free calls equals the number of news, that is, if not is runtime error, but it isn't vice versa, if you broke this then you are obviously unsafely corrupted, but as far as the process is concerned it will be growing in time the heap until termination. If this is exacerbated then you will run out of memory either in Modula-3 or C which is dangerous, so this tells you why the scheduler must be aware of this calls, at least that's my rationale. You are right I think there is a unixdb client program that crashes that can't be build in cm3 as it asks to provide their external implementation function calls so you can't declare external calls on your own, you need like compile it too, which is very strange, though my memories about this are very diffuse. Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 11:10 > > Tony Hosking writes: > >I suspect the problem is re-entrance to the library. > Should you be = > >disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the > "rules" are > for calling malloc and free. I notice that the m3core > code has > Scheduler.DisableSwitching around all calls to malloc but > not around calls > to free. Why would one be unsafe and the other > safe? It is always free > that my program hangs up on by the way. Surrounding > all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve > the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my > having linked the > program with -pthread. I am also guessing that this > is the same reason > you can't link FreeBSD user threads programs with -pthread: > namely that if > you go reentrantly into the pthread malloc library some > code assumes > the two invocations are from separate threads and locks up > if they are from > the same thread. That's my guess anyhow. But > that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without > -pthread. > (I don't think I have, anyhow.) > > Mika > > From mika at async.caltech.edu Mon Apr 18 20:37:20 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:37:20 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <20110418183720.E4C311A207A@async.async.caltech.edu> I don't think the pthreads implementation has ever been 100% correct. If it has, please let me know which CVS date to check out!!! (I use a PM3 that is almost ten years old simply because this stuff works and am happy to work with ANYTHING newer that works.) As far as user ("green") threads goes, I think they *may* be right if you are careful not to link with -pthread. This means you have to change RTThreadC.c (remove pthread_atfork) or your program won't link. (Please don't call pthreads "Posix" threads on this list: I know it is correct but it is confusing because Modula-3's user threads implementation for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 usage predates IEEE/ISO standardization of threads so there was no confusion at the time the name was introduced.) Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >But, what recent change when my now problematic platform is on = >5.8.6....? > >On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > >>=20 >> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>=20 >>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>> So the question is what has changed. >>>> I know threads were functional and stable at one point. >>>=20 >>> Was that back when they were green threads? >>=20 >> User threads have historically been pretty stable, and I believe still = >are. >> Similarly, our use of pthreads was stable at one point but there have = >been recent changes that appear to have degraded them. >>=20 >>> Has our use of posix threads ever been stable? >>> Are green threads still available? >>>=20 >>> -- hendrik >>=20 From mika at async.caltech.edu Mon Apr 18 20:45:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:45:40 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> Message-ID: <20110418184540.C2D801A2078@async.async.caltech.edu> Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From mika at async.caltech.edu Mon Apr 18 20:47:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:47:40 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <886452.94715.qm@web29705.mail.ird.yahoo.com> References: <886452.94715.qm@web29705.mail.ird.yahoo.com> Message-ID: <20110418184740.D20261A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >the only thing you need to keep control as of CM3 is concerned is the modul= >a-3 free calls equals the number of news, that is, if not is runtime error,= Not exactly. You can't necessarily call malloc/free from multiple threads. This is a big problem for Modula-3 programs that call out to nontrivial C code. (Why don't we wrap malloc/free ourselves in a special library? Linker problems I suppose... and I also suppose the effort should rather be on getting pthreads working again than messing around with problems with user threads...) Mika From mika at async.caltech.edu Mon Apr 18 21:01:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:01:44 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> <20110418183720.E4C311A207A@async.async.caltech.edu> <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> Message-ID: <20110418190144.406E41A207A@async.async.caltech.edu> If you have a copy, can you try the thread tester program on it? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >First implementation of pthreads for m3 was made for pm3 in December of = >2004... To the best of my knowledge, it was correct. As long as you do = >not define correct like "repetitive behaviour of green threads". I used = >it to run DC++ hub I wrote as a pet project for stress testign it. = >Worked well. > >On Apr 18, 2011, at 8:37 PM, Mika Nystrom wrote: > >> I don't think the pthreads implementation has ever been 100% correct. >>=20 >> If it has, please let me know which CVS date to check out!!! (I use a >> PM3 that is almost ten years old simply because this stuff works and = >am >> happy to work with ANYTHING newer that works.) >>=20 >> As far as user ("green") threads goes, I think they *may* be right if = >you >> are careful not to link with -pthread. This means you have to change = >RTThreadC.c >> (remove pthread_atfork) or your program won't link. >>=20 >> (Please don't call pthreads "Posix" threads on this list: I know it is >> correct but it is confusing because Modula-3's user threads = >implementation >> for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 = >usage=20 >> predates IEEE/ISO standardization of threads so there was no confusion = >at=20 >> the time the name was introduced.) >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>> But, what recent change when my now problematic platform is on =3D >>> 5.8.6....? >>>=20 >>> On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: >>>=20 >>>> =3D20 >>>> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>>> =3D20 >>>>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>>>> So the question is what has changed. >>>>>> I know threads were functional and stable at one point. >>>>> =3D20 >>>>> Was that back when they were green threads? >>>> =3D20 >>>> User threads have historically been pretty stable, and I believe = >still =3D >>> are. >>>> Similarly, our use of pthreads was stable at one point but there = >have =3D >>> been recent changes that appear to have degraded them. >>>> =3D20 >>>>> Has our use of posix threads ever been stable? >>>>> Are green threads still available? >>>>> =3D20 >>>>> -- hendrik >>>> =3D20 From dabenavidesd at yahoo.es Mon Apr 18 20:57:01 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 19:57:01 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418184740.D20261A207A@async.async.caltech.edu> Message-ID: <564805.90054.qm@web29714.mail.ird.yahoo.com> Hi all: ok, but are you telling this about in user threads or working ill or all threads and pthreads? Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 13:47 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >the only thing you need to keep control as of CM3 is > concerned is the modul= > >a-3 free calls equals the number of news, that is, if > not is runtime error,= > > Not exactly. You can't necessarily call malloc/free > from multiple threads. > This is a big problem for Modula-3 programs that call out > to nontrivial C code. > > (Why don't we wrap malloc/free ourselves in a special > library? Linker problems > I suppose... and I also suppose the effort should rather be > on getting pthreads > working again than messing around with problems with user > threads...) > > Mika > From mika at async.caltech.edu Mon Apr 18 21:05:55 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:05:55 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <564805.90054.qm@web29714.mail.ird.yahoo.com> References: <564805.90054.qm@web29714.mail.ird.yahoo.com> Message-ID: <20110418190555.526411A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >ok, but are you telling this about in user threads or working ill or all th= >reads and pthreads? >Thanks in advance > I don't know exactly. There are certainly bad combinations of threading implementations and memory allocators. And I'm sure it may be system dependent as well. In any case, linking user threads with the -pthreads version of libc is probably bad. Calling malloc willy-nilly from multiple user threads is probably bad. Calling free, I don't know. Pthreads currently don't work so maybe malloc works there but that doesn't help very much if the system is anyhow unstable. Mika >--- El lun, 18/4/11, Mika Nystrom escribi=F3: > >> De: Mika Nystrom >> Asunto: Re: [M3devel] malloc/free from Modula-3? >> Para: "Daniel Alejandro Benavides D." >> CC: m3devel at elegosoft.com >> Fecha: lunes, 18 de abril, 2011 13:47 >> "Daniel Alejandro Benavides D." >> writes: >> >Hi all: >> >the only thing you need to keep control as of CM3 is >> concerned is the modul=3D >> >a-3 free calls equals the number of news, that is, if >> not is runtime error,=3D >>=20 >> Not exactly. You can't necessarily call malloc/free >> from multiple threads. >> This is a big problem for Modula-3 programs that call out >> to nontrivial C code. >>=20 >> (Why don't we wrap malloc/free ourselves in a special >> library? Linker problems >> I suppose... and I also suppose the effort should rather be >> on getting pthreads >> working again than messing around with problems with user >> threads...) >>=20 >> Mika >> From dabenavidesd at yahoo.es Mon Apr 18 22:18:49 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 21:18:49 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418190555.526411A207A@async.async.caltech.edu> Message-ID: <967344.66270.qm@web29718.mail.ird.yahoo.com> Hi all: yes pthread allocation must keep their pthread under system constraints, but per thread user space the outer space is limited, then that's why perhaps RT is breaking due system behaviour not by its own consideration, if I may say so, so perhaps this is just the unsafe pattern break codes showed, since the implementation is responsible for avoiding this flaws but assuming they satisfies their restrictions, which is modularly sound, but if not then somebody either doesn't guarantee our assumptions, or can't meet their specifications and ours too so don't assume they satisfy theirs own ones too. Then in fact would explain why there are many crash patterns around which is very system dependent not us (that's why I favour one implementation per target source, mixing both is like meeting their specifications and so like if it would be our goal, correct if I'm wrong here please). Anything else is just falling apart of breakages that somehow rather than literally must be addressed by their own, if its so breaking the code they should, or us adapt to it literally. --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 14:05 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >ok, but are you telling this about in user threads or > working ill or all th= > >reads and pthreads? > >Thanks in advance > > > > I don't know exactly. There are certainly bad > combinations of threading > implementations and memory allocators. And I'm sure > it may be system > dependent as well. In any case, linking user threads > with the -pthreads > version of libc is probably bad. Calling malloc > willy-nilly from multiple > user threads is probably bad. Calling free, I don't > know. Pthreads currently > don't work so maybe malloc works there but that doesn't > help very much if > the system is anyhow unstable. > > Mika > > >--- El lun, 18/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] malloc/free from Modula-3? > >> Para: "Daniel Alejandro Benavides D." > >> CC: m3devel at elegosoft.com > >> Fecha: lunes, 18 de abril, 2011 13:47 > >> "Daniel Alejandro Benavides D." > >> writes: > >> >Hi all: > >> >the only thing you need to keep control as of > CM3 is > >> concerned is the modul=3D > >> >a-3 free calls equals the number of news, that > is, if > >> not is runtime error,=3D > >>=20 > >> Not exactly. You can't necessarily call > malloc/free > >> from multiple threads. > >> This is a big problem for Modula-3 programs that > call out > >> to nontrivial C code. > >>=20 > >> (Why don't we wrap malloc/free ourselves in a > special > >> library? Linker problems > >> I suppose... and I also suppose the effort should > rather be > >> on getting pthreads > >> working again than messing around with problems > with user > >> threads...) > >>=20 > >> Mika > >> > From dabenavidesd at yahoo.es Mon Apr 18 23:01:10 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 22:01:10 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <967344.66270.qm@web29718.mail.ird.yahoo.com> Message-ID: <411737.17355.qm@web29709.mail.ird.yahoo.com> Hi all: this unsound trend is what it makes software development uncontrolled, so many unstable codes, it has an effect as Dr Nelson said software quality and perhaps more dangerous ever more, at least with Modula-3 you can keep it under certain control you know are restricted but this is the result of that, their codes are broken then why one need to fix them if it's worse than before. What a mess if so. Thanks in advance --- El lun, 18/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 15:18 > Hi all: > yes pthread allocation must keep their pthread under system > constraints, but per thread user space the outer space is > limited, then that's why perhaps RT is breaking due system > behaviour not by its own consideration, if I may say so, so > perhaps this is just the unsafe pattern break codes showed, > since the implementation is responsible for avoiding this > flaws but assuming they satisfies their restrictions, which > is modularly sound, but if not then somebody either doesn't > guarantee our assumptions, or can't meet their > specifications and ours too so don't assume they satisfy > theirs own ones too. > Then in fact would explain why there are many crash > patterns around which is very system dependent not us > (that's why I favour one implementation per target source, > mixing both is like meeting their specifications and so like > if it would be our goal, correct if I'm wrong here please). > Anything else is just falling apart of breakages that > somehow rather than literally must be addressed by their > own, if its so breaking the code they should, or us adapt to > it literally. > > --- El lun, 18/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] malloc/free from Modula-3? > > Para: "Daniel Alejandro Benavides D." > > CC: m3devel at elegosoft.com > > Fecha: lunes, 18 de abril, 2011 14:05 > > "Daniel Alejandro Benavides D." > > writes: > > >Hi all: > > >ok, but are you telling this about in user threads > or > > working ill or all th= > > >reads and pthreads? > > >Thanks in advance > > > > > > > I don't know exactly. There are certainly bad > > combinations of threading > > implementations and memory allocators. And I'm > sure > > it may be system > > dependent as well. In any case, linking user > threads > > with the -pthreads > > version of libc is probably bad. Calling malloc > > willy-nilly from multiple > > user threads is probably bad. Calling free, I > don't > > know. Pthreads currently > > don't work so maybe malloc works there but that > doesn't > > help very much if > > the system is anyhow unstable. > > > > Mika > > > > >--- El lun, 18/4/11, Mika Nystrom > > escribi=F3: > > > > > >> De: Mika Nystrom > > >> Asunto: Re: [M3devel] malloc/free from > Modula-3? > > >> Para: "Daniel Alejandro Benavides D." > > >> CC: m3devel at elegosoft.com > > >> Fecha: lunes, 18 de abril, 2011 13:47 > > >> "Daniel Alejandro Benavides D." > > >> writes: > > >> >Hi all: > > >> >the only thing you need to keep control > as of > > CM3 is > > >> concerned is the modul=3D > > >> >a-3 free calls equals the number of news, > that > > is, if > > >> not is runtime error,=3D > > >>=20 > > >> Not exactly. You can't necessarily > call > > malloc/free > > >> from multiple threads. > > >> This is a big problem for Modula-3 programs > that > > call out > > >> to nontrivial C code. > > >>=20 > > >> (Why don't we wrap malloc/free ourselves in > a > > special > > >> library? Linker problems > > >> I suppose... and I also suppose the effort > should > > rather be > > >> on getting pthreads > > >> working again than messing around with > problems > > with user > > >> threads...) > > >>=20 > > >> Mika > > >> > > > From rcolebur at SCIRES.COM Mon Apr 18 23:35:40 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 18 Apr 2011 17:35:40 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: -pthreads -user threads -Win32 native threads -green threads -etc. I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Monday, April 18, 2011 2:46 PM To: Tony Hosking Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From hosking at cs.purdue.edu Tue Apr 19 00:11:26 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 18:11:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > Tony Hosking writes: > ... >> pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } From dabenavidesd at yahoo.es Tue Apr 19 00:46:52 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 23:46:52 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <937894.40594.qm@web29704.mail.ird.yahoo.com> Hi all: I think if each thread handle is RT specific form then you need to be aware of that before wanting to change system and platform, rather than of implementation, so each specific module procedure must be not UNSAFE but platform ready for its available platforms, now if something is specified implementation dependent must have a CM3 suffix for that purpose so everybody keeps and eye on its implementation aspects, so each RTThreadCM3.i3 is clearer to group as they are intended to serve as are RTCollectorSRC.i3 to RTCollector.i3 to mean is not specific to CM3 so to have implementations e.g SOLsun user threads, LINUXLIBC6 x86 user threads (implemented for Multimedia Modula-3, called MM3, etc) but anything else like for unique for Windows, SPIN, is like normal like RTException.i3 and if something is supported just for DEC Unix for instance, then it is better to say RTExceptionOSF.i3 etc in that way if so and not CM3 or anything else. Thanks in advance --- El lun, 18/4/11, Coleburn, Randy escribi?: > De: Coleburn, Randy > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "m3devel at elegosoft.com" > Fecha: lunes, 18 de abril, 2011 16:35 > I think it would be good at this > point to clarify for folks on the m3devel list what is meant > by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" > threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still > using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not > behaving well. This situation concerns me because > thread safety is something I've always relied on Modula-3 to > provide. > > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel > 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user > threads. > ... > > The following code from RTProcessC.c ensures that it is > neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in > source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to > use autoconf. > * VMS lacks pthread_atfork? Would be good to use > autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, > parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > > From jay.krell at cornell.edu Tue Apr 19 01:20:57 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:20:57 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: And "yes" you need to disable switching around free, the same as malloc. And calloc. And realloc. The disable switching *might* be at a lower level than you realize, so *might* not be missing. That is, it might be in our C wrappers. I can check later. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 12:30:45 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Yes, I think you should avoid -pthread. > > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > > > > Tony Hosking writes: > >> I suspect the problem is re-entrance to the library. Should you be = > >> disabling user thread switching somewhere? > > > > I suspect the same. That's why I asked what the "rules" are > > for calling malloc and free. I notice that the m3core code has > > Scheduler.DisableSwitching around all calls to malloc but not around calls > > to free. Why would one be unsafe and the other safe? It is always free > > that my program hangs up on by the way. Surrounding all "suspicious" calls > > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > > far as I can tell. > > > > I am further suspecting that my problem is related to my having linked the > > program with -pthread. I am also guessing that this is the same reason > > you can't link FreeBSD user threads programs with -pthread: namely that if > > you go reentrantly into the pthread malloc library some code assumes > > the two invocations are from separate threads and locks up if they are from > > the same thread. That's my guess anyhow. But that doesn't explain why > > I've never seen any malloc/free problems on FreeBSD without -pthread. > > (I don't think I have, anyhow.) > > > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:26:26 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:26:26 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ Ok, "we" should probably go ahead and fix that. I'll try to, but no promises, sorry. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 18:11:26 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? > > On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > > > Tony Hosking writes: > > ... > >> pthread_atfork should not be needed under user threads. > > ... > > > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > FreeBSD before 6. The comment is from the checked-in source file. > > > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ > > > > INTEGER > > __cdecl > > RTProcess__RegisterForkHandlers( > > ForkHandler prepare, > > ForkHandler parent, > > ForkHandler child) > > { > > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > > * VMS lacks pthread_atfork? Would be good to use autoconf. > > * Win32 lacks pthread_atfork and fork. OK. > > * > > * As well, for all Posix systems, we could implement > > * atfork ourselves, as long as we provide a fork() > > * wrapper that code uses. > > */ > > #if defined(_WIN32) \ > > || defined(__vms) \ > > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > > return 0; > > #else > > while (1) > > { > > int i = pthread_atfork(prepare, parent, child); > > if (i != EAGAIN) > > return i; > > sleep(0); > > } > > #endif > > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:28:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:28:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, Message-ID: "Green" threads are probably another name for user threads. Java used to use something with this name. > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? Certainly. Has anything ever given you a different impression? Cygwin maybe uses pthreads, but that'd still use Win32 threads under the covers. It *might* be nice to have a user thread option that used fibers or Win7's UMS (64-bit only!) but don't hold your breath. Native Win32 threads have been around forever and will be a good option forever and will be in heavy use forever. - Jay > From: rcolebur at SCIRES.COM > To: m3devel at elegosoft.com > Date: Mon, 18 Apr 2011 17:35:40 -0400 > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. > > I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 04:27:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:27:04 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: <20110419022704.6EFB71A207A@async.async.caltech.edu> Jay K writes: >--_f28739e3-2465-450f-b041-5795980d5bfe_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >And "yes" you need to disable switching around free=2C the same as malloc. = >And calloc. And realloc. >The disable switching *might* be at a lower level than you realize=2C so *m= >ight* not be missing. >That is=2C it might be in our C wrappers. I can check later. >=20 > - Jay Well in that case someone should remove the SchedulerPosix.DisableSwitching (etc) from around the mallocs. Or are you telling me that perhaps free() is wrapped but malloc() isn't? That would be... confusing. Mika From mika at async.caltech.edu Tue Apr 19 04:30:58 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:30:58 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: <20110419023058.4FBB11A207A@async.async.caltech.edu> If you or Tony could describe roughly what you think needs to be done I'd be happy to look into it. The basic problem is that a decision that's made/described in m3core/thread/m3makefile needs to someone find its way to controlling what C code gets compiled elsewhere in the system. Or maybe there should be an indirection into the thread library to pick up pthread_atfork (or not). But then again you still haven't explained why you made the user threads use pthread_atfork. I just remove it from my installations, but then again I'm not trying to run CVSup so I don't know if it breaks that program to do so. Mika Jay K writes: >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> The following code from RTProcessC.c ensures that it is neeeded on every = >Unix except >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ > >=20 >Ok=2C "we" should probably go ahead and fix that. >I'll try to=2C but no promises=2C sorry. >=20 > - Jay >=20 >> From: hosking at cs.purdue.edu >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >> Probably unnecessary=2C given that I think there is another entry point t= >o forking a process (I forget where) in the thread-specific portion of m3co= >re. In which case the necessary work might be done there? >>=20 >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: >>=20 >> > Tony Hosking writes: >> > ... >> >> pthread_atfork should not be needed under user threads. >> > ... >> >=20 >> > The following code from RTProcessC.c ensures that it is neeeded on ever= >y Unix except >> > FreeBSD before 6. The comment is from the checked-in source file. >> >=20 >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ >> >=20 >> > INTEGER >> > __cdecl >> > RTProcess__RegisterForkHandlers( >> > ForkHandler prepare=2C >> > ForkHandler parent=2C >> > ForkHandler child) >> > { >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> > * Win32 lacks pthread_atfork and fork. OK. >> > * >> > * As well=2C for all Posix systems=2C we could implement >> > * atfork ourselves=2C as long as we provide a fork() >> > * wrapper that code uses. >> > */ >> > #if defined(_WIN32) \ >> > || defined(__vms) \ >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> > return 0=3B >> > #else >> > while (1) >> > { >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B >> > if (i !=3D EAGAIN) >> > return i=3B >> > sleep(0)=3B >> > } >> > #endif >> > } >>=20 > = > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > fixed if need be.
>=3B >=3B */

> =3B
>Ok=2C "we" should probably go ahead and fix that.
>I'll try to=2C but no =3Bpromises=2C sorry.
> =3B
> =3B- Jay
 =3B
>>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = >that I think there is another entry point to forking a process (I forget wh= >ere) in the thread-specific portion of m3core. In which case the necessary = >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child)>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = >as long as we provide a fork()
>=3B >=3B * wrapper that code uses.R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > >= > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- From jay.krell at cornell.edu Tue Apr 19 08:07:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:07:01 +0000 Subject: [M3devel] pthread_atfork Message-ID: I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: no-op for Win32: just return 0 It is tempting to have this assert(0) but that is wrong. The function will be called and should just do nothing. as-is for pthread: call pthread_atfork manually for userthreads: FreeBSD < 6: does it have pthreads? Can we declare such systems use user threads? take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 VMS: Punt for now. Maybe newer versions than I was using have it. And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible RTProcessPthread__Fork: return fork(); RTProcessUserThreads__Fork: wrapper around fork() that handles the callbacks from RTProcess__RegisterForkHandlers. RTProcessVms__Fork: assert(false) I think, unless newer VMS has fork(). One thing not stated here, is that I believe something "like" pthread_atfork is somewhere between reasonable and actually needed, on all Posix/Unix systems. Because, what isn't well known here, but I think is true, is that we depend on pthread_atfork (or a reimplementation thereof) as described here, so that fork + do more work with user threads behaves like pthreads, and doesn't have all the threads survive. Or..and I assume the answer is no: Do we want fork + do more work to keep all threads for user threads and keep only one thread for pthreads? How it was for a long time, but we didn't discover until fairly recently. I'm 99% sure this is all exactly correct. Possibly this is enough explanation for others to do it. Otherwise, maybe I'll get to it "soon". Maybe. This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:15:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:15:49 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110419022704.6EFB71A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , , , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, , <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu>, , <20110419022704.6EFB71A207A@async.async.caltech.edu> Message-ID: Here is where malloc/calloc/free are handled, in a new, somewhat redundant way, that reduces/eliminates the burden at higher levels: jbook2:src jay$ pwd /dev2/cm3/m3-libs/m3core/src jbook2:src jay$ edit C/Common/CstdlibC.c M3WRAP_NO_SWITCHING(void*, Cstdlib__malloc, malloc, (WORD_T a), (a)) M3WRAP_NO_SWITCHING(void*, Cstdlib__calloc, calloc, (WORD_T a, WORD_T b), (a, b)) M3WRAP_RETURN_VOID_NO_SWITCHING(Cstdlib__free, free, (void* a), (a)) Now, if you have other C code that is calling malloc/free, you still need to disable switching around that. Perhaps all calls to <*extern*> functions should disable/enableswitching. Perhaps. The real answer is though, make sure pthreads and Win32 threads work, and then don't do much to make user threads faster or easier to maintain. see m3core/src/m3core.h for definitions of those macros. They should be pretty clear. If not, run the C preprocessor over those files and look at the results. -- The traditional correct thing is to wrap Cstdlib.malloc/Cstdlib.free whereever they are called. But I kept finding this missing, got fed up with it, fixed it at a lower level. Is that bad? Maybe. Maybe not. -- I wouldn't argue against wrapping all <*extern*> calls with disable/enableswitching. Given that you have to recompile everything to switch between user threads and kernel threads anyway, the compiler could insert these calls only for user threads, perhaps. - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:27:04 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Jay K writes: > >--_f28739e3-2465-450f-b041-5795980d5bfe_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >And "yes" you need to disable switching around free=2C the same as malloc. = > >And calloc. And realloc. > >The disable switching *might* be at a lower level than you realize=2C so *m= > >ight* not be missing. > >That is=2C it might be in our C wrappers. I can check later. > >=20 > > - Jay > > Well in that case someone should remove the > SchedulerPosix.DisableSwitching (etc) from around the mallocs. > > Or are you telling me that perhaps free() is wrapped but malloc() isn't? > That would be... confusing. > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:23:55 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <937894.40594.qm@web29704.mail.ird.yahoo.com> References: , <937894.40594.qm@web29704.mail.ird.yahoo.com> Message-ID: > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:33:31 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:33:31 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: , , <937894.40594.qm@web29704.mail.ird.yahoo.com>, Message-ID: Caveat: OpenBSD now only offers pthreads. Modula-3 user threads require either get/set/make/swapcontext, or sigaltstack. OpenBSD offers neither. There were implementations using setjmp, but they were most unportable, only remained for OpenBSD, and I removed them. - Jay From: jay.krell at cornell.edu To: dabenavidesd at yahoo.es; m3devel at elegosoft.com; rcolebur at scires.com Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 13:51:06 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 04:51:06 -0700 Subject: [M3devel] LONGINT -> more bugs in m3tk Message-ID: <20110419115106.7E6A31A207A@async.async.caltech.edu> Hello m3devel, If anyone knows how to fix the following trivially, please go ahead. "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal (errors reported by m3tk). The offending line is: INTERFACE BasicCtypes; IMPORT Word, Long; TYPE (* the four signed integer types *) signed_char = [-16_7f-1 .. 16_7f]; short_int = [-16_7fff-1 .. 16_7fff]; int = [-16_7fffffff-1 .. 16_7fffffff]; long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) (* the four unsigned integer types *) unsigned_char = [16_0 .. 16_ff]; unsigned_short_int = [16_0 .. 16_ffff]; unsigned_int = [16_0 .. 16_ffffffff]; unsigned_long_int = Word.T; unsigned_long_long = Long.T; (* the three floating types *) float = REAL; double = LONGREAL; long_double = EXTENDED; (* char *) char = signed_char; END BasicCtypes. I'll look at it myself, but not immediately (no time). Workaround for now. Mika From jay.krell at cornell.edu Tue Apr 19 17:06:38 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 15:06:38 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419023058.4FBB11A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Apr 19 17:13:55 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 16:13:55 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <768127.64883.qm@web29706.mail.ird.yahoo.com> Hi all: but I think there should be a way with the first implementation of MM3 if so, what is the risk of not having it? Can you see it, please: https://mail.elegosoft.com/pipermail/m3devel/2007-July/000272.html Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 10:06 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: ? user thread/pthread inconsistency ? cvsupd incompatibility with pthreads ? user threads using pthread_atfork/-pthread ? fork() calls having to go through a wrapper (ok -- you could miss this and ?? not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) ?- Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 17:29:24 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 08:29:24 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: <20110419152924.22C411A207A@async.async.caltech.edu> Never explained this clearly before. Thanks, Jay. I suppose the reason CVSup forks and does work is to save copying overhead because the two forked processes can share read-only pages without any copying? I remember that the last time I looked at this my reaction was "fork() is not part of Modula-3 : people who use it are on their own". In a sense this is still true as FreeBSD simply will not work with -pthread. I'm afraid that, however, that removing -pthread didn't fix my Linux lockup problems. Turns out the library I was having trouble with is a PostgreSQL library written by Critical Mass. To fix it, I wrote a program using m3tk and some Scheme code that will take any M3 interface and wrap every routine in that interface in the following code structure: PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = BEGIN SchedulerIndirection.DisableSwitching(); TRY RETURN PQ.PQconnectPoll(conn) FINALLY SchedulerIndirection.EnableSwitching() END END PQconnectPoll; which then gave rise to my previous bug report about BasicCTypes, which m3tk cannot process a.t.m., owing to LONGINT breakage. Also I have to call "SchedulerIndirection" since the routines to turn off switching are named differently on CM3 (Scheduler.DisableSwitching) and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous to me but oh well I must have a dozen of these CM3/PM3 differences to deal with, what's another one? Mika Jay K writes: >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- From dabenavidesd at yahoo.es Tue Apr 19 18:08:43 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 17:08:43 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: <86414.26723.qm@web29716.mail.ird.yahoo.com> Hi all: I think the worst case scenario is to adapt changes to modification of such bugs, or UNSAFE parts, if it just gets to pick of a portion of the system well stabilized then all we can that depended on that, I hope this stabilizes soon, then we can all share the rest of the things. Thanks in advance --- El mar, 19/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: martes, 19 de abril, 2011 10:29 > Never explained this clearly > before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save > copying > overhead because the two forked processes can share > read-only pages > without any copying? > > I remember that the last time I looked at this my reaction > was "fork() > is not part of Modula-3 : people who use it are on their > own". In a sense > this is still true as FreeBSD simply will not work with > -pthread. > > I'm afraid that, however, that removing -pthread didn't fix > my Linux > lockup problems. Turns out the library I was having > trouble with is > a PostgreSQL library written by Critical Mass. To fix > it, I wrote a > program using m3tk and some Scheme code that will take any > M3 interface > and wrap every routine in that interface in the following > code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : > PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about > BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the > routines to turn > off switching are named differently on CM3 > (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just > seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 > differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() > survived fork() > >Now: user threads and pthreads: only the thread calling > fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user > threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads > and user threads. > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost > immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to > fork(). > >Each module with a need to do something before/after > fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is > needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must > implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own > fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this > approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now > become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok > -- you could miss this an= > >d=20 > > not likely notice -- only fork() > calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but > people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same > thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you > think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's > made/described in > >> m3core/thread/m3makefile needs to someone find its > way to controlling > >> what C code gets compiled elsewhere in the > system. Or maybe there should > >> be an indirection into the thread library to pick > up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you > made the user threads = > >use > >> pthread_atfork. I just remove it from my > installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it > breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c > ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix > that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I > think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the > thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be > done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C > Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be > needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c > ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is > from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks > pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be > good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and > fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix > systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as > we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > > > > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of > pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B > >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix > that.
> >> >I'll try to=3D2C but > no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- > Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B > Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B > CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C > glibc-2.6-4
>=3D3B
>=3D3B > Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to > forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. > In which case the necessa= > >ry =3D > >> >work might be done > there?
>=3D3B
>=3D3B On Apr > 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom > wrote:
>=3D3B
>=3D3B > >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B > ...
>=3D3B >=3D3B>=3D3B > pthread_atfork sh= > >ould not be n=3D > >> >eeded under user > threads.
>=3D3B >=3D3B > ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B > >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B > >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if > need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B > >=3D3B INTEGER
>=3D3B >=3D3B > __cdecl
&= > >gt=3D3B >=3D3B =3D > >> > >RTProcess__RegisterForkHandlers(
>=3D3B > >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler > parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B > {
>=3D3B >=3D3B /* FreeBSD <=3D3B > 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use > autoconf.
>=3D3B >=3D3B * VMS lacks > pthread= > >_atfork=3D > >> >? Would be good to use > autoconf.
>=3D3B >=3D3B * Win32 lacks > pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B > >=3D3B *
>=3D3B >=3D3B * As > well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could > implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a > fork()
>=3D3B >=3D3B * wrapper that > code = > >uses. >> >R>>=3D3B >=3D3B > */
>=3D3B >=3D3B #if defined(_WIN32) > \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B > >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ > )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B > while (1)
>=3D3B >=3D3B > {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C > parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B > >=3D3B return i=3D3B
>=3D3B > >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B > }
>=3D3B >=3D3B > #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > > > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > > > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived > fork()
Previously: pthrea= > >ds: only the thread calling fork() survived > fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives > fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and > pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads > and user threads.
= > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or > exec almost immediately a= > >fter fork.


pthread_atfork > offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and > change all the calls to fork()= > >.
Each module with a need to do something > before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and > pthread_atfork cooperate to do w= > >hat is needed.


No function > pointer is needed.
Instead move th= > >e code to > m3core/thread.


If you really > must not use -pthread=2C = > >then you must implement pthread_atfork > functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that > cooperates
with = > >your pthread_atfork replacement.
There is no > free lunch -- there is a do= > >wnside to this approach=2C as plain fork() > calls
are ok and correct if p= > >thread_atfork is used=2C but now become > incorrect.
Pick your poison:
= > > =3B user thread/pthread > inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads > using pthread_atfork/-pthread >r> =3B fork() calls having to go through a > wrapper (ok -- you could mis= > >s this and
 =3B =3B not > likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper). >


(Most of this has = > >been explained multiple times=2C but people only pay > attention
when they= > > think it affects them. I'm guilty of the same > thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B > Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B > CC= > >: m3devel at elegosoft.com
>=3B > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C > glibc-2.6-4
>=3B
>=3B >
>=3B If = > >you or Tony could describe roughly what you think needs > to be done
>= > >=3B I'd be happy to look into it.
>=3B >
>=3B The basic problem is= > > that a decision that's made/described > in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to > controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there > should
>=3B be an ind= > >irection into the thread library to pick up > pthread_atfork (or not).
>= > >=3B
>=3B But then again you still > haven't explained why you made the = > >user threads use
>=3B > pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not > trying to run CVSup so I don't = > >know if it breaks that program to do > so.
>=3B
>=3B > Mika
= > >>=3B
>=3B Jay K > writes:
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B > >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B= > > >=3B
>=3B > >=3B
>=3B >=3B>=3B The > following code from RTP= > >rocessC.c ensures that it is neeeded on every > =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B > /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability > of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need > be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B > >=3B=3D20
>=3B >=3BOk=3D2C "we" > shoul= > >d probably go ahead and fix that.
>=3B > >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B > >=3B=3D20
>=3B >=3B - > Jay
>=3B = > >>=3B=3D20
>=3B > >=3B>=3B From: hosking at cs.purdue.edu
>=3B > &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B > >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B > >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B > Probably unnecessary=3D2C given that I thi= > >nk there is another entry point > t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of > m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done > there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On > Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B > >=3B>=3B=3D20
>=3B > >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B > >=3B>=3B >=3B ...
>=3B > >=3B>= > >=3B >=3B>=3B pthread_atfork should not be > needed under user threads. >>>=3B >=3B>=3B >=3B > ...
>=3B >=3B>=3B > >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code > from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By > Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from > the checked-in source file= > >.
>=3B >=3B>=3B > >=3B=3D20
>=3B >=3B>=3B > >=3B /* NOTE:= > > Even userthreads now depends
>=3B > >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B > >=3B>=3B >=3B * This can be fixed if need > b= > >e.
>=3B >=3B>=3B >=3B > */
>=3B >=3B>=3B > >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B > INTEGER
>=3B >=3B>=3B >=3B > __cdecl
= > >>=3B >=3B>=3B >=3B > RTProcess__RegisterForkHandlers(
>=3B > >= > >=3B>=3B >=3B ForkHandler > prepare=3D2C
>=3B >=3B>=3B > >=3B For= > >kHandler parent=3D2C
>=3B > >=3B>=3B >=3B ForkHandler > child)
&g= > >t=3B >=3B>=3B >=3B > {
>=3B >=3B>=3B >=3B /* > FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use > autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be > good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks > pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B > *
>=3B >=3B>=3B >=3B * As > well=3D2C for = > >all Posix systems=3D2C we could > implement
>=3B >=3B>=3B > >=3B * a= > >tfork ourselves=3D2C as long as we provide a > fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code > uses.
>=3B >=3B>=3B >=3B > */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) > \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B > >=3B>=3B >=3B || (defined(__FreeBSD__) > /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ > )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B > >=3B #else
>=3B >=3B>=3B > >= > >=3B while (1)
>=3B >=3B>=3B > >=3B {
>=3B >=3B>=3B > >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C > child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D > EAGAIN)
>=3B >=3B>=3B >=3B > retu= > >rn i=3D3B
>=3B >=3B>=3B > >=3B sleep(0)=3D3B
>=3B > >=3B>= > >=3B >=3B }
>=3B > >=3B>=3B >=3B #endif
>=3B > >=3B>=3B &g= > >t=3B }
>=3B > >=3B>=3B=3D20
>=3B >=3B > > > > =3D
>= > >=3B >=3B
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B > charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B > >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B > >=3B<=3Bhead>=3B
>=3B > >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B > >=3B.hmmessage P
>=3B > >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B > >=3Bpadding:0px
>=3B >=3B} >r>>=3B > >=3Bbody.hmmessage
>=3B > >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B > >=3Bfont-family:Tahoma
>=3B > >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B > >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody > class=3D3D'hmmessage'>=3B
>=3B > >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it > is neeeded on e=3D >>>=3B >=3Bvery Unix > except<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /= > >* NOTE: Even userthreads now > depends<=3BBR>=3B=3D
>=3B > >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of > pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can > be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > */<=3BBR>=3B<=3BBR>= > >=3B
>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3BOk=3D2C = > >"we" should probably go ahead and fix > that.<=3BBR>=3B
>=3B > >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C > sorry.<=3BBR>=3B >>>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3Bnbsp=3D= > >3B- > Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B > Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B > >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B > CC: m3devel at el= > >egos=3D
>=3B > >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B > Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C > kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B > >=3Bthat I think there is anot= > >her entry point to forking a process (I forget > wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case > the necessary =3D >>>=3B >=3Bwork might be done > there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C > at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom > wrote:<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking > writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B > ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be > n=3D
>=3B >=3Beede= > >d under user > threads.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B > =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code > from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B > >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The > comment is from the=3D
&= > >gt=3B >=3B checked-in source > file.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now > depends<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B= > > * on availability of > pthreads.<=3BBR>=3B&=3B=3D
>=3B > >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need > be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B > __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B > >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler > prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 > lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use > autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks > pthread_atfork=3D
>=3B >=3B? Would be > good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. > OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix > systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork > ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a > fork()<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3= > >B * wrapper that code > uses.<=3BB=3D
>=3B > >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) > \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > || (defined(__F= > >reeBSD__) /* > &=3Bamp=3D3B&=3Bam=3D
>=3B > >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ > )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D
>=3B >=3B=3D3B > #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > in=3D
= > >>=3B >=3Bt i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B > sleep(0)=3D3B<=3BBR=3D
>=3B > >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B > }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B > =3D
>=3B >=3B > > > <=3B/body>=3B<= > >br>>=3B > >=3B<=3B/html>=3B=3D
>=3B > >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> > > > > >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- > From jay.krell at cornell.edu Tue Apr 19 20:50:16 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 18:50:16 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: It's easy enough for us to support either name. - Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 22:32:54 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 20:32:54 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: > This change depends on pthread_atfork, for pthreads and user threads. Clarification: There's no change described there for pthreads, and indeed, getting this behavior with pthreads doesn't depend on using pthread_atfork. That is just how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs. fork()). The Solaris default used to be fork => forkall(). The current Solaris default is fork => fork1(). There really do exist on Solaris to functions you can call, fork1() and forkall(). All other systems and the Posix specification have the "fork1" semantic for fork. Therefore "fork1" is what we should provide unconditionally. If you really want "forkall", then you write: #ifdef __sun forkall() #else #error #endif and/or you go and hack up m3core/src/thread/POSIX, branch off m3core/src/thread/POSIX_FORKALL or such. I don't expect anyone to ever do that. The need for pthread_atfork is sort of different for user threads vs. pthreads, but it is needed either way. For pthreads, there is this problem -- at the time a thread calls fork(), what are other threads doing? Answer: Anything. Follow up question: What locks might they hold? Answer: Any. Follow up: So, if arbitrary locks are held, and you fork, and don't exec, and the child process attempts to use some of the locks that were held in the parent, and only the forking thread survived, then the other threads will never leave their locks, and later attempts to acquire them in the children will deadlock. Therefore, what pthread_atfork provides for, what you are supposed to use it for, is roughly speaking, right before fork(), in the parent, acquire all locks, and then fork(), and then in the parent and children, release all locks. Therefore, you have to come up with a locking order. Things are a little different for user threads. In user threads, the existance of threads is established by virtue of a global array or list of data describing the threads. And a timer to occasionally preempt. When the timer interrupts, the scheduler picks another thread from the global array/list. Previously, the data went unchanged when fork() was called. Therefore all threads survived. What we do now is that in the child "handler" for pthread_atfork, is reinitialize the various data, so as to forget about all but the current thread. Something like that. I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork, then ForkBeforeExec might get away with just fork() and ignore all this. So far we have not drawn this line -- there is just the one fork and it is used prior to exec or do-more-work and does the same thing w/o knowing what is to follow. The Posix documentation I think describes this all fairly well. Bing for "posix opengroup pthread_atfork". Really. I think it is saying much of the same thing I am saying, but in language that has received much more thought. Be sure to read the "rationale" part. - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Date: Tue, 19 Apr 2011 15:06:38 +0000 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 20 00:10:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 23:10:45 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <718150.90366.qm@web29720.mail.ird.yahoo.com> Hi all: since there was just that interface in *nix platforms, then it must be called like that , but another name will be just create SchedulerCM3, thus suggests SchedulerPosix would be called rather than nothing suffixing it SchedulerSRC and the other one Scheduler alone be the default like to say SchedulerCM3 AS Scheduler? so to switch to SchedulerSRC AS Scheduler provided that both are implementation ready so one can select SchedulerCM3 as SchedulerSRC or Scheduler so one name comes after each rep rather than changing every other place each time, even of Internal Nodes as in m3tk which specifies its members as? SRC_NODE <: AST_NODE then something like CM3_NODE likiwise subtyped if so should be created to hold the extensions (can somebody name them WIDECHAR, LONGINT, LONGCARD, what else am I missing if so?),? like we could come up with something cleaner than just put all there. I will try to se a proposal for that. Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 13:50 It's easy enough for us to support either name. ?- Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Apr 20 04:18:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 20 Apr 2011 02:18:25 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: Can you by chance see the other threads? I suspect not -- user threads I presume. Can you try sprinking a lot more Disable/EnableSwitching around? I guess you tried that..and..? - Jay > From: hosking at cs.purdue.edu > Date: Fri, 15 Apr 2011 11:26:15 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > > > Hi m3devel, > > > > I'm debugging a problem with some code I have that calls out to C > > code. It's calling pqlib (PostgreSQL library) and you might expect > > things to go wrong there, but as it turns out it's dying in CM3's code. > > > > The system is AMD64_LINUX (user threading because of the problems with > > pthreads) and the program is just freezing up hard. No segfault, > > bus error or anything. It locks up a couple of times a day. > > > > What are the rules on calling malloc/free from Modula-3? > > > > Stack trace: > > > > (gdb) where > > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > > ) at ../src/C/Common/CstdlibC.c:26 > > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > > ) at ../src/C/Common/M3toC.m3:46 > > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > > ) at ../src/UnsafeDatabase.m3:230 > > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > > ) at ../src/UnsafeDatabase.m3:228 > > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > > ) at ../src/UnsafeDatabase.m3:258 > > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > > ) at ../src/DesynchronizedDB.m3:126 > > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > > ) at ../src/DesynchronizedDB.m3:293 > > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > > ) at ../src/GCOMSOrderManager.m3:596 > > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > > ) at ../src/GCOMSOrderManager.m3:310 > > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > > ) at ../src/EuroDriver.m3:1108 > > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > > ) at ../src/EuroDriver.m3:1545 > > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > > #17 0x0000000000000000 in ?? () > > (gdb) > > > > Here is the code that causes the problem: > > > > VAR > > s := CopyTtoS(query); > > BEGIN > > TRY > > t.res := t.myExec(s,busyWait) > > FINALLY > > FreeCopiedS(s) (* stuck here *) > > END > > END; > > > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > > routine (PQexec I think it's called). > > > > Since this is already an unsafe module I suppose I could just make an > > array of CHARs on the stack and pass a pointer to that... > > > > > > Regards, > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Apr 20 07:55:57 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 22:55:57 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: <20110420055557.089101A207A@async.async.caltech.edu> The rationale you directed us to says: "It is suggested that programs that use fork() call an exec function very soon afterwards in the child process, thus resetting all states. In the meantime, only a short list of async-signal-safe library routines are promised to be available." and later "Application programs may not be aware that a multi-threaded library is in use, and they feel free to call any number of library routines between the fork() and exec calls, just as they always have. Indeed, they may be extant single-threaded programs and cannot, therefore, be expected to obey new restrictions imposed by the threads library." In C maybe that is true. In Modula-3, every application has *always* had to assume that there are other threads floating around. There is no such thing as a "single-threaded" Modula-3 program---or at least, the distinction doesn't matter. I really think fork-and-do-more-work is just a bug. Maybe it is a special pattern used by some special program---maybe. In any case support for it I do not think belongs in m3core. Mika Jay K writes: >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > This change depends on pthread_atfork=2C for pthreads and user threads. > > >Clarification: There's no change described there for pthreads=2C and indeed= >=2C getting >this behavior with pthreads doesn't depend on using pthread_atfork. That is= > just >how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > fork()). >The Solaris default used to be fork =3D> forkall(). >The current Solaris default is fork =3D> fork1(). >There really do exist on Solaris to functions you can call=2C fork1() and f= >orkall(). >All other systems and the Posix specification have the "fork1" semantic for= > fork. >Therefore "fork1" is what we should provide unconditionally. >If you really want "forkall"=2C then you write: >#ifdef __sun > forkall() >#else > #error >#endif > > > >and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >thread/POSIX_FORKALL or such. >I don't expect anyone to ever do that. > > > >The need for pthread_atfork is sort of different for user threads vs. pthre= >ads=2C but it is needed either way. >For pthreads=2C there is this problem -- at the time a thread calls fork()= >=2C what are other threads doing? >Answer: Anything. >Follow up question: What locks might they hold? >Answer: Any. >Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >xec=2C and the child process >attempts to use some of the locks that were held in the parent=2C and only = >the forking thread survived=2C >then the other threads will never leave their locks=2C and later attempts t= >o acquire them in the children >will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >e supposed to use it for=2C >is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >locks=2C and then fork()=2C and then in the >parent and children=2C release all locks. Therefore=2C you have to come up = >with a locking order. > > > >Things are a little different for user threads. >In user threads=2C the existance of threads is established by virtue of a g= >lobal array or list of data describing the threads. >And a timer to occasionally preempt. When the timer interrupts=2C the sched= >uler picks another thread from the global >array/list. Previously=2C the data went unchanged when fork() was called. T= >herefore all threads survived. >What we do now is that in the child "handler" for pthread_atfork=2C is rein= >itialize the various data=2C so as to forget about >all but the current thread. Something like that. > > > >I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >rk=2C then ForkBeforeExec >might get away with just fork() and ignore all this. >So far we have not drawn this line -- there is just the one fork and it is = >used prior to exec or do-more-work >and does the same thing w/o knowing what is to follow. > > >The Posix documentation I think describes this all fairly well. >Bing for "posix opengroup pthread_atfork". >Really. I think it is saying much of the same thing I am saying=2C but in l= >anguage that has received >much more thought. Be sure to read the "rationale" part. > > > - Jay > > >From: jay.krell at cornell.edu >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >2.6-4 >Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > > > > > > > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > > =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >er threads.


Clarification: There's no change described there for= > pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >epend on using pthread_atfork. That is just
how pthreads are -- except o= >ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >ault used to be fork =3D>=3B forkall().
The current Solaris default is= > fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >t we should provide unconditionally.
If you really want "forkall"=2C the= >n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >ror
#endif



and/or you go and hack up m3core/src/thread/PO= >SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >t anyone to ever do that.



The need for pthread_atfork is sor= >t of different for user threads vs. pthreads=2C but it is needed either way= >.
For pthreads=2C there is this problem -- at the time a thread calls fo= >rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >process
attempts to use some of the locks that were held in the parent= >=2C and only the forking thread survived=2C
then the other threads will = >never leave their locks=2C and later attempts to acquire them in the childr= >en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >you are supposed to use it for=2C
is roughly speaking=2C right before fo= >rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >n the
parent and children=2C release all locks. Therefore=2C you have to= > come up with a locking order.



Things are a little different= > for user threads.
In user threads=2C the existance of threads is establ= >ished by virtue of a global array or list of data describing the threads.r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >eduler picks another thread from the global
array/list. Previously=2C th= >e data went unchanged when fork() was called. Therefore all threads survive= >d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >s reinitialize the various data=2C so as to forget about
all but the cur= >rent thread. Something like that.



I suspect if we had RTProc= >ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >br>might get away with just fork() and ignore all this.
So far we have n= >ot drawn this line -- there is just the one fork and it is used prior to ex= >ec or do-more-work
and does the same thing w/o knowing what is to follow= >.


The Posix documentation I think describes this all fairly well= >.
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >ying much of the same thing I am saying=2C but in language that has receive= >d
much more thought. Be sure to read the "rationale" part.


&n= >bsp=3B- Jay


From: jay.krell at cornell.edu<= >br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >[M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >: Tue=2C 19 Apr 2011 15:06:38 +0000

> >"> > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From mika at async.caltech.edu Wed Apr 20 08:12:17 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 23:12:17 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: <20110420061217.C539A1A207A@async.async.caltech.edu> Jay K writes: >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume. >Can you try sprinking a lot more Disable/EnableSwitching around? >I guess you tried that..and..? So far so good. I'll let you know tomorrow. (The application runs 8 1/2 hours a day and has been crashing about twice a day.) The annoying thing is that when it does lock up, it locks up in one of the wrapped free()s, not in the unwrapped one. Mika > > - Jay > > >> From: hosking at cs.purdue.edu >> Date: Fri=2C 15 Apr 2011 11:26:15 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] malloc/free from Modula-3? >>=20 >> I suspect the problem is re-entrance to the library. Should you be disabl= >ing user thread switching somewhere? >>=20 >> Antony Hosking | Associate Professor | Computer Science | Purdue Universi= >ty >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Office +1 765 494 6001 | Mobile +1 765 427 5484 >>=20 >>=20 >>=20 >>=20 >> On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote: >>=20 >> > Hi m3devel=2C >> >=20 >> > I'm debugging a problem with some code I have that calls out to C >> > code. It's calling pqlib (PostgreSQL library) and you might expect >> > things to go wrong there=2C but as it turns out it's dying in CM3's cod= >e. >> >=20 >> > The system is AMD64_LINUX (user threading because of the problems with >> > pthreads) and the program is just freezing up hard. No segfault=2C >> > bus error or anything. It locks up a couple of times a day. >> >=20 >> > What are the rules on calling malloc/free from Modula-3? >> >=20 >> > Stack trace: >> >=20 >> > (gdb) where >> > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 >> > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 >> > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 >> > #3 0x000000000091462f in Cstdlib__free (a=3DCannot access memory at ad= >dress 0x2ac73b0ef0d8 >> > ) at ../src/C/Common/CstdlibC.c:26 >> > #4 0x000000000091477d in M3toC__FreeCopiedS (s=3DCannot access memory = >at address 0x2ac73b0ef0f8 >> > ) at ../src/C/Common/M3toC.m3:46 >> > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot= > access memory at address 0x2ac73b0ef118 >> > ) at ../src/UnsafeDatabase.m3:230 >> > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access memo= >ry at address 0x2ac73b0ef148 >> > ) at ../src/UnsafeDatabase.m3:228 >> > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef268 >> > ) at ../src/UnsafeDatabase.m3:258 >> > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=3DCannot access me= >mory at address 0x2ac73b0ef328 >> > ) at ../src/DesynchronizedDB.m3:126 >> > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=3DCannot access m= >emory at address 0x2ac73b0ef3e8 >> > ) at ../src/DesynchronizedDB.m3:293 >> > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=3DCannot a= >ccess memory at address 0x2ac73b0ef5a8 >> > ) at ../src/GCOMSOrderManager.m3:596 >> > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=3DCann= >ot access memory at address 0x2ac73b0ef668 >> > ) at ../src/GCOMSOrderManager.m3:310 >> > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price=3D= >{}=2C count=3DCannot access memory at address 0x2ac73b0ef7b= >8 >> > ) at ../src/EuroDriver.m3:1108 >> > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../sr= >c/EuroDriver.m3:1401 >> > #14 0x00000000004197a1 in EuroDriver__OApply (cl=3DCannot access memory= > at address 0x2ac73b0efd38 >> > ) at ../src/EuroDriver.m3:1545 >> > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/PO= >SIX/ThreadPosix.m3:1174 >> > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 >> > #17 0x0000000000000000 in ?? () >> > (gdb)=20 >> >=20 >> > Here is the code that causes the problem: >> >=20 >> > VAR >> > s :=3D CopyTtoS(query)=3B >> > BEGIN >> > TRY >> > t.res :=3D t.myExec(s=2CbusyWait) >> > FINALLY >> > FreeCopiedS(s) (* stuck here *) >> > END >> > END=3B >> >=20 >> > The query is a SQL query---t.myExec eventually calls the libpq SQL exec >> > routine (PQexec I think it's called). >> >=20 >> > Since this is already an unsafe module I suppose I could just make an >> > array of CHARs on the stack and pass a pointer to that... >> >=20 >> >=20 >> > Regards=2C >> > Mika >>=20 > = > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume.
Can you try sprinking a lot more Disable/EnableSwitching around?<= >br>I guess you tried that..and..?

 =3B- Jay


>=3B Fr= >om: hosking at cs.purdue.edu
>=3B Date: Fri=2C 15 Apr 2011 11:26:15 -0400= >
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegosoft.comr>>=3B Subject: Re: [M3devel] malloc/free from Modula-3?
>=3B
&g= >t=3B I suspect the problem is re-entrance to the library. Should you be dis= >abling user thread switching somewhere?
>=3B
>=3B Antony Hosking= > | Associate Professor | Computer Science | Purdue University
>=3B 305= > N. University Street | West Lafayette | IN 47907 | USA
>=3B Office +1= > 765 494 6001 | Mobile +1 765 427 5484
>=3B
>=3B
>=3B
= >>=3B
>=3B On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote:>>=3B
>=3B >=3B Hi m3devel=2C
>=3B >=3B
>=3B >=3B = >I'm debugging a problem with some code I have that calls out to C
>=3B= > >=3B code. It's calling pqlib (PostgreSQL library) and you might expect= >
>=3B >=3B things to go wrong there=2C but as it turns out it's dyin= >g in CM3's code.
>=3B >=3B
>=3B >=3B The system is AMD64_LIN= >UX (user threading because of the problems with
>=3B >=3B pthreads) = >and the program is just freezing up hard. No segfault=2C
>=3B >=3B = >bus error or anything. It locks up a couple of times a day.
>=3B >= >=3B
>=3B >=3B What are the rules on calling malloc/free from Modula= >-3?
>=3B >=3B
>=3B >=3B Stack trace:
>=3B >=3B
&g= >t=3B >=3B (gdb) where
>=3B >=3B #0 0x00002ac519a9202e in ?? () fr= >om /lib/libc.so.6
>=3B >=3B #1 0x00002ac519a2fb43 in ?? () from /li= >b/libc.so.6
>=3B >=3B #2 0x00002ac519a2caab in free () from /lib/li= >bc.so.6
>=3B >=3B #3 0x000000000091462f in Cstdlib__free (a=3DCanno= >t access memory at address 0x2ac73b0ef0d8
>=3B >=3B ) at ../src/C/Co= >mmon/CstdlibC.c:26
>=3B >=3B #4 0x000000000091477d in M3toC__FreeCo= >piedS (s=3DCannot access memory at address 0x2ac73b0ef0f8
>=3B >=3B = >) at ../src/C/Common/M3toC.m3:46
>=3B >=3B #5 0x00000000007884f0 in= > UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot access memory at address 0x2= >ac73b0ef118
>=3B >=3B ) at ../src/UnsafeDatabase.m3:230
>=3B &g= >t=3B #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef148
>=3B >=3B ) at ../src/UnsafeDatabase.m= >3:228
>=3B >=3B #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t= >=3DCannot access memory at address 0x2ac73b0ef268
>=3B >=3B ) at ../= >src/UnsafeDatabase.m3:258
>=3B >=3B #8 0x000000000077ad5b in Desync= >hronizedDB__SExec (t=3DCannot access memory at address 0x2ac73b0ef328
&g= >t=3B >=3B ) at ../src/DesynchronizedDB.m3:126
>=3B >=3B #9 0x0000= >00000077c190 in DesynchronizedDB__SMExec (t=3DCannot access memory at addre= >ss 0x2ac73b0ef3e8
>=3B >=3B ) at ../src/DesynchronizedDB.m3:293
&= >gt=3B >=3B #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t= >=3DCannot access memory at address 0x2ac73b0ef5a8
>=3B >=3B ) at ../= >src/GCOMSOrderManager.m3:596
>=3B >=3B #11 0x0000000000566b95 in GCO= >MSOrderManager__PlaceSingleOrder (t=3DCannot access memory at address 0x2ac= >73b0ef668
>=3B >=3B ) at ../src/GCOMSOrderManager.m3:310
>=3B &= >gt=3B #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price= >=3D{<=3BNo data fields>=3B}=2C count=3DCannot access memory at address = >0x2ac73b0ef7b8
>=3B >=3B ) at ../src/EuroDriver.m3:1108
>=3B &g= >t=3B #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../s= >rc/EuroDriver.m3:1401
>=3B >=3B #14 0x00000000004197a1 in EuroDriver= >__OApply (cl=3DCannot access memory at address 0x2ac73b0efd38
>=3B >= >=3B ) at ../src/EuroDriver.m3:1545
>=3B >=3B #15 0x00000000009112c1 = >in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174
= >>=3B >=3B #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6
>=3B= > >=3B #17 0x0000000000000000 in ?? ()
>=3B >=3B (gdb)
>=3B &= >gt=3B
>=3B >=3B Here is the code that causes the problem:
>=3B= > >=3B
>=3B >=3B VAR
>=3B >=3B s :=3D CopyTtoS(quer= >y)=3B
>=3B >=3B BEGIN
>=3B >=3B TRY
>=3B >=3B = > t.res :=3D t.myExec(s=2CbusyWait)
>=3B >=3B FINALLY
&= >gt=3B >=3B FreeCopiedS(s) (* stuck here *)
>=3B >=3B E= >ND
>=3B >=3B END=3B
>=3B >=3B
>=3B >=3B The query = >is a SQL query---t.myExec eventually calls the libpq SQL exec
>=3B >= >=3B routine (PQexec I think it's called).
>=3B >=3B
>=3B >= >=3B Since this is already an unsafe module I suppose I could just make anr>>=3B >=3B array of CHARs on the stack and pass a pointer to that...r>>=3B >=3B
>=3B >=3B
>=3B >=3B Regards=2C
>=3B = >>=3B Mika
>=3B
>= > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_-- From hosking at cs.purdue.edu Wed Apr 20 16:58:06 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 10:58:06 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110420055557.089101A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: Agreed. But CVSup used it and we were trying to be supportive. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > The rationale you directed us to says: > > "It is suggested that programs that use fork() call an exec function very > soon afterwards in the child process, thus resetting all states. In the > meantime, only a short list of async-signal-safe library routines are > promised to be available." > > and later > > "Application programs may not be aware that a multi-threaded library > is in use, and they feel free to call any number of library routines > between the fork() and exec calls, just as they always have. Indeed, > they may be extant single-threaded programs and cannot, therefore, > be expected to obey new restrictions imposed by the threads library." > > In C maybe that is true. In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is > no such thing as a "single-threaded" Modula-3 program---or at least, > the distinction doesn't matter. > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > pattern used by some special program---maybe. In any case support for it > I do not think belongs in m3core. > > Mika > > > > Jay K writes: >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >>> This change depends on pthread_atfork=2C for pthreads and user threads. >> >> >> Clarification: There's no change described there for pthreads=2C and indeed= >> =2C getting >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= >> just >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= >> fork()). >> The Solaris default used to be fork =3D> forkall(). >> The current Solaris default is fork =3D> fork1(). >> There really do exist on Solaris to functions you can call=2C fork1() and f= >> orkall(). >> All other systems and the Posix specification have the "fork1" semantic for= >> fork. >> Therefore "fork1" is what we should provide unconditionally. >> If you really want "forkall"=2C then you write: >> #ifdef __sun >> forkall() >> #else >> #error >> #endif >> >> >> >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >> thread/POSIX_FORKALL or such. >> I don't expect anyone to ever do that. >> >> >> >> The need for pthread_atfork is sort of different for user threads vs. pthre= >> ads=2C but it is needed either way. >> For pthreads=2C there is this problem -- at the time a thread calls fork()= >> =2C what are other threads doing? >> Answer: Anything. >> Follow up question: What locks might they hold? >> Answer: Any. >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >> xec=2C and the child process >> attempts to use some of the locks that were held in the parent=2C and only = >> the forking thread survived=2C >> then the other threads will never leave their locks=2C and later attempts t= >> o acquire them in the children >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >> e supposed to use it for=2C >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >> locks=2C and then fork()=2C and then in the >> parent and children=2C release all locks. Therefore=2C you have to come up = >> with a locking order. >> >> >> >> Things are a little different for user threads. >> In user threads=2C the existance of threads is established by virtue of a g= >> lobal array or list of data describing the threads. >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= >> uler picks another thread from the global >> array/list. Previously=2C the data went unchanged when fork() was called. T= >> herefore all threads survived. >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= >> itialize the various data=2C so as to forget about >> all but the current thread. Something like that. >> >> >> >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >> rk=2C then ForkBeforeExec >> might get away with just fork() and ignore all this. >> So far we have not drawn this line -- there is just the one fork and it is = >> used prior to exec or do-more-work >> and does the same thing w/o knowing what is to follow. >> >> >> The Posix documentation I think describes this all fairly well. >> Bing for "posix opengroup pthread_atfork". >> Really. I think it is saying much of the same thing I am saying=2C but in l= >> anguage that has received >> much more thought. Be sure to read the "rationale" part. >> >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >> 2.6-4 >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 >> >> >> >> >> >> >> >> >> Previously: user threads: all threads survived fork() >> Previously: pthreads: only the thread calling fork() survived fork() >> Now: user threads and pthreads: only the thread calling fork() survives for= >> k() >> This must be how pthreads behave=2C and this makes user threads and pthread= >> s consistent. >> >> This change depends on pthread_atfork=2C for pthreads and user threads. >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd. >> Most programs either never fork=2C or exec almost immediately after fork. >> >> >> pthread_atfork offers a good model. >> A sort of "distributed" model. >> You don't have to go and change all the calls to fork(). >> Each module with a need to do something before/after fork=2C calls the cent= >> ral pthread_atfork=2C >> and fork and pthread_atfork cooperate to do what is needed. >> >> >> No function pointer is needed. >> Instead move the code to m3core/thread. >> >> >> If you really must not use -pthread=2C then you must implement pthread_atfo= >> rk functionality >> yourself and have all fork() calls go through your own fork() wrapper that = >> cooperates >> with your pthread_atfork replacement. >> There is no free lunch -- there is a downside to this approach=2C as plain = >> fork() calls >> are ok and correct if pthread_atfork is used=2C but now become incorrect. >> Pick your poison: >> user thread/pthread inconsistency >> cvsupd incompatibility with pthreads=20 >> user threads using pthread_atfork/-pthread=20 >> fork() calls having to go through a wrapper (ok -- you could miss this an= >> d=20 >> not likely notice -- only fork() calls in fork+do-more-work programs nee= >> d the wrapper).=20 >> >> >> (Most of this has been explained multiple times=2C but people only pay atte= >> ntion >> when they think it affects them. I'm guilty of the same thing.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >>> From: mika at async.caltech.edu >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >> c-2.6-4 >>> =20 >>> =20 >>> If you or Tony could describe roughly what you think needs to be done >>> I'd be happy to look into it. >>> =20 >>> The basic problem is that a decision that's made/described in >>> m3core/thread/m3makefile needs to someone find its way to controlling >>> what C code gets compiled elsewhere in the system. Or maybe there should >>> be an indirection into the thread library to pick up pthread_atfork (or n= >> ot). >>> =20 >>> But then again you still haven't explained why you made the user threads = >> use >>> pthread_atfork. I just remove it from my installations=2C but then again= >> I'm >>> not trying to run CVSup so I don't know if it breaks that program to do s= >> o. >>> =20 >>> Mika >>> =20 >>> Jay K writes: >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= >> ry =3D >>>> Unix except >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>> >>>> =3D20 >>>> Ok=3D2C "we" should probably go ahead and fix that. >>>> I'll try to=3D2C but no promises=3D2C sorry. >>>> =3D20 >>>> - Jay >>>> =3D20 >>>>> From: hosking at cs.purdue.edu >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >>>>> To: mika at async.caltech.edu >>>>> CC: m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >> 2C glib=3D >>>> c-2.6-4 >>>>> =3D20 >>>>> Probably unnecessary=3D2C given that I think there is another entry po= >> int t=3D >>>> o forking a process (I forget where) in the thread-specific portion of m= >> 3co=3D >>>> re. In which case the necessary work might be done there? >>>>> =3D20 >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >>>>> =3D20 >>>>>> Tony Hosking writes: >>>>>> ... >>>>>>> pthread_atfork should not be needed under user threads. >>>>>> ... >>>>>> =3D20 >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= >> ver=3D >>>> y Unix except >>>>>> FreeBSD before 6. The comment is from the checked-in source file. >>>>>> =3D20 >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>>>> =3D20 >>>>>> INTEGER >>>>>> __cdecl >>>>>> RTProcess__RegisterForkHandlers( >>>>>> ForkHandler prepare=3D2C >>>>>> ForkHandler parent=3D2C >>>>>> ForkHandler child) >>>>>> { >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. >>>>>> * Win32 lacks pthread_atfork and fork. OK. >>>>>> * >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement >>>>>> * atfork ourselves=3D2C as long as we provide a fork() >>>>>> * wrapper that code uses. >>>>>> */ >>>>>> #if defined(_WIN32) \ >>>>>> || defined(__vms) \ >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >>>>>> return 0=3D3B >>>>>> #else >>>>>> while (1) >>>>>> { >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >>>>>> if (i !=3D3D EAGAIN) >>>>>> return i=3D3B >>>>>> sleep(0)=3D3B >>>>>> } >>>>>> #endif >>>>>> } >>>>> =3D20 >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>> >>>> >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >> nds
=3D >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >> This can be=3D >>>> fixed if need be.
>=3D3B >=3D3B */

>>>>  =3D3B
>>>> Ok=3D2C "we" should probably go ahead and fix that.
>>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>>>>  =3D3B
>>>>  =3D3B- Jay
 =3D3B
>>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >> 011 18:11=3D >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >> @elegos=3D >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >> C kerne=3D >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >> =3D2C given =3D >>>> that I think there is another entry point to forking a process (I forget= >> wh=3D >>>> ere) in the thread-specific portion of m3core. In which case the necessa= >> ry =3D >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >> =3D2C at 2:45=3D >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >> ing writes:=3D >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >> ould not be n=3D >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >>
>=3D3B =3D >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >> rom the=3D >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >> OTE: Even u=3D >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >> s.
&=3D >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >>
>=3D3B =3D >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >> gt=3D3B >=3D3B =3D >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >> e=3D2C>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >> ler child)>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >> hread_atfork.=3D >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >> _atfork=3D >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >> ead_atf=3D >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >> =3D2C for all =3D >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >> elves=3D2C =3D >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >> uses.>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >> =3D3B >=3D3B =3D >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >> p=3D3B&am=3D >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B> R>>=3D3B >=3D >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >> =3D3B >=3D3B in=3D >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >> B >=3D3B if (=3D >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= >> sleep(0)=3D3B>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >> BR>>=3D3B
=3D >>>> >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- >> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >> er threads.


Clarification: There's no change described there for= >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >> epend on using pthread_atfork. That is just
how pthreads are -- except o= >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >> t we should provide unconditionally.
If you really want "forkall"=2C the= >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >> t anyone to ever do that.



The need for pthread_atfork is sor= >> t of different for user threads vs. pthreads=2C but it is needed either way= >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >> process
attempts to use some of the locks that were held in the parent= >> =2C and only the forking thread survived=2C
then the other threads will = >> never leave their locks=2C and later attempts to acquire them in the childr= >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >> n the
parent and children=2C release all locks. Therefore=2C you have to= >> come up with a locking order.



Things are a little different= >> for user threads.
In user threads=2C the existance of threads is establ= >> ished by virtue of a global array or list of data describing the threads.> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >> eduler picks another thread from the global
array/list. Previously=2C th= >> e data went unchanged when fork() was called. Therefore all threads survive= >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >> s reinitialize the various data=2C so as to forget about
all but the cur= >> rent thread. Something like that.



I suspect if we had RTProc= >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >> br>might get away with just fork() and ignore all this.
So far we have n= >> ot drawn this line -- there is just the one fork and it is used prior to ex= >> ec or do-more-work
and does the same thing w/o knowing what is to follow= >> .


The Posix documentation I think describes this all fairly well= >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >> ying much of the same thing I am saying=2C but in language that has receive= >> d
much more thought. Be sure to read the "rationale" part.


&n= >> bsp=3B- Jay


From: jay.krell at cornell.edu<= >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >> : Tue=2C 19 Apr 2011 15:06:38 +0000

>> >> > "> >> >> >> >> >> Previously: user threads: all threads survived fork()
Previously: pthrea= >> ds: only the thread calling fork() survived fork()
Now: user threads and= >> pthreads: only the thread calling fork() survives fork()
This must be h= >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >> br> >> This change depends on pthread_atfork=2C for pthreads and user threads.
= >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= >> tributed" model.
You don't have to go and change all the calls to fork()= >> .
Each module with a need to do something before/after fork=2C calls the= >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >> hat is needed.


No function pointer is needed.
Instead move th= >> e code to m3core/thread.


If you really must not use -pthread=2C = >> then you must implement pthread_atfork functionality
yourself and have a= >> ll fork() calls go through your own fork() wrapper that cooperates
with = >> your pthread_atfork replacement.
There is no free lunch -- there is a do= >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread > r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >> s this and
 =3B =3B not likely notice -- only fork() calls in f= >> ork+do-more-work programs need the wrapper).


(Most of this has = >> been explained multiple times=2C but people only pay attention
when they= >> think it affects them. I'm guilty of the same thing.)


 =3B-= >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >> you or Tony could describe roughly what you think needs to be done
>= >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= >> e needs to someone find its way to controlling
>=3B what C code gets c= >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >> irection into the thread library to pick up pthread_atfork (or not).
>= >> =3B
>=3B But then again you still haven't explained why you made the = >> user threads use
>=3B pthread_atfork. I just remove it from my instal= >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >> know if it breaks that program to do so.
>=3B
>=3B Mika
= >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >> . In which case the necessary work might be done there?
>=3B >=3B>= >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >> =3B >=3B>=3B pthread_atfork should not be needed under user threads.>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= >> The following code from RTProcessC.c ensures that it is neeeded on e=3D>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >> n the thread-specific portion of m3core. In which case the necessary =3D>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
>> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From jay.krell at cornell.edu Thu Apr 21 02:27:15 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 00:27:15 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." Any idea why? I don't know. Is this such a burden? We need a global lock order for it. Unreasonable? I understand pthread_atfork is causing a problem with user threads, but - It really ought not. The libc vs. libpthread design is broken. I believe many systems don't have this broken design, but I guess some do. The broken design is that libc and libpthread implement some functions with the same name, and you get to chose which one. And some of them might be thread-safe. > In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is This is the case in C too. I can't assume any library I link to doesn't create some helper threads with pthreads. pthreads should be assumed always in use, and there should be no ramifications of this. I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread safe vs. ones that are. We really need pthreads to work and to worry less about user threads imho. I believe we have seen the same problems with Win32 threads as with pthreads. (Yes, I know, some people might protest that Win32 threads work fine, but pthreads work very well too. It is only the new stress testing that is turning up problems as I understand, and it turns up problems in pthreads and Win32 threads. Ok, there is now some new mention of problems with other programs?) I suspect the problem is in the allocator/garbage collector, but that user threads don't stress it enough. I suspect I suspect I suspect...whatever.. real debugging is needed, (my own) suspicion and speculation isn't going to help much. It would of course be good if someone anyone can find a point in the history where pthreads and Win32 threads are really solid, with these new one or two test programs, if such a point exists -- I'm not sure one does, or that recent changes are the problem. But, again, I don't know. Time, debugging, investigation is needed. - Jay > From: hosking at cs.purdue.edu > Date: Wed, 20 Apr 2011 10:58:06 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Agreed. But CVSup used it and we were trying to be supportive. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > > > > The rationale you directed us to says: > > > > "It is suggested that programs that use fork() call an exec function very > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." > > > > and later > > > > "Application programs may not be aware that a multi-threaded library > > is in use, and they feel free to call any number of library routines > > between the fork() and exec calls, just as they always have. Indeed, > > they may be extant single-threaded programs and cannot, therefore, > > be expected to obey new restrictions imposed by the threads library." > > > > In C maybe that is true. In Modula-3, every application has *always* > > had to assume that there are other threads floating around. There is > > no such thing as a "single-threaded" Modula-3 program---or at least, > > the distinction doesn't matter. > > > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > > pattern used by some special program---maybe. In any case support for it > > I do not think belongs in m3core. > > > > Mika > > > > > > > > Jay K writes: > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >>> This change depends on pthread_atfork=2C for pthreads and user threads. > >> > >> > >> Clarification: There's no change described there for pthreads=2C and indeed= > >> =2C getting > >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= > >> just > >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > >> fork()). > >> The Solaris default used to be fork =3D> forkall(). > >> The current Solaris default is fork =3D> fork1(). > >> There really do exist on Solaris to functions you can call=2C fork1() and f= > >> orkall(). > >> All other systems and the Posix specification have the "fork1" semantic for= > >> fork. > >> Therefore "fork1" is what we should provide unconditionally. > >> If you really want "forkall"=2C then you write: > >> #ifdef __sun > >> forkall() > >> #else > >> #error > >> #endif > >> > >> > >> > >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= > >> thread/POSIX_FORKALL or such. > >> I don't expect anyone to ever do that. > >> > >> > >> > >> The need for pthread_atfork is sort of different for user threads vs. pthre= > >> ads=2C but it is needed either way. > >> For pthreads=2C there is this problem -- at the time a thread calls fork()= > >> =2C what are other threads doing? > >> Answer: Anything. > >> Follow up question: What locks might they hold? > >> Answer: Any. > >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= > >> xec=2C and the child process > >> attempts to use some of the locks that were held in the parent=2C and only = > >> the forking thread survived=2C > >> then the other threads will never leave their locks=2C and later attempts t= > >> o acquire them in the children > >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= > >> e supposed to use it for=2C > >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = > >> locks=2C and then fork()=2C and then in the > >> parent and children=2C release all locks. Therefore=2C you have to come up = > >> with a locking order. > >> > >> > >> > >> Things are a little different for user threads. > >> In user threads=2C the existance of threads is established by virtue of a g= > >> lobal array or list of data describing the threads. > >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= > >> uler picks another thread from the global > >> array/list. Previously=2C the data went unchanged when fork() was called. T= > >> herefore all threads survived. > >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= > >> itialize the various data=2C so as to forget about > >> all but the current thread. Something like that. > >> > >> > >> > >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= > >> rk=2C then ForkBeforeExec > >> might get away with just fork() and ignore all this. > >> So far we have not drawn this line -- there is just the one fork and it is = > >> used prior to exec or do-more-work > >> and does the same thing w/o knowing what is to follow. > >> > >> > >> The Posix documentation I think describes this all fairly well. > >> Bing for "posix opengroup pthread_atfork". > >> Really. I think it is saying much of the same thing I am saying=2C but in l= > >> anguage that has received > >> much more thought. Be sure to read the "rationale" part. > >> > >> > >> - Jay > >> > >> > >> From: jay.krell at cornell.edu > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= > >> 2.6-4 > >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > >> > >> > >> > >> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork() > >> Previously: pthreads: only the thread calling fork() survived fork() > >> Now: user threads and pthreads: only the thread calling fork() survives for= > >> k() > >> This must be how pthreads behave=2C and this makes user threads and pthread= > >> s consistent. > >> > >> This change depends on pthread_atfork=2C for pthreads and user threads. > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd. > >> Most programs either never fork=2C or exec almost immediately after fork. > >> > >> > >> pthread_atfork offers a good model. > >> A sort of "distributed" model. > >> You don't have to go and change all the calls to fork(). > >> Each module with a need to do something before/after fork=2C calls the cent= > >> ral pthread_atfork=2C > >> and fork and pthread_atfork cooperate to do what is needed. > >> > >> > >> No function pointer is needed. > >> Instead move the code to m3core/thread. > >> > >> > >> If you really must not use -pthread=2C then you must implement pthread_atfo= > >> rk functionality > >> yourself and have all fork() calls go through your own fork() wrapper that = > >> cooperates > >> with your pthread_atfork replacement. > >> There is no free lunch -- there is a downside to this approach=2C as plain = > >> fork() calls > >> are ok and correct if pthread_atfork is used=2C but now become incorrect. > >> Pick your poison: > >> user thread/pthread inconsistency > >> cvsupd incompatibility with pthreads=20 > >> user threads using pthread_atfork/-pthread=20 > >> fork() calls having to go through a wrapper (ok -- you could miss this an= > >> d=20 > >> not likely notice -- only fork() calls in fork+do-more-work programs nee= > >> d the wrapper).=20 > >> > >> > >> (Most of this has been explained multiple times=2C but people only pay atte= > >> ntion > >> when they think it affects them. I'm guilty of the same thing.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu > >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >>> From: mika at async.caltech.edu > >>> CC: m3devel at elegosoft.com > >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >> c-2.6-4 > >>> =20 > >>> =20 > >>> If you or Tony could describe roughly what you think needs to be done > >>> I'd be happy to look into it. > >>> =20 > >>> The basic problem is that a decision that's made/described in > >>> m3core/thread/m3makefile needs to someone find its way to controlling > >>> what C code gets compiled elsewhere in the system. Or maybe there should > >>> be an indirection into the thread library to pick up pthread_atfork (or n= > >> ot). > >>> =20 > >>> But then again you still haven't explained why you made the user threads = > >> use > >>> pthread_atfork. I just remove it from my installations=2C but then again= > >> I'm > >>> not trying to run CVSup so I don't know if it breaks that program to do s= > >> o. > >>> =20 > >>> Mika > >>> =20 > >>> Jay K writes: > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= > >> ry =3D > >>>> Unix except > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>> > >>>> =3D20 > >>>> Ok=3D2C "we" should probably go ahead and fix that. > >>>> I'll try to=3D2C but no promises=3D2C sorry. > >>>> =3D20 > >>>> - Jay > >>>> =3D20 > >>>>> From: hosking at cs.purdue.edu > >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >>>>> To: mika at async.caltech.edu > >>>>> CC: m3devel at elegosoft.com > >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >> 2C glib=3D > >>>> c-2.6-4 > >>>>> =3D20 > >>>>> Probably unnecessary=3D2C given that I think there is another entry po= > >> int t=3D > >>>> o forking a process (I forget where) in the thread-specific portion of m= > >> 3co=3D > >>>> re. In which case the necessary work might be done there? > >>>>> =3D20 > >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >>>>> =3D20 > >>>>>> Tony Hosking writes: > >>>>>> ... > >>>>>>> pthread_atfork should not be needed under user threads. > >>>>>> ... > >>>>>> =3D20 > >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= > >> ver=3D > >>>> y Unix except > >>>>>> FreeBSD before 6. The comment is from the checked-in source file. > >>>>>> =3D20 > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>>>> =3D20 > >>>>>> INTEGER > >>>>>> __cdecl > >>>>>> RTProcess__RegisterForkHandlers( > >>>>>> ForkHandler prepare=3D2C > >>>>>> ForkHandler parent=3D2C > >>>>>> ForkHandler child) > >>>>>> { > >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. > >>>>>> * Win32 lacks pthread_atfork and fork. OK. > >>>>>> * > >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement > >>>>>> * atfork ourselves=3D2C as long as we provide a fork() > >>>>>> * wrapper that code uses. > >>>>>> */ > >>>>>> #if defined(_WIN32) \ > >>>>>> || defined(__vms) \ > >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >>>>>> return 0=3D3B > >>>>>> #else > >>>>>> while (1) > >>>>>> { > >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >>>>>> if (i !=3D3D EAGAIN) > >>>>>> return i=3D3B > >>>>>> sleep(0)=3D3B > >>>>>> } > >>>>>> #endif > >>>>>> } > >>>>> =3D20 > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >> nds
=3D > >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >> This can be=3D > >>>> fixed if need be.
>=3D3B >=3D3B */

> >>>>  =3D3B
> >>>> Ok=3D2C "we" should probably go ahead and fix that.
> >>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >>>>  =3D3B
> >>>>  =3D3B- Jay
 =3D3B
> >>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >> 011 18:11=3D > >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >> @elegos=3D > >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >> C kerne=3D > >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >> =3D2C given =3D > >>>> that I think there is another entry point to forking a process (I forget= > >> wh=3D > >>>> ere) in the thread-specific portion of m3core. In which case the necessa= > >> ry =3D > >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >> =3D2C at 2:45=3D > >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >> ing writes:=3D > >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >> ould not be n=3D > >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >>
>=3D3B =3D > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >> rom the=3D > >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >> OTE: Even u=3D > >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >> s.
&=3D > >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >>
>=3D3B =3D > >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >> gt=3D3B >=3D3B =3D > >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >> e=3D2C >>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >> ler child) >>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >> hread_atfork.=3D > >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >> _atfork=3D > >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >> ead_atf=3D > >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >> =3D2C for all =3D > >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >> elves=3D2C =3D > >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >> uses. >>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >> =3D3B >=3D3B =3D > >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >> p=3D3B&am=3D > >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >> R>>=3D3B >=3D > >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >> =3D3B >=3D3B in=3D > >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >> B >=3D3B if (=3D > >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > >> sleep(0)=3D3B >>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >> BR>>=3D3B
=3D > >>>> > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >> > >> > >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= > >> er threads.


Clarification: There's no change described there for= > >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= > >> epend on using pthread_atfork. That is just
how pthreads are -- except o= > >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= > >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= > >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= > >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= > >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= > >> t we should provide unconditionally.
If you really want "forkall"=2C the= > >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= > >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= > >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= > >> t anyone to ever do that.



The need for pthread_atfork is sor= > >> t of different for user threads vs. pthreads=2C but it is needed either way= > >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= > >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= > >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= > >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = > >> process
attempts to use some of the locks that were held in the parent= > >> =2C and only the forking thread survived=2C
then the other threads will = > >> never leave their locks=2C and later attempts to acquire them in the childr= > >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = > >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= > >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= > >> n the
parent and children=2C release all locks. Therefore=2C you have to= > >> come up with a locking order.



Things are a little different= > >> for user threads.
In user threads=2C the existance of threads is establ= > >> ished by virtue of a global array or list of data describing the threads. >> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= > >> eduler picks another thread from the global
array/list. Previously=2C th= > >> e data went unchanged when fork() was called. Therefore all threads survive= > >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= > >> s reinitialize the various data=2C so as to forget about
all but the cur= > >> rent thread. Something like that.



I suspect if we had RTProc= > >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= > >> br>might get away with just fork() and ignore all this.
So far we have n= > >> ot drawn this line -- there is just the one fork and it is used prior to ex= > >> ec or do-more-work
and does the same thing w/o knowing what is to follow= > >> .


The Posix documentation I think describes this all fairly well= > >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= > >> ying much of the same thing I am saying=2C but in language that has receive= > >> d
much more thought. Be sure to read the "rationale" part.


&n= > >> bsp=3B- Jay


From: jay.krell at cornell.edu<= > >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = > >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= > >> : Tue=2C 19 Apr 2011 15:06:38 +0000

> >> > >> >> "> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork()
Previously: pthrea= > >> ds: only the thread calling fork() survived fork()
Now: user threads and= > >> pthreads: only the thread calling fork() survives fork()
This must be h= > >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >> br> > >> This change depends on pthread_atfork=2C for pthreads and user threads.
= > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >> tributed" model.
You don't have to go and change all the calls to fork()= > >> .
Each module with a need to do something before/after fork=2C calls the= > >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >> hat is needed.


No function pointer is needed.
Instead move th= > >> e code to m3core/thread.


If you really must not use -pthread=2C = > >> then you must implement pthread_atfork functionality
yourself and have a= > >> ll fork() calls go through your own fork() wrapper that cooperates
with = > >> your pthread_atfork replacement.
There is no free lunch -- there is a do= > >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread >> r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >> s this and
 =3B =3B not likely notice -- only fork() calls in f= > >> ork+do-more-work programs need the wrapper).


(Most of this has = > >> been explained multiple times=2C but people only pay attention
when they= > >> think it affects them. I'm guilty of the same thing.)


 =3B-= > >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >> you or Tony could describe roughly what you think needs to be done
>= > >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >> e needs to someone find its way to controlling
>=3B what C code gets c= > >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >> irection into the thread library to pick up pthread_atfork (or not).
>= > >> =3B
>=3B But then again you still haven't explained why you made the = > >> user threads use
>=3B pthread_atfork. I just remove it from my instal= > >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >> know if it breaks that program to do so.
>=3B
>=3B Mika
= > >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >> . In which case the necessary work might be done there?
>=3B >=3B>= > >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >> =3B >=3B>=3B pthread_atfork should not be needed under user threads. >>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > >> The following code from RTProcessC.c ensures that it is neeeded on e=3D >>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >> n the thread-specific portion of m3core. In which case the necessary =3D >>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Apr 21 04:02:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:02:47 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: <20110421020247.31A0A1A2078@async.async.caltech.edu> Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From mika at async.caltech.edu Thu Apr 21 04:53:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:53:04 -0700 Subject: [M3devel] wrapping postgresql library not enough Message-ID: <20110421025304.685501A2078@async.async.caltech.edu> Hi again, So, as I described before, I wrote a program that wraps all of Critical Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency at which my application locks up, but it has not eliminated the lockups. User threads, AMD64_LINUX. I'm not quite sure what to try next. How can I find out where malloc and free might be used reentrantly, assuming that is the problem? There may not be any reliable threading in CM3 anymore. Possibly it's reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. Mika (gdb) where #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 #3 0x000000000077b08d in PQsendQuery () #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 ) at ../AMD64_LINUX/PQSchedulerWrap.m3:210 #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 ) at ../src/UnsafeDatabase.m3:92 #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 ) at ../src/UnsafeDatabase.m3:239 #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 ) at ../src/UnsafeDatabase.m3:269 #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 ) at ../src/DesynchronizedDB.m3:74 #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () (gdb) From hosking at cs.purdue.edu Thu Apr 21 05:12:53 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:12:53 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421025304.685501A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> Message-ID: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Can you dump all the user threads at this point? ThreadPosix__DumpEverybody() should do it. We want to see where the other threads might be. On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > Hi again, > > So, as I described before, I wrote a program that wraps all of Critical > Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / > Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency > at which my application locks up, but it has not eliminated the lockups. > > User threads, AMD64_LINUX. > > I'm not quite sure what to try next. How can I find out where malloc > and free might be used reentrantly, assuming that is the problem? > > There may not be any reliable threading in CM3 anymore. Possibly it's > reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. > > Mika > > (gdb) where > #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 > #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 > #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 > #3 0x000000000077b08d in PQsendQuery () > #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 > ) > at ../AMD64_LINUX/PQSchedulerWrap.m3:210 > #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 > ) at ../src/UnsafeDatabase.m3:92 > #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 > ) at ../src/UnsafeDatabase.m3:239 > #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 > ) at ../src/UnsafeDatabase.m3:269 > #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 > ) at ../src/DesynchronizedDB.m3:74 > #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > (gdb) From mika at async.caltech.edu Thu Apr 21 05:20:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 20:20:44 -0700 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Message-ID: <20110421032044.9BDAA1A2078@async.async.caltech.edu> Aha, I will try that if it happens again. I wrapped more things that I think might be iffy in my program. But I have a feeling ThreadPosix__DumpEverybody won't work... the application is probably deadlocked by having a single thread locking against itself with some sort of pthread lock. I should look at the source code of free. Mika Tony Hosking writes: >Can you dump all the user threads at this point? = >ThreadPosix__DumpEverybody() should do it. We want to see where the = >other threads might be. > >On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > >> Hi again, >>=20 >> So, as I described before, I wrote a program that wraps all of = >Critical >> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >frequency >> at which my application locks up, but it has not eliminated the = >lockups. >>=20 >> User threads, AMD64_LINUX. >>=20 >> I'm not quite sure what to try next. How can I find out where malloc >> and free might be used reentrantly, assuming that is the problem? >>=20 >> There may not be any reliable threading in CM3 anymore. Possibly it's >> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>=20 >> Mika >>=20 >> (gdb) where >> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >> #3 0x000000000077b08d in PQsendQuery () >> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >access memory at address 0x7f7b848a4298 >> ) >> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >memory at address 0x7f7b848a4328 >> ) at ../src/UnsafeDatabase.m3:92 >> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >memory at address 0x7f7b848a43f8 >> ) at ../src/UnsafeDatabase.m3:239 >> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >memory at address 0x7f7b848a4528 >> ) at ../src/UnsafeDatabase.m3:269 >> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >access memory at address 0x7f7b848a45e8 >> ) at ../src/DesynchronizedDB.m3:74 >> #9 0x00000000005aeac6 in = >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >(finalAttempt=3DCannot access memory at address 0x7f7b848a463f >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >(cl=3DCannot access memory at address 0x7f7b848a4778 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >> #11 0x00000000005b171e in = >DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >at address 0x7f7b848a4958 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >(cl=3DCannot access memory at address 0x7f7b848a4ba8 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >(cl=3DCannot access memory at address 0x7f7b848a4d58 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >../src/thread/POSIX/ThreadPosix.m3:1174 >> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >> #16 0x0000000000000000 in ?? () >> (gdb)=20 From hosking at cs.purdue.edu Thu Apr 21 05:46:37 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:46:37 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421032044.9BDAA1A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> <20110421032044.9BDAA1A2078@async.async.caltech.edu> Message-ID: <2EDBC39D-053F-4C7A-AD64-7D202E080581@cs.purdue.edu> If you are able to get the debugger to stop there then you should still be able to dump the threads from the debugger: call ThreadPosix__DumpEverybody() Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 11:20 PM, Mika Nystrom wrote: > Aha, I will try that if it happens again. I wrapped more things that I think > might be iffy in my program. > > But I have a feeling ThreadPosix__DumpEverybody won't work... the > application is probably deadlocked by having a single thread locking > against itself with some sort of pthread lock. I should look at the > source code of free. > > Mika > > Tony Hosking writes: >> Can you dump all the user threads at this point? = >> ThreadPosix__DumpEverybody() should do it. We want to see where the = >> other threads might be. >> >> On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: >> >>> Hi again, >>> =20 >>> So, as I described before, I wrote a program that wraps all of = >> Critical >>> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >>> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >> frequency >>> at which my application locks up, but it has not eliminated the = >> lockups. >>> =20 >>> User threads, AMD64_LINUX. >>> =20 >>> I'm not quite sure what to try next. How can I find out where malloc >>> and free might be used reentrantly, assuming that is the problem? >>> =20 >>> There may not be any reliable threading in CM3 anymore. Possibly it's >>> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>> =20 >>> Mika >>> =20 >>> (gdb) where >>> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >>> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >>> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >>> #3 0x000000000077b08d in PQsendQuery () >>> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >> access memory at address 0x7f7b848a4298 >>> ) >>> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >>> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >> memory at address 0x7f7b848a4328 >>> ) at ../src/UnsafeDatabase.m3:92 >>> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >> memory at address 0x7f7b848a43f8 >>> ) at ../src/UnsafeDatabase.m3:239 >>> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >> memory at address 0x7f7b848a4528 >>> ) at ../src/UnsafeDatabase.m3:269 >>> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >> access memory at address 0x7f7b848a45e8 >>> ) at ../src/DesynchronizedDB.m3:74 >>> #9 0x00000000005aeac6 in = >> DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >> (finalAttempt=3DCannot access memory at address 0x7f7b848a463f >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >>> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >> (cl=3DCannot access memory at address 0x7f7b848a4778 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >>> #11 0x00000000005b171e in = >> DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >> at address 0x7f7b848a4958 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >>> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >> (cl=3DCannot access memory at address 0x7f7b848a4ba8 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >>> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >> (cl=3DCannot access memory at address 0x7f7b848a4d58 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >>> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >> ../src/thread/POSIX/ThreadPosix.m3:1174 >>> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >>> #16 0x0000000000000000 in ?? () >>> (gdb)=20 From hendrik at topoi.pooq.com Thu Apr 21 15:19:04 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 09:19:04 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: <20110421131904.GB22222@topoi.pooq.com> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > Agreed. But CVSup used it and we were trying to be supportive. So, at present, CVSup presumably works with user threads. Maybe the cure is to fix CVSup instead of fork(). Why does CVSup needs a fork() that preserves all threads? Is it essential to its design, or incidental? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:20:54 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:20:54 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421131904.GB22222@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> Message-ID: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >> Agreed. But CVSup used it and we were trying to be supportive. > > So, at present, CVSup presumably works with user threads. > Maybe the cure is to fix CVSup instead of fork(). > > Why does CVSup needs a fork() that preserves all threads? Is > it essential to its design, or incidental? CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > -- hendrik From hendrik at topoi.pooq.com Thu Apr 21 16:28:26 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:28:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> Message-ID: <20110421142826.GA23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: > > On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > > > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > >> Agreed. But CVSup used it and we were trying to be supportive. > > > > So, at present, CVSup presumably works with user threads. > > Maybe the cure is to fix CVSup instead of fork(). > > > > Why does CVSup needs a fork() that preserves all threads? Is > > it essential to its design, or incidental? > > CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. Does it need to use fork() at all? Wuld it suffice to use Modula 3's own thread system? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:30:25 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:30:25 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421142826.GA23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: I'm not familiar with the architecture of CVSup. On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >> >> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >> >>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>> Agreed. But CVSup used it and we were trying to be supportive. >>> >>> So, at present, CVSup presumably works with user threads. >>> Maybe the cure is to fix CVSup instead of fork(). >>> >>> Why does CVSup needs a fork() that preserves all threads? Is >>> it essential to its design, or incidental? >> >> CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > Does it need to use fork() at all? Wuld it suffice to use Modula 3's > own thread system? > > -- hendrik From wagner at elegosoft.com Thu Apr 21 16:47:33 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 16:47:33 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Quoting Tony Hosking : > I'm not familiar with the architecture of CVSup. > > On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > >> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>> >>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>> >>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>> >>>> So, at present, CVSup presumably works with user threads. >>>> Maybe the cure is to fix CVSup instead of fork(). >>>> >>>> Why does CVSup needs a fork() that preserves all threads? Is >>>> it essential to its design, or incidental? >>> >>> CVSup uses fork in an ill-defined way. fork is only well-defined >>> for use as fork+exec. So, the fact that there are continuing >>> threads is immaterial. CVSup wants to continue doing real work >>> (without exec) after fork. >> >> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >> own thread system? Not for real word server applications, where one problem in a user session would crash dozens of other sessions. At least this was what John Polstra reported. The additional safety of separate address spaces _has_ its advantages. I thought CVSup had been changed to work with pthreads though in CM3? Or do I misremember? Anyway, this shouldn't be our most important problem, but rather make System pthreads and also M3 user threads work in a reliable and fault-free way again. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hendrik at topoi.pooq.com Thu Apr 21 16:54:27 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:54:27 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Message-ID: <20110421145427.GC23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: > Quoting Tony Hosking : > >> I'm not familiar with the architecture of CVSup. >> >> On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: >> >>> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>>> >>>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>>> >>>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>>> >>>>> So, at present, CVSup presumably works with user threads. >>>>> Maybe the cure is to fix CVSup instead of fork(). >>>>> >>>>> Why does CVSup needs a fork() that preserves all threads? Is >>>>> it essential to its design, or incidental? >>>> >>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>> for use as fork+exec. So, the fact that there are continuing >>>> threads is immaterial. CVSup wants to continue doing real work >>>> (without exec) after fork. >>> >>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>> own thread system? > > Not for real word server applications, where one problem in a user > session would crash dozens of other sessions. At least this was what > John Polstra reported. The additional safety of separate address > spaces _has_ its advantages. > > I thought CVSup had been changed to work with pthreads though in CM3? > Or do I misremember? > > Anyway, this shouldn't be our most important problem, but rather make > System pthreads and also M3 user threads work in a reliable and > fault-free way again. Once our only problems are with CVSup's abuse of fork(), I'd consider that done. -- hendrik From mika at async.caltech.edu Thu Apr 21 17:07:48 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 21 Apr 2011 08:07:48 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421150748.62E2B1A2078@async.async.caltech.edu> Hendrik Boom writes: ... >>>>> >>>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>>> for use as fork+exec. So, the fact that there are continuing >>>>> threads is immaterial. CVSup wants to continue doing real work >>>>> (without exec) after fork. >>>> >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real word server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > >Once our only problems are with CVSup's abuse of fork(), I'd >consider that done. Unfortunately, no. CVSup's ab/use of fork() only affects some software-engineering issues at the margin of the threading problem. It does do bad things to portability to have to support it. The Modula-3 way of doing what CVSup does is to use Process.Create to create a new process. If there is state in the current process that needs to be transmitted to the new process, Pickle the state and send it over a Pipe into the new process. Using fork() to do a kind of pseudo-shared-memory should not be necessary. I think. Mika From wagner at elegosoft.com Thu Apr 21 17:14:24 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 17:14:24 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Quoting Hendrik Boom : > On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: [...] >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real world server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > > Once our only problems are with CVSup's abuse of fork(), I'd > consider that done. I think Mika has pointed out quite a few problems that haven't been solved yet (or I missed something, which may well be); he even contributed a sophisticated test program for validation of threads. If it hasn't been done yet, CVSup can probably easily be adapted to other thread-survival-during-fork patterns; we just should be able to provide a consistent specification and implementation for all platforms, for both threads and process semantics. However, I'll leave the office now for Easter holidays, and wish you all sunny and restful Easter days! Olaf PS: I'm still surprised that none of those recent problems has found its way into our fault tracker... -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 21 18:27:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 21 Apr 2011 17:27:45 +0100 (BST) Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421151025.A8AC31A2078@async.async.caltech.edu> Message-ID: <980045.69138.qm@web29713.mail.ird.yahoo.com> Hi all: sure I can actually it work ?s with WeakRef, somehow interesting, is there a m3-db/m3postgresql or ? Besides the compilation problems, zeus3D package misses sources of at least 4 interface files and its not public dependences: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/src/ Aux, Camera, G, Light perhaps they were exported as hide_Interface() with its own modules if so. They are a kind of helper or friend modules ti it, so they are missing in public three there. There is also a local copy of Obliq and Modula-3 packages of modules of models of lego tools, so it might be interesting for somebody to play with them, it's fairly a good size library written by an old M3 friend :), as are the algorithm animations done for zeus3D at DEC SRC. There is another one built package to animate distributed protocols called Zada, it might be interesting to plug it all, even there is a formal specification of the same problem, a Manufacturing Cell, local projects name was Didactic Manufacturing Cell thesis (all sources available for free just that Obliq is not working as far I can tell) Zada related relevant links (missing main web page is not under development any more, maybe we should ask them what they have about it and perhaps even their thesis) they had even audio: http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-agarwal.html http://ls4-www.cs.tu-dortmund.de/RVS/Exports/EB_PG225.ps.Z http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-kukasch.html So its would a good package animation example of lego tools, specifications, and protocols of all the framework, don't you think? Thanks in advance PD: just if you care of and have at 2 PM ET today there is a conference on embedded software verification, so it might be good to see what this guys are looking for, maybe we can give them a look of it? http://www.eetimes.com/design/eda-design/4215099/Using-verification-coverage-with-formal-analysis --- El jue, 21/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] wrapping postgresql library not enough > Para: "Daniel Alejandro Benavides D." > Fecha: jueves, 21 de abril, 2011 10:10 > Hi Daniel, > > Sure I'd be happy to look at it (although I *may* already > have fixed my > problems, I don't know). Would it be OK to add the > PSQL driver you are talking > about to CM3? > > Can you give me a filename of some file you think would be > in zeus3d and I can > go looking in all my files for it? > > Mika > > "Daniel Alejandro Benavides D." writes: > >Hi Mika: > >I have a local copy of Daniel Solaz's 2001 PSQL driver, > if needed, just in = > >case you want it I can send right away to you. > >Besides: Did you know or know how to find out zeus3d > sources besides the pu= > >blic package? > > > >Thanks in advance > > > >--- El mi=E9, 20/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] wrapping postgresql library > not enough > >> Para: "Tony Hosking" > >> CC: m3devel at elegosoft.com > >> Fecha: mi=E9rcoles, 20 de abril, 2011 22:20 > >> Aha, I will try that if it happens > >> again. I wrapped more things that I think > >> might be iffy in my program. > >>=20 > >> But I have a feeling ThreadPosix__DumpEverybody > won't > >> work... the > >> application is probably deadlocked by having a > single > >> thread locking > >> against itself with some sort of pthread > lock. I > >> should look at the > >> source code of free. > >>=20 > >> Mika > >>=20 > >> Tony Hosking writes: > >> >Can you dump all the user threads at this > point?=20 > >> =3D > >> >ThreadPosix__DumpEverybody() should do > it. We > >> want to see where the =3D > >> >other threads might be. > >> > > >> >On Apr 20, 2011, at 10:53 PM, Mika Nystrom > wrote: > >> > > >> >> Hi again, > >> >>=3D20 > >> >> So, as I described before, I wrote a > program that > >> wraps all of =3D > >> >Critical > >> >> Mass's "postgresql" library (PQ.i3) with > >> Scheduler.DisableSwitching / > >> >> Scheduler.EnableSwitching. Yes, it > seems to > >> have reduced the =3D > >> >frequency > >> >> at which my application locks up, but it > has not > >> eliminated the =3D > >> >lockups. > >> >>=3D20 > >> >> User threads, AMD64_LINUX. > >> >>=3D20 > >> >> I'm not quite sure what to try > next. How can > >> I find out where malloc > >> >> and free might be used reentrantly, > assuming that > >> is the problem? > >> >>=3D20 > >> >> There may not be any reliable threading > in CM3 > >> anymore. Possibly it's > >> >> reliable on FreeBSD (without -pthread) > but I > >> wouldn't bet money on it. > >> >>=3D20 > >> >> Mika > >> >>=3D20 > >> >> (gdb) where > >> >> #0 0x00007f7a8da0102e in ?? () > from > >> /lib/libc.so.6 > >> >> #1 0x00007f7a8d99eb43 in ?? () > from > >> /lib/libc.so.6 > >> >> #2 0x00007f7a8d99baab in free () > from > >> /lib/libc.so.6 > >> >> #3 0x000000000077b08d in > PQsendQuery () > >> >> #4 0x0000000000767c12 in > >> PQSchedulerWrap__PQsendQuery (conn=3D3DCannot =3D > >> >access memory at address 0x7f7b848a4298 > >> >> ) > >> >> at > >> ../AMD64_LINUX/PQSchedulerWrap.m3:210 > >> >> #5 0x0000000000763ce0 in > >> UnsafeDatabase__MyExec (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4328 > >> >> ) at ../src/UnsafeDatabase.m3:92 > >> >> #6 0x0000000000764a29 in > >> UnsafeDatabase__ExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a43f8 > >> >> ) at ../src/UnsafeDatabase.m3:239 > >> >> #7 0x0000000000764cee in > >> UnsafeDatabase__TExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4528 > >> >> ) at ../src/UnsafeDatabase.m3:269 > >> >> #8 0x0000000000756638 in > >> DesynchronizedDB__TransSExec (trans=3D3DCannot > =3D > >> >access memory at address 0x7f7b848a45e8 > >> >> ) at ../src/DesynchronizedDB.m3:74 > >> >> #9 0x00000000005aeac6 in =3D > >> > >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 > >> =3D > >> >(finalAttempt=3D3DCannot access memory at > address > >> 0x7f7b848a463f > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > >> >> #10 0x00000000005ae639 in > >> DBTable_gcoms_ordr_statusMonitor__ScanDirty =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4778 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > >> >> #11 0x00000000005b171e in =3D > >> >DBTable_gcoms_ordr_statusMonitor__CheckForNew > >> (cl=3D3DCannot access memory =3D > >> >at address 0x7f7b848a4958 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > >> >> #12 0x00000000005b1382 in > >> DBTable_gcoms_ordr_statusMonitor__Sync =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4ba8 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > >> >> #13 0x00000000005b0e1c in > >> DBTable_gcoms_ordr_statusMonitor__ApplyC =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4d58 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > >> >> #14 0x0000000000922eb5 in > ThreadPosix__RunThread > >> () at =3D > >> >../src/thread/POSIX/ThreadPosix.m3:1174 > >> >> #15 0x00007f7a8d9697b0 in ?? () from > >> /lib/libc.so.6 > >> >> #16 0x0000000000000000 in ?? () > >> >> (gdb)=3D20 > >> > From rcolebur at SCIRES.COM Thu Apr 21 19:48:57 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 13:48:57 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I concur with Mika ! We need to stick to the design tennants of Modula-3. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Wednesday, April 20, 2011 10:03 PM To: Jay K Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From jay.krell at cornell.edu Thu Apr 21 19:47:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 17:47:01 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421150748.62E2B1A2078@async.async.caltech.edu> References: , , <20110420055557.089101A207A@async.async.caltech.edu>, , <20110421131904.GB22222@topoi.pooq.com>, <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu>, <20110421142826.GA23127@topoi.pooq.com>, , <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com>, <20110421145427.GC23127@topoi.pooq.com>, <20110421150748.62E2B1A2078@async.async.caltech.edu> Message-ID: > Unfortunately, no. CVSup's ab/use of fork() only affects some > software-engineering issues at the margin of the threading problem. > It does do bad things to portability to have to support it. > > The Modula-3 way of doing what CVSup does is to use Process.Create to > create a new process. If there is state in the current process that > needs to be transmitted to the new process, Pickle the state and send > it over a Pipe into the new process. Using fork() to do a kind of > pseudo-shared-memory should not be necessary. I think. What cvsup does is more efficient. But I'm not sure if by much. You get a bunch of read-only shared state, including sharing physical memory. Most other forms of inter-process-communication will duplicate the data. What cvsup does is not portable to Windows. And is not efficient on Cygwin. Though it is portable to Interix and efficient there. The NT kernel supports fork() in the manner everyone is familiar with, but Win32 doesn't expose it. Anyway, we still don't know what are the problems currently, and they might exist on Win32 also. Or at least there are problems on Win64. The test program needs to be tried on 32bit Windows. And we need to consider removing use of GetThreadContext, and probably SuspendThread/ResumeThread as well. I understand the -pthread dependency might be undesirable in user threads and it should be easy to remove it, with the tradeoff that direct fork() won't do the right thing (though it will be racy and therefore often appear to work). - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Apr 21 19:35:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 13:35:45 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Message-ID: fork + exec works fine and has always worked fine. fork + carry on is what Jay fixed with pthread_atfork. On Apr 21, 2011, at 11:14 AM, Olaf Wagner wrote: > If it hasn't been done yet, CVSup can probably easily be adapted to > other thread-survival-during-fork patterns; we just should be able > to provide a consistent specification and implementation for all > platforms, for both threads and process semantics. From jay.krell at cornell.edu Thu Apr 21 20:42:21 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 18:42:21 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 21 21:21:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 19:21:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I forgot to reply to this point, which keep coming up: > starts blurring the line between what is and is not Modula-3. This could > turn out to be a problem if Modula-3 is in future again ported to systems > without complete, modern C environments. By "problem" I mean an extra > bifurcation of application programs: some programs for "POSIX Modula-3" Cvsup won't work in such a system without a certain amount of work. Such a system requires a fairly significant amount of work. Nothing discussed here really touches on the work required. Such a system needs threads implemented in *some* fashion. Either pthreads, or sigaltstack, or jmpbuf smashing, or somehow natively in Modula-3, but ultimately, you need assembly and/or processor-specificness *somewhere*. You must save restore registers somewhere. You must setup a stack somewhere. You need preemption somehow (well, maybe not -- "cooperative suspend"). And you need to provide for file I/O somewhere, depending on what sort of program you want to run. And socket I/O, ditto. And keyboard/video/mouse/console functionality, ditto. You can use more or less of the facilities of the underlying system, giving you more or less functionality for free/cheap, obligating you to implement more or less of the functionality you require or desire. Modula-3 layers on top of Posix and Win32, with more or less thickness and state. Modula-3 was kind of interesting in that it provided threads with a certain amount of portability without requiring an underlying threading capability, with just timers for preemption and setjmp/longjmp for context save/restore. This is still somewhat interesting, e.g. maybe for DJGPP, but pthreads and Win32 threads are widespread now, and providing these features isn't particularly valuable or differentiating. You know -- why use Modula-3? One of the reasons in the past was maybe for portable threading. That is still a reason, but portable threading is not as difficult to write for one's self these days, merely by layering over pthreads and Win32. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 00:23:41 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 01:15:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 23:15:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 01:28:59 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 00:28:59 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <284553.92617.qm@web29701.mail.ird.yahoo.com> Hi all: one way of being us safe and capable of doing more than traditional modification of sources each time the system changes (which is basically almost everyone blindly agrees, sorry if I'm mad, but who else aside of us cares about this?) and avoid this dangerous path to near disaster of Modula-3 specifications is to selfhost in our threads. Modula-3 has been almost hard to break system, and even if Modula-3 is modified must remain there, no UNSAFE stuff imploded, no every one in its place, no dangerous threading with locks every 10, 20 30 milliseconds, no more, we are here to say again guys just make things clear and quick and simple, don't make us to change our implementations each time you are making its. I mean Linux KVM, Solaris Zones, Win HV, it isn't that hard to believe it, don't you think? Is about what guys there did with SPIN, I'm going back to that, but is useful for one reason, the only way to keep function stability and I mean that for us in some sense could be that, remember we are able to test stress a VM in such an environment as most as we would want, knowing every fault there is ones besides HyperThreading is making its presence in hardware assisted virtualization? then is a matter of what we want to achieve the next level is up to us. That said originally the DEC Unix already had emulation of threads, of C POSIX threads? (now called pthreads today), of Mach 3.0 layered ones and DEC OSF too and besides and own strends for more than the most of any system can think of. See: http://qstream.org/~krasic/cs508-2006/summaries/paper17/SpinOS.ppt Thanks in advance --- El jue, 21/4/11, Coleburn, Randy escribi?: De: Coleburn, Randy Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "m3devel" Fecha: jueves, 21 de abril, 2011 17:23 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. ?Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.?? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages.? I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible.? ?I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?.? The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika.? It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform.? And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only.? Everything else is completely portable. ?Regards,Randy Coleburn ?From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 ?> I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 05:24:02 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I'm not advocating we "throw the baby out with the bath water", but if the "fork-and-do-more" pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay ________________________________ From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 08:29:08 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 06:29:08 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , , , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , , , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , , , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , , , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , , , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , , , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , ,,, , , ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , ,,, , , , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , , , , , , Message-ID: > As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Exchanging Modula-3 for C has tremendously improved the maintainability, portability, correctness, size-in-source, and correctness-by-inspection of the system. Debuggability has improved as well, though hopefully that will "even out" -- hopefully Modula-3 code will become just as debuggable with stock gdb, and on systems without stabs support. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I?m not advocating we ?throw the baby out with the bath water?, but if the ?fork-and-do-more? pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 22 18:25:36 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 22 Apr 2011 09:25:36 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Message-ID: <20110422162536.E63631A2078@async.async.caltech.edu> Jay K writes: >--_f508cc8d-d8d9-47ec-89e0-fcca258f1fc1_ >Content-Type: text/plain; charset="Windows-1252" >Content-Transfer-Encoding: quoted-printable > > > > As for the use of C vs. Modula-3=2C it obvious that you and I have a dif= >ference of opinion. >Exchanging Modula-3 for C has tremendously improved the maintainability=2C = >portability=2C correctness=2C size-in-source=2C and correctness-by-inspecti= >on of the system. >Debuggability has improved as well=2C though hopefully that will "even out"= > -- hopefully Modula-3 code will become just as debuggable with stock gdb= >=2C and on systems without stabs support. > > - Jay In theory you are right. In practice, almost ten years ago now others at Caltech and I started to try to switch from PM3 to CM3 since it was obvious it was CM3 that was going to be maintained in future. The process still isn't over, as my frequent postings to this list bear witness to. My perspective is really very simple. I've been using Modula-3 as a tool to solve almost all programming problems I've had since about 1999, when I gave up C due to my inability to write a correct C program. I'm a user of the system and only reluctantly a peripheral maintainer. A few of my observations as a user are the following. -- Threads work perfectly in PM3. In CM3 it's very much ??? -- It took five years to get TEXT working acceptably (as in, no stack overflows!) in CM3. Now seems more or less OK, but Text.Hash is still possibly an order of magnitude slower than . -- I have no debugger that works for CM3. m3gdb is *very useful* for the old PM3. No it doesn't work perfectly but you can set breakpoints and print variables but that's what at least I use a debugger for. -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE are very slow (like orders of magnitude slower than for PM3). I looked at it once and realized that someone had walked all over RT0 with very heavy feet. Not a pretty sight, on my (long) to-do list. -- m3tk is broken owing to the introduction of LONGINT. This is not a peripheral issue when a lot of the code you produce is generated from other code (a process that improves programming productivity manyfold---when it works). As with all the other items on this list, m3tk has no trouble parsing the Modula-3 accepted by PM3. -- other LONGINT problems... -- It's taken me literally months of struggle to get my application working with CM3. The old PM3-compiled one traded half a billion dollars worth of stocks last year without a single glitch. The CM3-compiled version has been hanging every day except the last (so hopefully it's OK now...???) I'm literally losing sleep over this because it's running during European business hours. Only reason I got sleep last night is because it's Good Friday. Now I'm not blaming anyone in particular for these problems. In fact I think they mostly come from things that were done at Critical Mass, Inc. However there is a common thread. Just about all the issues I've run into have had *something* to do with someone's attempt to make Modula-3 more of "all things to all people" than it was. Its main strength was and is that it is a programming language with almost equal expressibility (both at the machine and data-structure level) as C++ with a *fifty* page definition in reasonably clear English---that's just a few pages longer than R5RS and in a bigger font to boot. Portability, library support, multicore use, comprehensive base datatypes, ... all those things are nice but not part of the "core mission" of the language. It makes sense to try to expand out and be all things to all people if you have the budget and development team for it. As it is now, all the improvements have done is threaten the survival of the programming language for any use whatsoever. Here's an experiment anyone on this list can try. Find a friend who doesn't use Modula-3 but is a computer programmer, software engineer, etc. Ask him to install CM3 and configure it so that the thread testing program works. See how long it takes before he gives up. Mika From jay.krell at cornell.edu Fri Apr 22 21:25:42 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 19:25:42 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: <20110422162536.E63631A2078@async.async.caltech.edu> References: <20110422162536.E63631A2078@async.async.caltech.edu> Message-ID: Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 23:02:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 22:02:28 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: Message-ID: <927697.40402.qm@web29714.mail.ird.yahoo.com> Hi all: if the problems were the thread package then we will be easyly solved for us, but instead the question how much we owe to each platforms depednence part, e.g: Cvsup was created with Pm3 and/or Ezm3, Pm3 with Linux almost perfect coverage of all source of incompatibilities and so, DEC SRC was make it freely available in any site without too much effort, though by doing it In Modula-3 made almost hard to erase fame of hard thing to do. Then Digital split ended with this efforts going on and meanwhile DEC SRC was being distributed as source compatible with pm3, or with cm3 even. Later Cm3 became an open product and everyone kept their objectives so don't think is easy to do that. Instead, there should be an easy way I Think to recover to be in that point. I mean: CM3 -> DEC SRC, DEC SRC -> PM3, and PM3 -> EZM3 -> Cvsup & Spin. I think there should be enough number of similar or almost equal modules same, so most of the time is relinking stuff. I think every one hast its own rights to believe the things they believe. I think what my old M3 Professor told me once before retired, Modula-3 didn't gain more users because the lack of an IDE, today there is not reason to say that, instead the market has grown a lot times faster then why should I believe it doesn't hold true (why we don't have Cm3 running a Cvsup server, why Cm3 is not running a Spin community, why Cm3 is not hosting several projects in web platforms, why Cm3 is not running Obliq3D stuff, and I could say more but but we don't want to be there I guess this is enough too much or little I don't know, we are not there still, Mika's projects, DEC SRC missing packages, zeus3D, etc). Let me know your thoughts and thanks in advance --- El vie, 22/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 22 de abril, 2011 14:25 Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:14:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:14:49 +0000 Subject: [M3devel] pthread_atfork In-Reply-To: References: Message-ID: This is in now. Mika, can you try user threads again, and this time remove -pthread/-pthreads/-lpthread from your config files? Really, if user threads are to be supported, I wish they were easier to use -- no need to rebuild m3core or anything else, and specifiable in link or runtime commads. More details below: From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: pthread_atfork Date: Tue, 19 Apr 2011 06:07:01 +0000 I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: ... manually for userthreads: ... take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 [Jay April 23] done: implemented in C, locking via Disable/EnableSwitching, ok? And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. .. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible [Jay April 23] instead, just not provided This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. [Jay April 23] still a major concern in need of significant investigation - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:19:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:19:25 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? Message-ID: I've asked before. I'm asking again. I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. @M3pthreads @M3userthreads or somesuch. I'd also like to be able to switch the default via a line in a m3makefile that builds a program. I know roughly what implementing this would look like. I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. I'll have to reread. Whatever problem it has, this change probably would too. I'm willing to implement this myself. I'm not asking for anyone else to do it. I understand it makes LOCK harder to efficiently inline. If/when LOCK becomes partly inlined, provide a switch? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sat Apr 23 21:04:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:04:16 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: <13CE2151-2CF8-4773-B32F-91E753256DAA@cs.purdue.edu> The problem is that you are forced to link with pthreads for the whole executable which may cause problems. Not sure why though. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From hosking at cs.purdue.edu Sat Apr 23 21:06:22 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:06:22 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From dabenavidesd at yahoo.es Sun Apr 24 19:51:57 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 18:51:57 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: Message-ID: <423962.44965.qm@web29707.mail.ird.yahoo.com> Hi all: In ref [2] it is claimed implemented stack walking exception handling for Modula-3 in SPINE using GCC 2.96 RT. Can you watch it? http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.8017 As also the strind schedulling for the SMP CPU Thanks in advance [1] D. Gough, S. Drew, K. J. Gough, and J. Ledermann, ?Implementing Zero Overhead Exception Handling,? 1995. [2] M. E. Fiuczynski, ?Safe and efficient resource sharing in component-based systems,? University of Washington, 2004. --- El s?b, 23/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: "Jay K" > CC: "m3devel" > Fecha: s?bado, 23 de abril, 2011 14:06 > Good point about lock. I really think > this is the showstopper because compiled code will be fo a > particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K > wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and > pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a > line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look > like. > > > > > > I am a bit confused as to what problems > -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would > too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently > inline. > > If/when LOCK becomes partly inlined, provide a > switch? > > > > > > Thank you, > > - Jay > > > From dabenavidesd at yahoo.es Sun Apr 24 22:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 21:47:28 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ Message-ID: <319537.44680.qm@web29718.mail.ird.yahoo.com> Hi all: using google found this ftp directory (Modula2+ VAX): ftp://ftp.cisco.com/pub/rfc/DEC If DEC guys allowed to read and copy this is because they worked not just its products, but also their specifications, would be nice to see the Time modules explained in: /TIME_V1_0_0.PS If it's of enterprise and historical (academic) interest language could gain recognition from that, I mean they founded several open source projects back in time, which became later de-facto standards, it would be nice to see if this is of enough interest of their internal sources (can't forget Modula-3 was just one of them). They further developed Modula2+ into Modula2+Epsilon, which could be said to be a Modula3 in house brother if so. Furthermore there is a private source code library of internal design/layout chips components, later things could show they did watch to continue some of their work of prior language(s) or of its younger brother. See: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 Anyway this might be seen as a history on programming languages PhD project or so and computers if so (my CS professor told DEC machines were able to be specified to some large extent by its consumers and it seems by their producers too e.g like VAXstations). I never had the opportunity of doing something like that, perhaps computers have change, or like most people has changed, interesting ... Thanks in advance From dabenavidesd at yahoo.es Mon Apr 25 00:54:26 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 23:54:26 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ In-Reply-To: <319537.44680.qm@web29718.mail.ird.yahoo.com> Message-ID: <34227.88904.qm@web29715.mail.ird.yahoo.com> Hi all: fortunately they had recognition was made by some sue, but for an issue; the claim they created a thing for the development of it: http://www.google.com/patents?id=PY0kAAAAEBAJ&zoom=4&pg=PA1#v=onepage&q&f=false Which is nice given that a patent is hold means that somehow everybody agrees with that I guess, but who *may not* be affected by it, exactly the counterexample of just another read my lips example :) of patents. Thanks in advance --- El dom, 24/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: [M3devel] Architecture descriptions of software/hardware in Modula-2+ > Para: m3devel at elegosoft.com > Fecha: domingo, 24 de abril, 2011 15:47 > Hi all: > using google found this ftp directory (Modula2+ VAX): > > ftp://ftp.cisco.com/pub/rfc/DEC > > If DEC guys allowed to read and copy this is because they > worked not just its products, but also their specifications, > would be nice to see the Time modules explained in: > /TIME_V1_0_0.PS > > If it's of enterprise and historical (academic) interest > language could gain recognition from that, I mean they > founded several open source projects back in time, which > became later de-facto standards, it would be nice to see if > this is of enough interest of their internal sources (can't > forget Modula-3 was just one of them). > > They further developed Modula2+ into Modula2+Epsilon, which > could be said to be a Modula3 in house brother if so. > > Furthermore there is a private source code library of > internal design/layout chips components, later things could > show they did watch to continue some of their work of prior > language(s) or of its younger brother. See: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 > > Anyway this might be seen as a history on programming > languages PhD project or so and computers if so (my CS > professor told DEC machines were able to be specified to > some large extent by its consumers and it seems by their > producers too e.g like VAXstations). I never had the > opportunity of doing something like that, perhaps computers > have change, or like most people has changed, interesting > ... > > Thanks in advance > From rodney_bates at lcwb.coop Tue Apr 26 18:28:34 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 11:28:34 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <20110419115106.7E6A31A207A@async.async.caltech.edu> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> Message-ID: <4DB6F2B2.3090905@lcwb.coop> I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry more quickly than I can. On 04/19/2011 06:51 AM, Mika Nystrom wrote: > Hello m3devel, > > If anyone knows how to fix the following trivially, please go ahead. > > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal > > (errors reported by m3tk). > > The offending line is: > > INTERFACE BasicCtypes; > > IMPORT Word, Long; > > TYPE > (* the four signed integer types *) > signed_char = [-16_7f-1 .. 16_7f]; > short_int = [-16_7fff-1 .. 16_7fff]; > int = [-16_7fffffff-1 .. 16_7fffffff]; > long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; > long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) > > (* the four unsigned integer types *) > unsigned_char = [16_0 .. 16_ff]; > unsigned_short_int = [16_0 .. 16_ffff]; > unsigned_int = [16_0 .. 16_ffffffff]; > unsigned_long_int = Word.T; > unsigned_long_long = Long.T; > > (* the three floating types *) > float = REAL; > double = LONGREAL; > long_double = EXTENDED; > > (* char *) > char = signed_char; > > END BasicCtypes. > > I'll look at it myself, but not immediately (no time). Workaround for now. > > Mika > From jay.krell at cornell.edu Wed Apr 27 09:05:03 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 27 Apr 2011 07:05:03 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: About inlining LOCK I was more echoing your past sentiment than giving my own, and I provided a mitigation: a) specifically the suggestion that the switchability be an option; it'd probably be on by default but people that want to squeeze out the extra performance could enable the inlining and lose the switchability About the need to link to pthread. Good point. I hadn't thought of that. BUT two points in possible rebuttal (not that is necessary an argument! more like a conversation) I'm sure that on some platforms there is no distinction. No separate broken libc.so and working libpthread.so. But yes, I realize the recent talk was about Linux, apparently it is broken. Solaris is probably better. Others I don't know, I'd have to check. Specifically, there should not be two separate fork implementations. Or two separate malloc implementations. The one and only fork must know about pthread_atfork. The one and only malloc must be thread safe, with respect to pthreads. (but not necessarily Modula-3 user-threads, that is our own problem). If that means fork and pthread_atfork have to be in the same .so, and/or that malloc and pthread_* have to be in the same .so, both of which are easy to believe are true, so be it. I shouldn't get one malloc via cc foo.c and a different one via cc foo.c -lpthread. The rare single-threaded process isn't worth optimizing. Isn't worth the headache. As well, though..and this isn't trivial, but I wonder if we should dispense with "static dynamic" linking, where we just call fork, malloc, etc., but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. That might address some of this libc vs. libpthread problem? But then the ABI rather than the source interface becomes needed to be known. Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? Hm. No. Nevermind. What is the problem again with using -pthread on Linux with user threads? So, maybe it pulls in an extra .so and has some small startup cost. Surely it should work? I'll have to dig in to the email and reproduce the bug. I'm of a mind now to remove the recent pthread_atfork stuff. But first I or someone has to understand the problem I was solving. Right now, I don't. The LOCK inlining problem seems very hypothetical right now. We don't have such a feature. Will we really ever? Is pthread_lockmutex partially inlined in C? Or have pthreads generally been implemented well enough that everyone is satisfied with no inlining? Or does Modula-3 sort of promote threads as lighter weight than they are in typical practise? And ditto for locking? You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, I suspect all the commercial Unices, had already presented and reasonably implemented the ideal threading interface, but all the free Unices were still holding out and either not providing pthreads or not providing them well.. at the time Modula-3 was slightly ahead of its time, and one could imagine not using the underlying platform directly because it was too slow or not full featured enough. But hasn't everything just about caught up by now (except OpenBSD with its user pthreads...) (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 years ago circa Solaris 2.10 was pretty modern..) That is -- are pthreads really worth us optimizing beyond? Don't all the C and C++ programmers and systems providers get it by now and have provided almost exactly what we need? Sorry, I'm rambling... Later, - Jay > From: hosking at cs.purdue.edu > Date: Sat, 23 Apr 2011 15:06:22 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look like. > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently inline. > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > Thank you, > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Apr 26 21:50:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 14:50:08 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB6F2B2.3090905@lcwb.coop> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> <4DB6F2B2.3090905@lcwb.coop> Message-ID: <4DB721F0.3010101@lcwb.coop> On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this > bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry > more quickly than I can. > P.S.: Hats off to whoever adapted m3tk for LONGINT. There are places _everywhere_ that needed modification. The whole thing is about as pedantic as code gets. Layers upon layers of linguistic abstractions that do mostly zero semantic abstraction. I traced two declarations through 10 levels each of referrals to other interfaces, all but one of them just renames with the same simple name and different qualifying interfaces, before finally getting to something with some meat. The whole thing smells very strongly of the Ada/Diana culture of ASTs. > On 04/19/2011 06:51 AM, Mika Nystrom wrote: >> Hello m3devel, >> >> If anyone knows how to fix the following trivially, please go ahead. >> >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal >> >> (errors reported by m3tk). >> >> The offending line is: >> >> INTERFACE BasicCtypes; >> >> IMPORT Word, Long; >> >> TYPE >> (* the four signed integer types *) >> signed_char = [-16_7f-1 .. 16_7f]; >> short_int = [-16_7fff-1 .. 16_7fff]; >> int = [-16_7fffffff-1 .. 16_7fffffff]; >> long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; >> long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) >> >> (* the four unsigned integer types *) >> unsigned_char = [16_0 .. 16_ff]; >> unsigned_short_int = [16_0 .. 16_ffff]; >> unsigned_int = [16_0 .. 16_ffffffff]; >> unsigned_long_int = Word.T; >> unsigned_long_long = Long.T; >> >> (* the three floating types *) >> float = REAL; >> double = LONGREAL; >> long_double = EXTENDED; >> >> (* char *) >> char = signed_char; >> >> END BasicCtypes. >> >> I'll look at it myself, but not immediately (no time). Workaround for now. >> >> Mika >> > From hosking at cs.purdue.edu Wed Apr 27 17:20:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 27 Apr 2011 11:20:15 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: On Apr 27, 2011, at 3:05 AM, Jay K wrote: > About inlining LOCK I was more echoing your past sentiment > than giving my own, and I provided a mitigation: > a) specifically the suggestion that the switchability be an option; > it'd probably be on by default but people that want to squeeze > out the extra performance could enable the inlining > and lose the switchability It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > About the need to link to pthread. > Good point. I hadn't thought of that. > BUT two points in possible rebuttal (not that is necessary > an argument! more like a conversation) > I'm sure that on some platforms there is no distinction. > No separate broken libc.so and working libpthread.so. > But yes, I realize the recent talk was about Linux, apparently it is broken. > Solaris is probably better. > Others I don't know, I'd have to check. > > > Specifically, there should not be two separate fork implementations. > Or two separate malloc implementations. > > > The one and only fork must know about pthread_atfork. > The one and only malloc must be thread safe, with respect to pthreads. > (but not necessarily Modula-3 user-threads, that is our own problem). > > > If that means fork and pthread_atfork have to be in the same .so, > and/or that malloc and pthread_* have to be in the same .so, both > of which are easy to believe are true, so be it. > > > I shouldn't get one malloc via cc foo.c and a different > one via cc foo.c -lpthread. > The rare single-threaded process isn't worth optimizing. > Isn't worth the headache. > > > As well, though..and this isn't trivial, but I wonder if we should > dispense with "static dynamic" linking, where we just call fork, malloc, etc., > but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. > That might address some of this libc vs. libpthread problem? > But then the ABI rather than the source interface becomes needed to be known. > Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? > > > Hm. No. Nevermind. > What is the problem again with using -pthread on Linux with user threads? > So, maybe it pulls in an extra .so and has some small startup cost. > Surely it should work? > I'll have to dig in to the email and reproduce the bug. > I'm of a mind now to remove the recent pthread_atfork stuff. > But first I or someone has to understand the problem I was solving. > Right now, I don't. > > > The LOCK inlining problem seems very hypothetical right now. > We don't have such a feature. Will we really ever? > > > Is pthread_lockmutex partially inlined in C? > Or have pthreads generally been implemented well enough that everyone is satisfied > with no inlining? > > > Or does Modula-3 sort of promote threads as lighter weight than they are > in typical practise? And ditto for locking? > > > You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, > I suspect all the commercial Unices, had already presented and reasonably > implemented the ideal threading interface, but all the free Unices were still > holding out and either not providing pthreads or not providing them well.. > at the time Modula-3 was slightly ahead of its time, and one could imagine > not using the underlying platform directly because it was too slow > or not full featured enough. But hasn't everything just about caught up by > now (except OpenBSD with its user pthreads...) > (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 > years ago circa Solaris 2.10 was pretty modern..) > > > That is -- are pthreads really worth us optimizing beyond? > Don't all the C and C++ programmers and systems providers get it by now > and have provided almost exactly what we need? > > > Sorry, I'm rambling... > > > Later, > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Sat, 23 Apr 2011 15:06:22 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > > > Sent from my iPhone > > > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > > > I've asked before. I'm asking again. > > > > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > > @M3pthreads @M3userthreads > > > or somesuch. > > > > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > > > > I know roughly what implementing this would look like. > > > > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > > I'll have to reread. > > > Whatever problem it has, this change probably would too. > > > > > > > > > I'm willing to implement this myself. > > > I'm not asking for anyone else to do it. > > > > > > > > > I understand it makes LOCK harder to efficiently inline. > > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > > > > Thank you, > > > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 27 23:53:31 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 27 Apr 2011 22:53:31 +0100 (BST) Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB721F0.3010101@lcwb.coop> Message-ID: <689426.54766.qm@web29719.mail.ird.yahoo.com> Hi all: how does the arithmetic of LONGREAL affects that of the LONGINT, since both depend on that every BITSIZE should have comparable sizes, shouldn't it? I mean opaque types must be declared in both size LONGINT type and/or INTEGER size(s) to be able to be instantiated from memory in FPU in both cases, I know x87 is bigger size for its integers, besides what happens if the system doesn't provide that, should one consider implementing all (the) type conversions? Here: http://modula3.elegosoft.com/cm3/doc/reference/real-intf.html I think this one lets open the door for such conversions if I may say so: http://modula3.elegosoft.com/cm3/doc/reference/arithmetic.html I know one of the annoyances of old x86 FPU was to make any format conversion to CPU arithmetic thus converting from floats was very onerous in terms of losing your maximized precision in one assignment and bigger performance issues which made impractical for both sides, since initially every cycle was different and so, but I guess all programmer must be aware of that, but if the systems provide better performance like in any reasonable bigger machine, why not? I will give a check of decndumber to seek if they do that at least they had elegant exception handling Thanks in advance --- El mar, 26/4/11, Rodney M. Bates escribi?: > De: Rodney M. Bates > Asunto: Re: [M3devel] LONGINT -> more bugs in m3tk > Para: m3devel at elegosoft.com > Fecha: martes, 26 de abril, 2011 14:50 > > > On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > > I just checked in a patch tp M3CExpValue.m3 in m3tk, > in the head, which I believe will fix this > > bug. I compiled it, but have not tested it, as I am > sure Mika has this case to retry > > more quickly than I can. > > > > P.S.: Hats off to whoever adapted m3tk for > LONGINT. There are places _everywhere_ that needed > modification. The whole thing is about as pedantic as > code gets. Layers upon layers of > linguistic abstractions that do mostly zero semantic > abstraction. I traced two declarations > through 10 levels each of referrals to other interfaces, > all but one of them just renames > with the same simple name and different qualifying > interfaces, before finally getting to > something with some meat. > > The whole thing smells very strongly of the Ada/Diana > culture of ASTs. > > > On 04/19/2011 06:51 AM, Mika Nystrom wrote: > >> Hello m3devel, > >> > >> If anyone knows how to fix the following > trivially, please go ahead. > >> > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,45: type error in arguments to binary operator > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,25: bad numeric literal > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,51: bad numeric literal > >> > >> (errors reported by m3tk). > >> > >> The offending line is: > >> > >> INTERFACE BasicCtypes; > >> > >> IMPORT Word, Long; > >> > >> TYPE > >> (* the four signed integer types *) > >> signed_char = [-16_7f-1 .. 16_7f]; > >> short_int = [-16_7fff-1 .. 16_7fff]; > >> int = [-16_7fffffff-1 .. 16_7fffffff]; > >> long_int = [-16_7fffffffffffffff -1 .. > 16_7fffffffffffffff ]; > >> long_long = [-16_7fffffffffffffffL-1L .. > 16_7fffffffffffffffL]; (** HERE **) > >> > >> (* the four unsigned integer types *) > >> unsigned_char = [16_0 .. 16_ff]; > >> unsigned_short_int = [16_0 .. 16_ffff]; > >> unsigned_int = [16_0 .. 16_ffffffff]; > >> unsigned_long_int = Word.T; > >> unsigned_long_long = Long.T; > >> > >> (* the three floating types *) > >> float = REAL; > >> double = LONGREAL; > >> long_double = EXTENDED; > >> > >> (* char *) > >> char = signed_char; > >> > >> END BasicCtypes. > >> > >> I'll look at it myself, but not immediately (no > time). Workaround for now. > >> > >> Mika > >> > > > From wagner at elegosoft.com Thu Apr 28 09:29:14 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 28 Apr 2011 09:29:14 +0200 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Quoting Tony Hosking : > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > >> About inlining LOCK I was more echoing your past sentiment >> than giving my own, and I provided a mitigation: >> a) specifically the suggestion that the switchability be an option; >> it'd probably be on by default but people that want to squeeze >> out the extra performance could enable the inlining >> and lose the switchability > > It is expected that LOCK will be implemented using CAS (and > equivalents) in the very near future. I have a student working on > it. This will be incompatible with the current user-threads > implementation of LOCK. It will be too painful to support both > using a compiler switch. Couldn't we change the existing user threads to use CAS for locks, too, in order to gain runtime switching of thread implementations? I have neither thought much about it nor looked at the implementation in recent years, so I hope it's no stupid question. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 28 15:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 28 Apr 2011 14:47:28 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: <212562.68766.qm@web29706.mail.ird.yahoo.com> Hi all: yes, that would be awesome ! :) instead of having the threads question of select one be able to switch, we can have sort of multiprocessor implementation capabilities and/or simple multi threading both for free in the RT like for special SPIN, however you should have migration capabilities between both address spaces and sort of a RT executive to control threads of both spaces: http://courses.cs.vt.edu/~cs5204/fall03/Papers/Migration/mathiske95migrating.pdf Similarly it would be good to see how much hard would it be to set distributed applications to get RT easily distributed in m3core, libm3, etc: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.9178 for something like the compiler, type checker, etc So any thread could live in a different machine in desired space --- El jue, 28/4/11, Olaf Wagner escribi?: > De: Olaf Wagner > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: m3devel at elegosoft.com > Fecha: jueves, 28 de abril, 2011 02:29 > Quoting Tony Hosking : > > > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > > > >> About inlining LOCK I was more echoing your past > sentiment > >> than giving my own, and I provided a mitigation: > >> a) specifically the suggestion that the > switchability be an option; > >> it'd probably be on by default > but people that want to squeeze > >> out the extra performance could > enable the inlining > >> and lose the switchability > > > > It is expected that LOCK will be implemented using CAS > (and equivalents) in the very near future. I > have a student working on it. This will be > incompatible with the current user-threads > implementation of LOCK. It will be too painful to > support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for > locks, too, > in order to gain runtime switching of thread > implementations? > I have neither thought much about it nor looked at the > implementation > in recent years, so I hope it's no stupid question. > > Olaf > --Olaf Wagner -- elego Software Solutions GmbH > > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 > 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | > USt-IdNr: DE163214194 > > From hosking at cs.purdue.edu Thu Apr 28 16:22:36 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 28 Apr 2011 10:22:36 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> References: , <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: On Apr 28, 2011, at 3:29 AM, Olaf Wagner wrote: > Quoting Tony Hosking : > >> On Apr 27, 2011, at 3:05 AM, Jay K wrote: >> >>> About inlining LOCK I was more echoing your past sentiment >>> than giving my own, and I provided a mitigation: >>> a) specifically the suggestion that the switchability be an option; >>> it'd probably be on by default but people that want to squeeze >>> out the extra performance could enable the inlining >>> and lose the switchability >> >> It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for locks, too, > in order to gain runtime switching of thread implementations? > I have neither thought much about it nor looked at the implementation > in recent years, so I hope it's no stupid question. Hmm. Yes, I suppose we could. It might degrade performance of the user-level (uniprocessor) threads implementation. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > From rodney_bates at lcwb.coop Fri Apr 1 19:43:25 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 01 Apr 2011 12:43:25 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: <20110305180558.451421A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu> Message-ID: <4D960EBD.2090206@lcwb.coop> I have looked at this a bit. I do not get this symptom on LINUXLIBC6, using the Cartesian product of release/head for m3gdb and the compiler. However, I see a lot of broken stuff that used to work. In all the cases I have seen, it correlates to debugging code compiled by the head compiler, while the release and head versions of m3gdb make no difference to the symptoms in anything I have tried. I believe Jay had started updating the m3gdb in head to use a newer gdb. I am not sure how far that got. Its source code is vastly different, and most of the other language support in gdb has been ripped out. There is quite a lot of change in the stabs info generated by the two compiler versions. The existing way for a long time has been that this is constructed during the first pass over the input to cm3cg, before anything other than initial tree building is done. This is a very limited strategy, because cm3cg can change things later, and this will not be reflected in the stabs info. Debugging anything involving static links has been broken for a long time, because tree-nested.c in cm3cg does some very convoluted stuff with the way static ancestor activation records are accessed, and I have not been able to see an easy way to get stabs info to reflect it. I have thought for a long time that we really need to go to a version of dwarf for debug info. stabs is very limited, and there is a lot of info in the stabs output that is not really stabs, it's just kludgily coded values crammed inside various fields of stabs that the compiler and m3gdb both agree on (well, they used to.) Maybe the current head cm3cg can be fixed, but I know Jay has made big changes in where type information is actually developed, so it may not be worth it. After all the work I have done on this to get m3gdb to where it was, I am unenthusiastic about a bunch more work just to get back to where we were. I have long lists of improvements to make to m3gdb, and it's hard to go back that way. And, It's all in C :-(. If I put much effort into fixing m3gdb and its support in cm3cg at this point, it will probably be a rework to use dwarf. That is likely a big job too, but at least more rewarding and likely to lead to better m3gdb function in the end. For now, it you want a working m3gdb, you really have to use the release compiler. On 03/05/2011 12:05 PM, Mika Nystrom wrote: > Hi m3devel, > > I don't know what's happened here. m3gdb used to work, and work fairly > well (never perfectly, but...). Now I'm trying to debug a program and > I get the following: > > (m3gdb) break Main.m3:65 > Debug info for file "Main.mc" not in stabs format > (m3gdb) > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > systems too. > > Everything is from the CVS head. > > Main was compiled as follows, says cm3 -commands: > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ ci > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > I'm not even sure where to begin looking. Is it a configuration issue, > something broken in the compiler, or what? > > I fear m3gdb is essentially useless at the moment. > > Mika > From jay.krell at cornell.edu Sat Apr 2 03:26:39 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 01:26:39 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <4D960EBD.2090206@lcwb.coop> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: I really tried not to break anything in the wierdo stubs/m3gdb stuff. The work I have been doing to properly declare types is I believe disabled currently. As types become more correct, things start breaking. Either because the optimizer is more aggressive, or because more sanity checks can be done by the compiler, and we don't give it particularly good trees. In most things I've looked it, it's really not any mystery what is wrong with our trees btw. It's not really generally the fault of gcc being obscure or changing. Though perhaps to some extent that is true. Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not accessible from C source, and therefore bugs in them go undetected. It's more generally clearly not great stuff in cm3cg. The main bad thing is that field access is done by just adding to base pointers and casting away. Note that stabs is not supported on Darwin. Nor as I recall HP-UX/HPPA64. All the more reason to no do things this way. Note furthermore that discussion here is often confused. Really no particular debug-info generation code should be cm3cg. In our part -- m3cg/parse.c Neither stabs nor dwarf nor coff nor anything else. The right approach is to build an intermediate form with reasonably good types. The backend handles it from there, whatever debug-info format is requested on the command line. This also enables using stock/current gdb, instead of hacked up/old gdb. I made no visible progress upgrading our fork of gdb. I'd rather just have it go away. I understand, that custom information is being tunneled through to m3gdb. But that "custom information" is largely just basic type information, that any decent C compiler/debug-info/debugger can handle. Someone should investigate if we really pass stuff through that can't be well represented in gcc's type information. For example: sub ranges? sets? enums? packed? Some higher level description of "methods" (really -- whatever C++ does should suffice; if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) At some point the new type information was really breaking stuff so I think I disabled it. The system is kind of incomplete and fragile and the way forward is not always easy or possible in small steps. :( - Jay > Date: Fri, 1 Apr 2011 12:43:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > using the Cartesian product of release/head for m3gdb and the compiler. > > However, I see a lot of broken stuff that used to work. In all the cases > I have seen, it correlates to debugging code compiled by the head compiler, > while the release and head versions of m3gdb make no difference to the > symptoms in anything I have tried. > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > I am not sure how far that got. Its source code is vastly different, and > most of the other language support in gdb has been ripped out. > > There is quite a lot of change in the stabs info generated by the two > compiler versions. The existing way for a long time has been that this > is constructed during the first pass over the input to cm3cg, before > anything other than initial tree building is done. This is a very > limited strategy, because cm3cg can change things later, and this will > not be reflected in the stabs info. > > Debugging anything involving static links has been broken for a long time, > because tree-nested.c in cm3cg does some very convoluted stuff with the > way static ancestor activation records are accessed, and I have not been > able to see an easy way to get stabs info to reflect it. > > I have thought for a long time that we really need to go to a version > of dwarf for debug info. stabs is very limited, and there is a lot of > info in the stabs output that is not really stabs, it's just kludgily > coded values crammed inside various fields of stabs that the compiler > and m3gdb both agree on (well, they used to.) > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > changes in where type information is actually developed, so it may not > be worth it. After all the work I have done on this to get m3gdb to where > it was, I am unenthusiastic about a bunch more work just to get back to > where we were. I have long lists of improvements to make to m3gdb, and > it's hard to go back that way. And, It's all in C :-(. > > If I put much effort into fixing m3gdb and its support in cm3cg at this > point, it will probably be a rework to use dwarf. That is likely a big > job too, but at least more rewarding and likely to lead to better m3gdb > function in the end. > > For now, it you want a working m3gdb, you really have to use the release > compiler. > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > Hi m3devel, > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > well (never perfectly, but...). Now I'm trying to debug a program and > > I get the following: > > > > (m3gdb) break Main.m3:65 > > Debug info for file "Main.mc" not in stabs format > > (m3gdb) > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > systems too. > > > > Everything is from the CVS head. > > > > Main was compiled as follows, says cm3 -commands: > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ > ci > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > something broken in the compiler, or what? > > > > I fear m3gdb is essentially useless at the moment. > > > > Mika > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Apr 2 04:42:33 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 01 Apr 2011 19:42:33 -0700 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <20110402024233.0BEC11A2078@async.async.caltech.edu> Jay K writes: ... >But that "custom information" is largely just basic type information=2C tha= >t any decent >C compiler/debug-info/debugger can handle. >Someone should investigate if we really pass stuff through that can't be we= >ll represented >in gcc's type information. >For example: sub ranges? sets? enums? packed? Subranges, sets, and enums certainly used to be shown in a "friendly" way by m3gdb. Mika From jay.krell at cornell.edu Sat Apr 2 05:24:02 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 03:24:02 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <20110402024233.0BEC11A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Apr 2 17:53:19 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 2 Apr 2011 16:53:19 +0100 (BST) Subject: [M3devel] m3gdb broken In-Reply-To: Message-ID: <196546.21262.qm@web29708.mail.ird.yahoo.com> Hi all: well, all that said, is it possible, but how many checks would be needed to all those errors, I perhaps think the best tool at hand is but to run undependently of the platform ESC, at least to catch the most common RT errors for now, but gcc as a backend is enough optimizable and also enough conservative to catch at compile time warnings you see as possible M3CG errors, in Modula-3 safe code there are not explicity needed in ESC terms but in UNSAFE ones, so the most RTError you want to catch as m3gdb exceptions would be advisable in foreseeable future, perhaps making some of them implementable at ESC level, but to check for those that are that in the RTCollector, RTException and the RTThread specially perhaps simpler ones in ESC terms but important, and in the M3CG memory handling, exception, and threads too, obviously in the RTType, even if those are not safely checked RTError should have some correspondent there and that is it must be caught in the M3CG before it can also generate bad codes, if so, also possible improved UNSAFE RTTypeSRC information to accomplish that perhaps a better but not limited to type checking and exceptions handled of both safe and UNSAFE code. Thinking in what's on the market there are recent reports in Logic Analyzer of embedded software-hardware verification environments for circuit analysis, such kind of automation tests are deployable even at the try and so we may get that, although this Embedded software Arena, but even at that level, cost effective knowing of the logic synthesis theories errors they want to be rid of, that is, is more important to shape some circuit level checks before product is deployed since may things depended on that too. We could seemingly adopt that strategic objective too, detect most of the common errors in those areas, also a nice RTTypeSRC interface for unsafe type code patterns. Perhaps some of this work is already done implemented in Modula-3 VM backend Quest abstract machine, since? it has nice constructs and basically same stack architecture and machine but with additional ability to support for separating sound and unsound modules. I think in this way the margin of RT errors will be diminished by at least by some volume bounded by the amount of each type inference in the checker, specially some of the ones in RTTypeSRC information portion plus desired good to watch at least some code modularity level sound handled in such code, so some of them that must be catched and, something it isn't right now for M3CG will not, but just because of correspondent MODULEs so it can be translated sound and caught in better CG compiler time in other representation and then generate code to its RT binary structures perhaps some may want for that too. Also that could be generous in terms of debugging both Quest level and m3gdb source level. Thanks in advance --- El vie, 1/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] m3gdb broken Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 1 de abril, 2011 22:24 The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. ?- Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Apr 3 03:28:18 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:28:18 -0700 Subject: [M3devel] floating point : problem with Extended.m3 Message-ID: <20110403012818.3560A1A2078@async.async.caltech.edu> Hello m3devel (mainly Tony), I believe I've found a problem with float/Common/Extended.m3. Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? That does not seem to me to be a safe assumption... in fact I don't think it's even safe to round up. EXTENDED could (and should) be 10 bytes wide on most machines. (I am aware that it is currently the same as LONGREAL, but...) Mika > UNSAFE MODULE Extended; > > IMPORT Word, Int32; > > PROCEDURE Equal(a, b: T): BOOLEAN = > BEGIN RETURN a = b END Equal; > > > CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); > > TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; > > PROCEDURE Hash(a: T): Word.T = > VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN > FOR i := 0 TO LAST(arr) DO > res := Word.Xor(res, Int32.Hash(arr[i])) > END (* FOR *); > RETURN res > END Hash; > > PROCEDURE Compare(a, b: T): [-1..1] = > BEGIN > IF a < b THEN RETURN -1 > ELSIF a > b THEN RETURN 1 > ELSE RETURN 0 > END (* IF *) > END Compare; From hosking at cs.purdue.edu Sun Apr 3 03:40:02 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 2 Apr 2011 21:40:02 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403012818.3560A1A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: Are you saying it is broken with EXTENDED=LONGREAL? That is the current implementation. This code is heritage. Not sure I understand all the implications. On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > Hello m3devel (mainly Tony), > > I believe I've found a problem with float/Common/Extended.m3. > > Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? > > That does not seem to me to be a safe assumption... in fact I don't > think it's even safe to round up. EXTENDED could (and should) be > 10 bytes wide on most machines. (I am aware that it is currently > the same as LONGREAL, but...) > > Mika > >> UNSAFE MODULE Extended; >> >> IMPORT Word, Int32; >> >> PROCEDURE Equal(a, b: T): BOOLEAN = >> BEGIN RETURN a = b END Equal; >> >> >> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >> >> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >> >> PROCEDURE Hash(a: T): Word.T = >> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >> FOR i := 0 TO LAST(arr) DO >> res := Word.Xor(res, Int32.Hash(arr[i])) >> END (* FOR *); >> RETURN res >> END Hash; >> >> PROCEDURE Compare(a, b: T): [-1..1] = >> BEGIN >> IF a < b THEN RETURN -1 >> ELSIF a > b THEN RETURN 1 >> ELSE RETURN 0 >> END (* IF *) >> END Compare; From mika at async.caltech.edu Sun Apr 3 03:52:08 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:52:08 -0700 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: <20110403015208.6B1511A2078@async.async.caltech.edu> No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 bits). I'm really just curious what has to be changed to make EXTENDED not be LONGREAL... probably many more things in the compiler? Mika Tony Hosking writes: >Are you saying it is broken with EXTENDED=LONGREAL? >That is the current implementation. >This code is heritage. Not sure I understand all the implications. > >On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > >> Hello m3devel (mainly Tony), >> >> I believe I've found a problem with float/Common/Extended.m3. >> >> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >> >> That does not seem to me to be a safe assumption... in fact I don't >> think it's even safe to round up. EXTENDED could (and should) be >> 10 bytes wide on most machines. (I am aware that it is currently >> the same as LONGREAL, but...) >> >> Mika >> >>> UNSAFE MODULE Extended; >>> >>> IMPORT Word, Int32; >>> >>> PROCEDURE Equal(a, b: T): BOOLEAN = >>> BEGIN RETURN a = b END Equal; >>> >>> >>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>> >>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>> >>> PROCEDURE Hash(a: T): Word.T = >>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>> FOR i := 0 TO LAST(arr) DO >>> res := Word.Xor(res, Int32.Hash(arr[i])) >>> END (* FOR *); >>> RETURN res >>> END Hash; >>> >>> PROCEDURE Compare(a, b: T): [-1..1] = >>> BEGIN >>> IF a < b THEN RETURN -1 >>> ELSIF a > b THEN RETURN 1 >>> ELSE RETURN 0 >>> END (* IF *) >>> END Compare; From hosking at cs.purdue.edu Sun Apr 3 22:15:00 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 3 Apr 2011 16:15:00 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403015208.6B1511A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> <20110403015208.6B1511A2078@async.async.caltech.edu> Message-ID: <3523B817-7987-4CA1-8230-0DF99EE76CCE@cs.purdue.edu> On Apr 2, 2011, at 9:52 PM, Mika Nystrom wrote: > No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 > bits). > > I'm really just curious what has to be changed to make EXTENDED not be > LONGREAL... probably many more things in the compiler? Yes, definitely. Compiler and run-time. > > Mika > > Tony Hosking writes: >> Are you saying it is broken with EXTENDED=LONGREAL? >> That is the current implementation. >> This code is heritage. Not sure I understand all the implications. >> >> On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: >> >>> Hello m3devel (mainly Tony), >>> >>> I believe I've found a problem with float/Common/Extended.m3. >>> >>> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >>> >>> That does not seem to me to be a safe assumption... in fact I don't >>> think it's even safe to round up. EXTENDED could (and should) be >>> 10 bytes wide on most machines. (I am aware that it is currently >>> the same as LONGREAL, but...) >>> >>> Mika >>> >>>> UNSAFE MODULE Extended; >>>> >>>> IMPORT Word, Int32; >>>> >>>> PROCEDURE Equal(a, b: T): BOOLEAN = >>>> BEGIN RETURN a = b END Equal; >>>> >>>> >>>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>>> >>>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>>> >>>> PROCEDURE Hash(a: T): Word.T = >>>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>>> FOR i := 0 TO LAST(arr) DO >>>> res := Word.Xor(res, Int32.Hash(arr[i])) >>>> END (* FOR *); >>>> RETURN res >>>> END Hash; >>>> >>>> PROCEDURE Compare(a, b: T): [-1..1] = >>>> BEGIN >>>> IF a < b THEN RETURN -1 >>>> ELSIF a > b THEN RETURN 1 >>>> ELSE RETURN 0 >>>> END (* IF *) >>>> END Compare; From rodney_bates at lcwb.coop Sun Apr 3 22:50:35 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:50:35 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <4D98DD9B.9080003@lcwb.coop> On 04/01/2011 08:26 PM, Jay K wrote: > I really tried not to break anything in the wierdo stubs/m3gdb stuff. > > > The work I have been doing to properly declare types is I believe disabled currently. > > Hmm, I looked at a sample of stabs output for the same program, produced by the release and head compilers. In the head, all of the stabs type information is missing, including the type definitions and the references to them. This may be enough to explain all the breakage in m3gdb I have seen. I see another small change, but it is something that, I think, m3gdb already adapts to, and, as in the release version, was weird anyway. > As types become more correct, things start breaking. > Either because the optimizer is more aggressive, or because more sanity checks > can be done by the compiler, and we don't give it particularly good trees. > > > In most things I've looked it, it's really not any mystery what is wrong with our trees btw. > It's not really generally the fault of gcc being obscure or changing. > Though perhaps to some extent that is true. > Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not > accessible from C source, and therefore bugs in them go undetected. > It's more generally clearly not great stuff in cm3cg. > The main bad thing is that field access is done by just adding to base pointers and casting away. > > > Note that stabs is not supported on Darwin. > Nor as I recall HP-UX/HPPA64. > All the more reason to no do things this way. > > > Note furthermore that discussion here is often confused. > Really no particular debug-info generation code should be cm3cg. > In our part -- m3cg/parse.c > Neither stabs nor dwarf nor coff nor anything else. > > > The right approach is to build an intermediate form with reasonably good types. > The backend handles it from there, whatever debug-info format is requested on the command line. > > > This also enables using stock/current gdb, instead of hacked up/old gdb. > > > I made no visible progress upgrading our fork of gdb. > I'd rather just have it go away. > > > I understand, that custom information is being tunneled through to m3gdb. > But that "custom information" is largely just basic type information, that any decent > C compiler/debug-info/debugger can handle. > Someone should investigate if we really pass stuff through that can't be well represented > in gcc's type information. > For example: sub ranges? sets? enums? packed? > Some higher level description of "methods" (really -- whatever C++ does should suffice; > if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) > > > At some point the new type information was really breaking stuff so I think I disabled it. > The system is kind of incomplete and fragile and the way forward is not always easy > or possible in small steps. :( > > > - Jay > > > > Date: Fri, 1 Apr 2011 12:43:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > > using the Cartesian product of release/head for m3gdb and the compiler. > > > > However, I see a lot of broken stuff that used to work. In all the cases > > I have seen, it correlates to debugging code compiled by the head compiler, > > while the release and head versions of m3gdb make no difference to the > > symptoms in anything I have tried. > > > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > > I am not sure how far that got. Its source code is vastly different, and > > most of the other language support in gdb has been ripped out. > > > > There is quite a lot of change in the stabs info generated by the two > > compiler versions. The existing way for a long time has been that this > > is constructed during the first pass over the input to cm3cg, before > > anything other than initial tree building is done. This is a very > > limited strategy, because cm3cg can change things later, and this will > > not be reflected in the stabs info. > > > > Debugging anything involving static links has been broken for a long time, > > because tree-nested.c in cm3cg does some very convoluted stuff with the > > way static ancestor activation records are accessed, and I have not been > > able to see an easy way to get stabs info to reflect it. > > > > I have thought for a long time that we really need to go to a version > > of dwarf for debug info. stabs is very limited, and there is a lot of > > info in the stabs output that is not really stabs, it's just kludgily > > coded values crammed inside various fields of stabs that the compiler > > and m3gdb both agree on (well, they used to.) > > > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > > changes in where type information is actually developed, so it may not > > be worth it. After all the work I have done on this to get m3gdb to where > > it was, I am unenthusiastic about a bunch more work just to get back to > > where we were. I have long lists of improvements to make to m3gdb, and > > it's hard to go back that way. And, It's all in C :-(. > > > > If I put much effort into fixing m3gdb and its support in cm3cg at this > > point, it will probably be a rework to use dwarf. That is likely a big > > job too, but at least more rewarding and likely to lead to better m3gdb > > function in the end. > > > > For now, it you want a working m3gdb, you really have to use the release > > compiler. > > > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > > Hi m3devel, > > > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > > well (never perfectly, but...). Now I'm trying to debug a program and > > > I get the following: > > > > > > (m3gdb) break Main.m3:65 > > > Debug info for file "Main.mc" not in stabs format > > > (m3gdb) > > > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > > systems too. > > > > > > Everything is from the CVS head. > > > > > > Main was compiled as follows, says cm3 -commands: > > > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika /t/ > > ci > > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > > something broken in the compiler, or what? > > > > > > I fear m3gdb is essentially useless at the moment. > > > > > > Mika > > > From rodney_bates at lcwb.coop Sun Apr 3 22:46:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:46:08 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: <4D98DC90.3090900@lcwb.coop> On 04/01/2011 10:24 PM, Jay K wrote: > The question is, can they be well represented within gcc's usual trees/types/debug-info > for good experience in stock gdb. > Enums for example -- yes, I had them working. > > Subranges -- what should the experience be? > For example, when I use debuggers, I use expression evaluation involving assignment > to set values. Should the debugger catch attempts to assign out of range values? > Or are subranges just integers with a certain number of bits and that is it and that suffices? > I understand the runtime does more precise checks than just bit-width. > > sets -- I can see this being....are they just an integer or an array of integers? > Or higher level? I can see this being poor...unless unless.. gcc maybe already > had good support for Pascal? Maybe Ada has a similar feature? > Ada does actually look like Modula-3 to a large extent! > > Consider also that on Darwin, none of this stuff works at all. > So even sets as integers or arrays of integers would be much better than current. > > One has to read some of the gcc documentation, browse tree.h, tree.def, etc., > learn what is well representable. > > One shouldn't just completely ignore the significant infrastructure and just > tunnel all the information through in a custom way that stock gdb is completely ignorant of. > There is no way stock gdb can possibly provide anything remotely similar to the support it gives other languages, just by providing it with better debug info. Every one of the languages already supported by stock gdb (C, C++, Objective C, Pascal, Ada, Java, Fortran, and Scheme come to mind) have significant amounts of language-specific code, comparable to what m3gdb adds for Modula-3 support. Some categories of language-specific code include: name demanglers, expression parsers, expression evaluators, identifier lookups, characteristics of builtin types, and value printers. You can find anecdotal cases of types, operators, and their syntax that are equivalent to another language, but this is highly incomplete. Value notations also differ. Semantics of same-looking operators can be significantly different. The dot is a good example. Not even Pascal nor Ada are the same as Modula-3. Interpretation of memory is also highly language dependent. There are significant differences in runtime representation of seemingly equivalent types and other stuff. Modula-3's TEXT, for example, is represented unlike any of the other languages. Heap objects have distinct ways of showing allocation status, GC info, open array sizes for differing dimensionalities, and just the plain allocated object type. The method dispatching mechanism is distinct. Actually, this stuff is really compiler-dependent, not just language-dependent. There are significant differences between pm3/friends and cm3, and occasional smaller differences between versions thereof. m3gdb recognizes many of these and adapts. Ditto for some of the differences in thread and GC implementations. Some procedures have hidden parameters that show up in the type info, but do not appear in source code calls. The rules for these are fairly conditional, on both the procedure signature and on the target OS. It is difficult for a debugger user to even know when, where, and of what type these need to be to supply them explicitly to make such a call, and as I recall, in some cases impossible to supply some of the needed values, even with full knowledge. In contrast, m3gdb allows typing procedure and method calls as they would appear in source code, and supplies the hidden parameters as needed. Assigning procedures to variables and parameters and calling through them also have unique runtime representations. As of the gdb the current m3gdb is based on, no stock gdb language handles up-level addressing. Much of this is working in the current m3gdb, or was until recently. Most of what isn't is documented as work to do. > > - Jay > > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > Date: Fri, 1 Apr 2011 19:42:33 -0700 > > From: mika at async.caltech.edu > > > > Jay K writes: > > ... > > >But that "custom information" is largely just basic type information=2C tha= > > >t any decent > > >C compiler/debug-info/debugger can handle. > > >Someone should investigate if we really pass stuff through that can't be we= > > >ll represented > > >in gcc's type information. > > >For example: sub ranges? sets? enums? packed? > > > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > > by m3gdb. > > > > Mika From dragisha at m3w.org Thu Apr 7 16:34:20 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 7 Apr 2011 16:34:20 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Message-ID: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> I am getting this, when my program does a Thread.Fork. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 *** Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... TIA, dd From hosking at cs.purdue.edu Thu Apr 7 17:41:07 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 11:41:07 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> Message-ID: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> OK, sounds like the stability of the system is going downhill. I'm not sure when I'll have time to look at this. Maybe soon, maybe in the summer. I am discouraged at all the reports we have been having about these sorts of problems. On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > I am getting this, when my program does a Thread.Fork. > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > *** > > Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... > > TIA, > dd From dabenavidesd at yahoo.es Thu Apr 7 18:30:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 7 Apr 2011 17:30:54 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <785020.50972.qm@web29715.mail.ird.yahoo.com> Hi all: it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. Thanks in advance PS there is Elego's product wiki article http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System Also related: http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf And also: http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf Perhaps is good to see this issue: http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). --- El jue, 7/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Dragi?a Duri?" > CC: "m3devel" > Fecha: jueves, 7 de abril, 2011 10:41 > OK, sounds like the stability of the > system is going downhill. I'm not sure when I'll have > time to look at this. Maybe soon, maybe in the > summer. I am discouraged at all the reports we have > been having about these sorts of problems. > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > I am getting this, when my program does a > Thread.Fork. > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > *** > > > > Any ideas? It worked... And stopped working... Some > changes were made to openssl libraries and similar... > > > > TIA, > > dd > > From mika at async.caltech.edu Thu Apr 7 21:02:19 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 07 Apr 2011 12:02:19 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <20110407190219.C00E31A2078@async.async.caltech.edu> Remember that for mission-critical applications there's always user threading.. I find that the system (up to and including the CVS head and on both amd64 and i386 flavors of Linux as well as every other OS I have tried) is very very stable with user threading. I'm eager to use pthreads again of course, for all sorts of reasons... Mika Tony Hosking writes: >OK, sounds like the stability of the system is going downhill. I'm not = >sure when I'll have time to look at this. Maybe soon, maybe in the = >summer. I am discouraged at all the reports we have been having about = >these sorts of problems. > >On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: > >> I am getting this, when my program does a Thread.Fork. >>=20 >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >> *** >>=20 >> Any ideas? It worked... And stopped working... Some changes were made = >to openssl libraries and similar... >>=20 >> TIA, >> dd From hosking at cs.purdue.edu Thu Apr 7 22:07:51 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 16:07:51 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110407190219.C00E31A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> Message-ID: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Well, that is reassuring news. If the problems are isolated to threading then they can be resolved easily. On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > > Remember that for mission-critical applications there's always user > threading.. I find that the system (up to and including the CVS head > and on both amd64 and i386 flavors of Linux as well as every other OS I > have tried) is very very stable with user threading. I'm eager to use > pthreads again of course, for all sorts of reasons... > > Mika > > Tony Hosking writes: >> OK, sounds like the stability of the system is going downhill. I'm not = >> sure when I'll have time to look at this. Maybe soon, maybe in the = >> summer. I am discouraged at all the reports we have been having about = >> these sorts of problems. >> >> On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: >> >>> I am getting this, when my program does a Thread.Fork. >>> =20 >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >>> *** >>> =20 >>> Any ideas? It worked... And stopped working... Some changes were made = >> to openssl libraries and similar... >>> =20 >>> TIA, >>> dd From jay.krell at cornell.edu Thu Apr 7 23:25:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 7 Apr 2011 21:25:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <785020.50972.qm@web29715.mail.ird.yahoo.com> References: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <785020.50972.qm@web29715.mail.ird.yahoo.com> Message-ID: Problems with cvsupd are understood and believed fixed. It depended on all threads surviving fork, instead of just the one forking thread. Which is what userthreads did and which is what pthreads never did and never will do. No threads survive fork+exec, which is what most users of fork do. But cvsupd uses the "fork and do more work" pattern. Userthreads have been changed to be like pthreads. Past problems with cvsupd on pthreads should NOT be used to think about problems anywhere else in the system. If cvsupd still doesn't work, then that is possibly relevant to the rest of the system. - Jay > Date: Thu, 7 Apr 2011 17:30:54 +0100 > From: dabenavidesd at yahoo.es > To: dragisha at m3w.org; hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Hi all: > it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? > > After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I > possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). > For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. > > Thanks in advance > > PS there is Elego's product wiki article > http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System > Also related: > http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf > And also: > http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf > > Perhaps is good to see this issue: > http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded > > This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). > > --- El jue, 7/4/11, Tony Hosking escribi?: > > > De: Tony Hosking > > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Para: "Dragi?a Duri?" > > CC: "m3devel" > > Fecha: jueves, 7 de abril, 2011 10:41 > > OK, sounds like the stability of the > > system is going downhill. I'm not sure when I'll have > > time to look at this. Maybe soon, maybe in the > > summer. I am discouraged at all the reports we have > > been having about these sorts of problems. > > > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > > > I am getting this, when my program does a > > Thread.Fork. > > > > > > *** > > > *** runtime error: > > > *** <*ASSERT*> failed. > > > *** file > > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > > *** > > > > > > Any ideas? It worked... And stopped working... Some > > changes were made to openssl libraries and similar... > > > > > > TIA, > > > dd > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 00:45:16 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 10 Apr 2011 15:45:16 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX Message-ID: <20110410224516.E72F41A207A@async.async.caltech.edu> Hello m3devel (especially Jay), I'm having some time conversion problems. I *believe* (a bit hard to verify because m3gdb isn't working!!) that Date.ToTime called on Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. I note the following (fairly recent) code in DatePosixC.c: #ifdef _TIME64_T t = mktime64(&tm); #else t = mktime(&tm); #endif #ifdef DATE_BSD if (t == -1) goto Exit; /* adjust result to reflect "date->offset" */ #ifdef _TIME64_T time64(&now); local_now = localtime64(&now); #else time(&now); local_now = localtime(&now); #endif assert(local_now != NULL); if (local_now->tm_isdst > 0) /* decrement the local time zone by one hour if DST is in effect */ local_now->m3_tm_gmtoff -= SecsPerHour; /* As above, we must negate "date->offset" to account for the opposite sense of that field compared to Unix. */ t -= ((-date->offset) - local_now->m3_tm_gmtoff); Exit: #endif return t; What I don't understand is the subtracting off of SecsPerHour if "now" is DST. Yeah, now is DST (it's April), but the time I'm converting was in the winter, and besides GMT and UTC don't even have DST. I don't understand the intent of the code.. can someone explain why we're subtracting off an hour? And why is it based on the value of tm_isdst of the *current* time? Some additional documentation within the file would be helpful too. Note my conversion returns the correct 0 on various other M3 implementations. Note the system time on my system is in a timezone that has DST (namely, BST) but I have changed the TZ environment variable to UTC or GMT before calling this code. Mika From jay.krell at cornell.edu Mon Apr 11 04:00:35 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 11 Apr 2011 02:00:35 +0000 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110410224516.E72F41A207A@async.async.caltech.edu> References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: This stuff is tricky. Maybe impossible. And requires a fair amount of time and attention. Which I might not have right now, but which I definitely tried to apply at the time. Can you compare with some historical versions? e.g. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=1.1.1.2;content-type=text%2Fplain http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=1.1.1.1;content-type=text%2Fplain and notice: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3 Revision 1.3: download - view: text, markup, annotated - select for diffs Note: This also fixes the UTC timezone on Solaris, Interix, HP-UX, Cygwin, which appear to have been historically broken. I'm usually a bit nervous when I make accusations of long-term breakage. And yeah there is a general problem of determining when daylight savings is in effect. The rules very often change. The operating system needs a lot of data to know this. This stuff should be easy to debug without m3gdb. Just print stuff to stdout liberally. Aha, see here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/Attic/DateBsd.m3?rev=1.1.1.2;content-type=text%2Fplain (* adjust result to reflect "d.offset" *) EVAL Utime.time(ADR(now)); local_now := Utime.localtime(ADR(now)); IF local_now.tm_isdst > 0 THEN (* decrement the local time zone by one hour if DST is in effect *) DEC(local_now.tm_gmtoff, 1 * SecsPerHour) END; - Jay > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Date: Sun, 10 Apr 2011 15:45:16 -0700 > From: mika at async.caltech.edu > Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > > Hello m3devel (especially Jay), > > I'm having some time conversion problems. I *believe* (a bit hard to > verify because m3gdb isn't working!!) that Date.ToTime called on > Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. > > I note the following (fairly recent) code in DatePosixC.c: > > #ifdef _TIME64_T > t = mktime64(&tm); > #else > t = mktime(&tm); > #endif > #ifdef DATE_BSD > if (t == -1) > goto Exit; > > /* adjust result to reflect "date->offset" */ > #ifdef _TIME64_T > time64(&now); > local_now = localtime64(&now); > #else > time(&now); > local_now = localtime(&now); > #endif > assert(local_now != NULL); > if (local_now->tm_isdst > 0) > /* decrement the local time zone by one hour if DST is in effect */ > local_now->m3_tm_gmtoff -= SecsPerHour; > > /* As above, we must negate "date->offset" to account for the > opposite sense of that field compared to Unix. */ > t -= ((-date->offset) - local_now->m3_tm_gmtoff); > Exit: > #endif > return t; > > > What I don't understand is the subtracting off of SecsPerHour if > "now" is DST. Yeah, now is DST (it's April), but the time I'm > converting was in the winter, and besides GMT and UTC don't even > have DST. > > I don't understand the intent of the code.. can someone explain why > we're subtracting off an hour? And why is it based on the value of > tm_isdst of the *current* time? Some additional documentation within > the file would be helpful too. > > Note my conversion returns the correct 0 on various other M3 > implementations. > > Note the system time on my system is in a timezone that has DST > (namely, BST) but I have changed the TZ environment variable to UTC or GMT > before calling this code. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 11:01:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 11 Apr 2011 02:01:37 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: <20110411090137.BEF721A207A@async.async.caltech.edu> I'm not sure about impossible. But maybe... Here's the situation. I'm using unix time in my code. Sometimes it needs to run on Windows. Some versions of Modula-3 on Windows use the Windows epoch for timekeeping. So I first request the time at midnight UTC Jan 1 1970 and set that to "UnixEpoch". On Unix systems I expect it to be zero, on some other systems it might be another number. Now I've found what the problem is, and it's not anything you have done. In fact PM3 exhibits the same problem if you set everything up precisely the same. The issue is that the server I am using is located in the United Kingdom and it has its time zone set to Europe/London. Nowadays the Brits use the following definitions for time: in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 in summer "British Summer Time" abbrev. "BST" = UTC + 3600 However, between October 26, 1968 and October 31, 1971, they used something else, to wit: all year "British Standard Time" abbrev. "BST" = UTC + 3600 Note the confusing abbreviations. British Standard Time (which no longer exists) and British Summer Time have the same abbreviation. You might think this is sensible, because British Summer Time has the same value as British Standard Time (when the former is defined). There is just one problem. British Standard Time is not a "daylight savings time" whereas British Summer Time *is*. Witness the output of zdump for London: Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar 19 03:00:00 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct 29 02:59:59 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct 29 02:00:00 1967 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb 18 01:59:59 1968 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb 18 03:00:00 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct 31 02:00:00 1971 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar 19 01:59:59 1972 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar 19 03:00:00 1972 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct 29 02:59:59 1972 BST isdst=1 gmtoff=3600 The following lines are the special ones: Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Note how the gmtoff stays the same but isdst changes. And this of course straddles the definition of Unix time at Jan 1 1970 00:00:00 UTC. Clearly there is a bug in M3's date code here. I just don't understand what invariant the original author appealed to when he inserted the dst-changing code. It is hardly unusual for a given locale to have changed time zone rules... it is completely wrong for that to affect the conversion of times for a *different* locale (here I am just trying to do a time conversion for UTC, it shouldn't matter what country my computer is in when it does that!) Mika Jay K writes: >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention. >Which I might not have right now=2C but which I definitely tried to apply a= >t the time. > >Can you compare with some historical versions? >e.g. > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > >and notice: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3 >Revision 1.3: download - view: text=2C markup=2C annotated - select for dif= >fs >Note: This also fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cy= >gwin=2C which appear to have been historically broken. > > >I'm usually a bit nervous when I make accusations of long-term breakage. > > >And yeah there is a general problem of determining when daylight savings is= > in effect. >The rules very often change. The operating system needs a lot of data to kn= >ow this. > > >This stuff should be easy to debug without m3gdb. >Just print stuff to stdout liberally. > >Aha=2C see here: >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > (* adjust result to reflect "d.offset" *) > EVAL Utime.time(ADR(now))=3B > local_now :=3D Utime.localtime(ADR(now))=3B > IF local_now.tm_isdst > 0 THEN > (* decrement the local time zone by one hour if DST is in effect *) > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour) > END=3B > > - Jay > >> To: m3devel at elegosoft.com=3B jay.krell at cornell.edu >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 >> From: mika at async.caltech.edu >> Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX >>=20 >> Hello m3devel (especially Jay)=2C >>=20 >> I'm having some time conversion problems. I *believe* (a bit hard to >> verify because m3gdb isn't working!!) that Date.ToTime called on >> Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. >>=20 >> I note the following (fairly recent) code in DatePosixC.c: >>=20 >> #ifdef _TIME64_T >> t =3D mktime64(&tm)=3B >> #else >> t =3D mktime(&tm)=3B >> #endif >> #ifdef DATE_BSD >> if (t =3D=3D -1) >> goto Exit=3B >>=20 >> /* adjust result to reflect "date->offset" */ >> #ifdef _TIME64_T >> time64(&now)=3B >> local_now =3D localtime64(&now)=3B >> #else >> time(&now)=3B >> local_now =3D localtime(&now)=3B >> #endif >> assert(local_now !=3D NULL)=3B >> if (local_now->tm_isdst > 0) >> /* decrement the local time zone by one hour if DST is in effect */ >> local_now->m3_tm_gmtoff -=3D SecsPerHour=3B >>=20 >> /* As above=2C we must negate "date->offset" to account for the >> opposite sense of that field compared to Unix. */ >> t -=3D ((-date->offset) - local_now->m3_tm_gmtoff)=3B >> Exit: >> #endif >> return t=3B >>=20 >>=20 >> What I don't understand is the subtracting off of SecsPerHour if >> "now" is DST. Yeah=2C now is DST (it's April)=2C but the time I'm >> converting was in the winter=2C and besides GMT and UTC don't even >> have DST.=20 >>=20 >> I don't understand the intent of the code.. can someone explain why >> we're subtracting off an hour? And why is it based on the value of=20 >> tm_isdst of the *current* time? Some additional documentation within >> the file would be helpful too. >>=20 >> Note my conversion returns the correct 0 on various other M3 >> implementations. >>=20 >> Note the system time on my system is in a timezone that has DST=20 >> (namely=2C BST) but I have changed the TZ environment variable to UTC or = >GMT >> before calling this code. >>=20 >> Mika > = > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention.
Which I might not have right now=2C but which I definitel= >y tried to apply at the time.

Can you compare with some historical v= >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= >lain


and notice:

http://modula3.elegosoft.com/cgi-bin/cvs= >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision 1.3b>: 7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fplain" class=3D"download-link">download - view: tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" class=3D"display= >-link">text=2C gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fx-cvsweb-markup" class=3D"display-link">markup=2C http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?annotate=3D1.3">annotated - a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: This a= >lso fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cygwin=2C whic= >h appear to have been historically broken.


I'm usually a bit ner= >vous when I make accusations of long-term breakage.


And yeah the= >re is a general problem of determining when daylight savings is in effect.<= >br>The rules very often change. The operating system needs a lot of data to= > know this.


This stuff should be easy to debug without m3gdb.>Just print stuff to stdout liberally.

Aha=2C see here:
http://mo= >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

  (*=
> adjust result to reflect "d.offset" *)
    EVAL Utime.time(ADR(now))=3B=
>
    local_now :=3D Utime.localtime(ADR(now))=3B
    IF local_now.tm_=
>isdst >=3B 0 THEN
      (* decrement the local time zone by one hour i=
>f DST is in effect *)
      DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
>br>    END=3B

 =3B- Jay

>=3B To: m3devel at elegosof= >t.com=3B jay.krell at cornell.edu
>=3B Date: Sun=2C 10 Apr 2011 15:45:16 = >-0700
>=3B From: mika at async.caltech.edu
>=3B Subject: [M3devel] t= >ime conversion problems : Date.ToTime on AMD64_LINUX
>=3B
>=3B H= >ello m3devel (especially Jay)=2C
>=3B
>=3B I'm having some time = >conversion problems. I *believe* (a bit hard to
>=3B verify because m= >3gdb isn't working!!) that Date.ToTime called on
>=3B Jan 1 1970 in ti= >mezone "UTC" or "GMT" is returning -3600 for me.
>=3B
>=3B I not= >e the following (fairly recent) code in DatePosixC.c:
>=3B
>=3B = >#ifdef _TIME64_T
>=3B t =3D mktime64(&=3Btm)=3B
>=3B #else= >
>=3B t =3D mktime(&=3Btm)=3B
>=3B #endif
>=3B #ifdef= > DATE_BSD
>=3B if (t =3D=3D -1)
>=3B goto Exit=3B
= >>=3B
>=3B /* adjust result to reflect "date->=3Boffset" */>>=3B #ifdef _TIME64_T
>=3B time64(&=3Bnow)=3B
>=3B = >local_now =3D localtime64(&=3Bnow)=3B
>=3B #else
>=3B time= >(&=3Bnow)=3B
>=3B local_now =3D localtime(&=3Bnow)=3B
>= >=3B #endif
>=3B assert(local_now !=3D NULL)=3B
>=3B if (l= >ocal_now->=3Btm_isdst >=3B 0)
>=3B /* decrement the local ti= >me zone by one hour if DST is in effect */
>=3B local_now->=3B= >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B
>=3B /* As above=2C we= > must negate "date->=3Boffset" to account for the
>=3B opposi= >te sense of that field compared to Unix. */
>=3B t -=3D ((-date-&g= >t=3Boffset) - local_now->=3Bm3_tm_gmtoff)=3B
>=3B Exit:
>=3B #e= >ndif
>=3B return t=3B
>=3B
>=3B
>=3B What I don't= > understand is the subtracting off of SecsPerHour if
>=3B "now" is DST= >. Yeah=2C now is DST (it's April)=2C but the time I'm
>=3B converting= > was in the winter=2C and besides GMT and UTC don't even
>=3B have DST= >.
>=3B
>=3B I don't understand the intent of the code.. can som= >eone explain why
>=3B we're subtracting off an hour? And why is it ba= >sed on the value of
>=3B tm_isdst of the *current* time? Some additi= >onal documentation within
>=3B the file would be helpful too.
>= >=3B
>=3B Note my conversion returns the correct 0 on various other M3= >
>=3B implementations.
>=3B
>=3B Note the system time on my= > system is in a timezone that has DST
>=3B (namely=2C BST) but I have= > changed the TZ environment variable to UTC or GMT
>=3B before calling= > this code.
>=3B
>=3B Mika
> >= > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- From dabenavidesd at yahoo.es Mon Apr 11 18:28:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 11 Apr 2011 17:28:45 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110411090137.BEF721A207A@async.async.caltech.edu> Message-ID: <926161.28913.qm@web29713.mail.ird.yahoo.com> Hi all: although my understanding of the problem is not that good, I think there are two routes for this abstract way one could try to prove the correctness of some implementation by setting directly instrumention code to try to see for your self dynamically if you may that or what you want to see: http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s This could would run with just 3 missing modules from this sources besides their examples, maybe another time the HP guys allow us to see their works on that: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ This package is missing from public repositories Elego too as I know, though they were in (see bottom links dead as of now, but, ... maybe you can look for yourselves): http://www.cs.arizona.edu/~collberg/Research/Modula-3/ I know some ESC/Modula-3 tries to make some inferencing of invariants, but as I said it dones't mean this is totally computable, im fact in general its isn't, but it would be good to try. Essentially where are the codes for showing that, or are you implementing this as private code? Thanks in advance --- El lun, 11/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 04:01 > I'm not sure about impossible. > But maybe... > > Here's the situation. > > I'm using unix time in my code. Sometimes it needs to > run on Windows. > Some versions of Modula-3 on Windows use the Windows epoch > for timekeeping. > So I first request the time at midnight UTC Jan 1 1970 and > set that to > "UnixEpoch". On Unix systems I expect it to be zero, > on some other systems > it might be another number. > > Now I've found what the problem is, and it's not anything > you have done. > In fact PM3 exhibits the same problem if you set everything > up precisely > the same. > > The issue is that the server I am using is located in the > United Kingdom > and it has its time zone set to Europe/London. > Nowadays the Brits use the > following definitions for time: > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 > in summer "British Summer Time" abbrev. "BST" = UTC + 3600 > > However, between October 26, 1968 and October 31, 1971, > they used something > else, to wit: > > all year "British Standard Time" abbrev. "BST" = UTC + > 3600 > > Note the confusing abbreviations. British Standard > Time (which no longer > exists) and British Summer Time have the same > abbreviation. You might think > this is sensible, because British Summer Time has the same > value as British > Standard Time (when the former is defined). > > There is just one problem. British Standard Time is > not a "daylight > savings time" whereas British Summer Time *is*. > Witness the output of > zdump for London: > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > The following lines are the special ones: > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Note how the gmtoff stays the same but isdst changes. > And this of course > straddles the definition of Unix time at Jan 1 1970 > 00:00:00 UTC. > > Clearly there is a bug in M3's date code here. I just > don't understand what > invariant the original author appealed to when he inserted > the dst-changing code. > It is hardly unusual for a given locale to have changed > time zone rules... > it is completely wrong for that to affect the conversion of > times for a *different* > locale (here I am just trying to do a time conversion for > UTC, it shouldn't > matter what country my computer is in when it does that!) > > Mika > > Jay K writes: > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention. > >Which I might not have right now=2C but which I > definitely tried to apply a= > >t the time. > > > >Can you compare with some historical versions? > >e.g. > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > >and notice: > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3 > >Revision 1.3: download - view: text=2C markup=2C > annotated - select for dif= > >fs > >Note: This also fixes the UTC timezone on Solaris=2C > Interix=2C HP-UX=2C Cy= > >gwin=2C which appear to have been historically broken. > > > > > >I'm usually a bit nervous when I make accusations of > long-term breakage. > > > > > >And yeah there is a general problem of determining when > daylight savings is= > > in effect. > >The rules very often change. The operating system needs > a lot of data to kn= > >ow this. > > > > > >This stuff should be easy to debug without m3gdb. > >Just print stuff to stdout liberally. > > > >Aha=2C see here: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > (* adjust result to reflect "d.offset" *) > > EVAL Utime.time(ADR(now))=3B > > local_now :=3D > Utime.localtime(ADR(now))=3B > > IF local_now.tm_isdst > 0 THEN > > (* decrement the local time zone > by one hour if DST is in effect *) > > DEC(local_now.tm_gmtoff=2C 1 * > SecsPerHour) > > END=3B > > > > - Jay > > > >> To: m3devel at elegosoft.com=3B > jay.krell at cornell.edu > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > >> From: mika at async.caltech.edu > >> Subject: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > >>=20 > >> Hello m3devel (especially Jay)=2C > >>=20 > >> I'm having some time conversion problems. I > *believe* (a bit hard to > >> verify because m3gdb isn't working!!) that > Date.ToTime called on > >> Jan 1 1970 in timezone "UTC" or "GMT" is returning > -3600 for me. > >>=20 > >> I note the following (fairly recent) code in > DatePosixC.c: > >>=20 > >> #ifdef _TIME64_T > >> t =3D > mktime64(&tm)=3B > >> #else > >> t =3D mktime(&tm)=3B > >> #endif > >> #ifdef DATE_BSD > >> if (t =3D=3D -1) > >> goto > Exit=3B > >>=20 > >> /* adjust result to > reflect "date->offset" */ > >> #ifdef _TIME64_T > >> time64(&now)=3B > >> local_now =3D > localtime64(&now)=3B > >> #else > >> time(&now)=3B > >> local_now =3D > localtime(&now)=3B > >> #endif > >> assert(local_now !=3D > NULL)=3B > >> if (local_now->tm_isdst > > 0) > >> /* decrement the > local time zone by one hour if DST is in effect */ > >> > local_now->m3_tm_gmtoff -=3D > SecsPerHour=3B > >>=20 > >> /* As above=2C we must > negate "date->offset" to account for the > >> opposite sense of that > field compared to Unix. */ > >> t -=3D ((-date->offset) > - local_now->m3_tm_gmtoff)=3B > >> Exit: > >> #endif > >> return t=3B > >>=20 > >>=20 > >> What I don't understand is the subtracting off of > SecsPerHour if > >> "now" is DST. Yeah=2C now is DST (it's > April)=2C but the time I'm > >> converting was in the winter=2C and besides GMT > and UTC don't even > >> have DST.=20 > >>=20 > >> I don't understand the intent of the code.. can > someone explain why > >> we're subtracting off an hour? And why is it > based on the value of=20 > >> tm_isdst of the *current* time? Some > additional documentation within > >> the file would be helpful too. > >>=20 > >> Note my conversion returns the correct 0 on > various other M3 > >> implementations. > >>=20 > >> Note the system time on my system is in a timezone > that has DST=20 > >> (namely=2C BST) but I have changed the TZ > environment variable to UTC or = > >GMT > >> before calling this code. > >>=20 > >> Mika > > > > > = > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention.
Which I might not have right > now=2C but which I definitel= > >y tried to apply at the time.

Can > you compare with some historical v= > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > >lain


and > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > 1.3 >b>: >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fplain" > class=3D"download-link">download - view: href=3D"ht= > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > class=3D"display= > >-link">text=2C >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fx-cvsweb-markup" > class=3D"display-link">markup=2C href=3D"= > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > - >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > This a= > >lso fixes the UTC timezone on Solaris=2C Interix=2C > HP-UX=2C Cygwin=2C whic= > >h appear to have been historically > broken.


I'm usually a bit ner= > >vous when I make accusations of long-term > breakage.


And yeah the= > >re is a general problem of determining when daylight > savings is in effect.<= > >br>The rules very often change. The operating system > needs a lot of data to= > > know this.


This stuff > should be easy to debug without m3gdb. >>Just print stuff to stdout > liberally.

Aha=2C see here:
http://mo= > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

 
> (*=
> > adjust result to reflect "d.offset" *)
 
>   EVAL Utime.time(ADR(now))=3B=
> >
    local_now :=3D
> Utime.localtime(ADR(now))=3B
    IF
> local_now.tm_=
> >isdst >=3B 0 THEN
     
> (* decrement the local time zone by one hour i=
> >f DST is in effect *)
     
> DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> >br>   
> END=3B

 =3B- > Jay

>=3B To: m3devel at elegosof= > >t.com=3B jay.krell at cornell.edu
>=3B > Date: Sun=2C 10 Apr 2011 15:45:16 = > >-0700
>=3B From: mika at async.caltech.edu
>=3B > Subject: [M3devel] t= > >ime conversion problems : Date.ToTime on > AMD64_LINUX
>=3B
>=3B H= > >ello m3devel (especially Jay)=2C
>=3B >
>=3B I'm having some time = > >conversion problems. I *believe* (a bit hard > to
>=3B verify because m= > >3gdb isn't working!!) that Date.ToTime called > on
>=3B Jan 1 1970 in ti= > >mezone "UTC" or "GMT" is returning -3600 for > me.
>=3B
>=3B I not= > >e the following (fairly recent) code in > DatePosixC.c:
>=3B
>=3B = > >#ifdef _TIME64_T
>=3B > t =3D > mktime64(&=3Btm)=3B
>=3B #else= > >
>=3B t =3D > mktime(&=3Btm)=3B
>=3B > #endif
>=3B #ifdef= > > DATE_BSD
>=3B > if (t =3D=3D -1)
>=3B > goto Exit=3B
= > >>=3B
>=3B > /* adjust result to reflect > "date->=3Boffset" */ >>>=3B #ifdef > _TIME64_T
>=3B > time64(&=3Bnow)=3B
>=3B > = > >local_now =3D > localtime64(&=3Bnow)=3B
>=3B > #else
>=3B time= > >(&=3Bnow)=3B
>=3B > local_now =3D > localtime(&=3Bnow)=3B
>= > >=3B #endif
>=3B > assert(local_now !=3D > NULL)=3B
>=3B if (l= > >ocal_now->=3Btm_isdst >=3B > 0)
>=3B /* > decrement the local ti= > >me zone by one hour if DST is in effect > */
>=3B > local_now->=3B= > >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B >
>=3B /* As above=2C > we= > > must negate "date->=3Boffset" to account for > the
>=3B opposi= > >te sense of that field compared to Unix. > */
>=3B t -=3D > ((-date-&g= > >t=3Boffset) - > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > Exit:
>=3B #e= > >ndif
>=3B return > t=3B
>=3B
>=3B >
>=3B What I don't= > > understand is the subtracting off of SecsPerHour > if
>=3B "now" is DST= > >. Yeah=2C now is DST (it's April)=2C but the time > I'm
>=3B converting= > > was in the winter=2C and besides GMT and UTC don't > even
>=3B have DST= > >.
>=3B
>=3B I don't > understand the intent of the code.. can som= > >eone explain why
>=3B we're subtracting > off an hour? And why is it ba= > >sed on the value of
>=3B tm_isdst of > the *current* time? Some additi= > >onal documentation within
>=3B the file > would be helpful too.
>= > >=3B
>=3B Note my conversion returns the > correct 0 on various other M3= > >
>=3B > implementations.
>=3B
>=3B > Note the system time on my= > > system is in a timezone that has DST >
>=3B (namely=2C BST) but I have= > > changed the TZ environment variable to UTC or > GMT
>=3B before calling= > > this code.
>=3B >
>=3B Mika
> > > > > > >= > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > From dabenavidesd at yahoo.es Wed Apr 13 17:14:51 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 13 Apr 2011 16:14:51 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <171917.14906.qm@web29716.mail.ird.yahoo.com> Hi all: it would nice to ask HP whether they have decommissioned algorithm animation examples in Zeus3D, to get their sources, since there hasn't been much advances in this area even without Modula-3 on the Scene (let's say of my knowledge there has not been too much attention to hold animation festivals regularly, etc), I just wonder what is about their example implementations: ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-110a.pdf The ones seen in first video report at least, the heap sort: http://replay.waybackmachine.org/20051217123443/http://www.research.compaq.com/SRC/zeus/heapsort3D.html http://replay.waybackmachine.org/20051101010009/http://research.compaq.com/SRC/zeus/sorting3D.html I think 234 trees are missing screenshots (binary three search) though redblack trees appears: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/redblack3D.html The same example for computational geometry kd-tree: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/2dtree3D.html And the last shortest path algorithm example: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/shortestpath3D.html The ones, decommissioned ones I read in initial paragraph here: http://www.internotesstrategy.com/AlgorithMain.htm The most interesting ones in Modula2+ if so are the parallel sorting algorithms, but who might know where are they? Thanks in advance --- El lun, 11/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" , "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 11:28 > Hi all: > although my understanding of the problem is not that good, > I think there are two routes for this abstract way one could > try to prove the correctness of some implementation by > setting directly instrumention code to try to see for your > self dynamically if you may that or what you want to see: > http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s > > This could would run with just 3 missing modules from this > sources besides their examples, maybe another time the HP > guys allow us to see their works on that: > http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ > > This package is missing from public repositories Elego too > as I know, though they were in (see bottom links dead as of > now, but, ... maybe you can look for yourselves): > http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > > I know some ESC/Modula-3 tries to make some inferencing of > invariants, but as I said it dones't mean this is totally > computable, im fact in general its isn't, but it would be > good to try. > > Essentially where are the codes for showing that, or are > you implementing this as private code? > > Thanks in advance > > --- El lun, 11/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > > Para: "Jay K" > > CC: m3devel at elegosoft.com > > Fecha: lunes, 11 de abril, 2011 04:01 > > I'm not sure about impossible. > > But maybe... > > > > Here's the situation. > > > > I'm using unix time in my code. Sometimes it > needs to > > run on Windows. > > Some versions of Modula-3 on Windows use the Windows > epoch > > for timekeeping. > > So I first request the time at midnight UTC Jan 1 1970 > and > > set that to > > "UnixEpoch". On Unix systems I expect it to be > zero, > > on some other systems > > it might be another number. > > > > Now I've found what the problem is, and it's not > anything > > you have done. > > In fact PM3 exhibits the same problem if you set > everything > > up precisely > > the same. > > > > The issue is that the server I am using is located in > the > > United Kingdom > > and it has its time zone set to Europe/London. > > Nowadays the Brits use the > > following definitions for time: > > > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + > 0 > > in summer "British Summer Time" abbrev. "BST" = UTC + > 3600 > > > > However, between October 26, 1968 and October 31, > 1971, > > they used something > > else, to wit: > > > > all year "British Standard Time" abbrev. "BST" = UTC > + > > 3600 > > > > Note the confusing abbreviations. British > Standard > > Time (which no longer > > exists) and British Summer Time have the same > > abbreviation. You might think > > this is sensible, because British Summer Time has the > same > > value as British > > Standard Time (when the former is defined). > > > > There is just one problem. British Standard Time > is > > not a "daylight > > savings time" whereas British Summer Time *is*. > > Witness the output of > > zdump for London: > > > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun > Mar > > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun > Oct > > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun > Oct > > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun > Feb > > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun > Feb > > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun > Oct > > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun > Mar > > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun > Mar > > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun > Oct > > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > > > The following lines are the special ones: > > > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > > > Note how the gmtoff stays the same but isdst changes. > > > And this of course > > straddles the definition of Unix time at Jan 1 1970 > > 00:00:00 UTC. > > > > Clearly there is a bug in M3's date code here. I > just > > don't understand what > > invariant the original author appealed to when he > inserted > > the dst-changing code. > > It is hardly unusual for a given locale to have > changed > > time zone rules... > > it is completely wrong for that to affect the > conversion of > > times for a *different* > > locale (here I am just trying to do a time conversion > for > > UTC, it shouldn't > > matter what country my computer is in when it does > that!) > > > > Mika > > > > Jay K writes: > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/plain; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention. > > >Which I might not have right now=2C but which I > > definitely tried to apply a= > > >t the time. > > > > > >Can you compare with some historical versions? > > >e.g. > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > > > > >and notice: > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > >/POSIX/TimePosix.m3 > > >Revision 1.3: download - view: text=2C markup=2C > > annotated - select for dif= > > >fs > > >Note: This also fixes the UTC timezone on > Solaris=2C > > Interix=2C HP-UX=2C Cy= > > >gwin=2C which appear to have been historically > broken. > > > > > > > > >I'm usually a bit nervous when I make accusations > of > > long-term breakage. > > > > > > > > >And yeah there is a general problem of determining > when > > daylight savings is= > > > in effect. > > >The rules very often change. The operating system > needs > > a lot of data to kn= > > >ow this. > > > > > > > > >This stuff should be easy to debug without m3gdb. > > >Just print stuff to stdout liberally. > > > > > >Aha=2C see here: > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > > > (* adjust result to reflect "d.offset" *) > > > EVAL Utime.time(ADR(now))=3B > > > local_now :=3D > > Utime.localtime(ADR(now))=3B > > > IF local_now.tm_isdst > 0 THEN > > > (* decrement the local time > zone > > by one hour if DST is in effect *) > > > DEC(local_now.tm_gmtoff=2C 1 > * > > SecsPerHour) > > > END=3B > > > > > > - Jay > > > > > >> To: m3devel at elegosoft.com=3B > > jay.krell at cornell.edu > > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > > >> From: mika at async.caltech.edu > > >> Subject: [M3devel] time conversion problems > : > > Date.ToTime on AMD64_LINUX > > >>=20 > > >> Hello m3devel (especially Jay)=2C > > >>=20 > > >> I'm having some time conversion > problems. I > > *believe* (a bit hard to > > >> verify because m3gdb isn't working!!) that > > Date.ToTime called on > > >> Jan 1 1970 in timezone "UTC" or "GMT" is > returning > > -3600 for me. > > >>=20 > > >> I note the following (fairly recent) code in > > DatePosixC.c: > > >>=20 > > >> #ifdef _TIME64_T > > >> t =3D > > mktime64(&tm)=3B > > >> #else > > >> t =3D > mktime(&tm)=3B > > >> #endif > > >> #ifdef DATE_BSD > > >> if (t =3D=3D -1) > > >> goto > > Exit=3B > > >>=20 > > >> /* adjust result to > > reflect "date->offset" */ > > >> #ifdef _TIME64_T > > >> time64(&now)=3B > > >> local_now =3D > > localtime64(&now)=3B > > >> #else > > >> time(&now)=3B > > >> local_now =3D > > localtime(&now)=3B > > >> #endif > > >> assert(local_now > !=3D > > NULL)=3B > > >> if > (local_now->tm_isdst > > > 0) > > >> /* decrement > the > > local time zone by one hour if DST is in effect */ > > >> > > local_now->m3_tm_gmtoff -=3D > > SecsPerHour=3B > > >>=20 > > >> /* As above=2C we > must > > negate "date->offset" to account for the > > >> opposite sense of > that > > field compared to Unix. */ > > >> t -=3D > ((-date->offset) > > - local_now->m3_tm_gmtoff)=3B > > >> Exit: > > >> #endif > > >> return t=3B > > >>=20 > > >>=20 > > >> What I don't understand is the subtracting > off of > > SecsPerHour if > > >> "now" is DST. Yeah=2C now is DST (it's > > April)=2C but the time I'm > > >> converting was in the winter=2C and besides > GMT > > and UTC don't even > > >> have DST.=20 > > >>=20 > > >> I don't understand the intent of the code.. > can > > someone explain why > > >> we're subtracting off an hour? And why > is it > > based on the value of=20 > > >> tm_isdst of the *current* time? Some > > additional documentation within > > >> the file would be helpful too. > > >>=20 > > >> Note my conversion returns the correct 0 on > > various other M3 > > >> implementations. > > >>=20 > > >> Note the system time on my system is in a > timezone > > that has DST=20 > > >> (namely=2C BST) but I have changed the TZ > > environment variable to UTC or = > > >GMT > > >> before calling this code. > > >>=20 > > >> Mika > > > > > > > > > = > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/html; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention.
Which I might not have > right > > now=2C but which I definitel= > > >y tried to apply at the > time.

Can > > you compare with some historical v= > > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > > > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > > > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > > >lain


and > > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > > > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > > 1.3 > >b>: > > >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fplain" > > class=3D"download-link">download - view: > > href=3D"ht= > > > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > > > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > > class=3D"display= > > >-link">text=2C > > >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fx-cvsweb-markup" > > class=3D"display-link">markup=2C > href=3D"= > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > > - > > >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > > > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > > This a= > > >lso fixes the UTC timezone on Solaris=2C > Interix=2C > > HP-UX=2C Cygwin=2C whic= > > >h appear to have been historically > > broken.


I'm usually a bit > ner= > > >vous when I make accusations of long-term > > breakage.


And yeah the= > > >re is a general problem of determining when > daylight > > savings is in effect.<= > > >br>The rules very often change. The operating > system > > needs a lot of data to= > > > know this.


This > stuff > > should be easy to debug without m3gdb. > >>Just print stuff to stdout > > liberally.

Aha=2C see > here:
http://mo= > > > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > > > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

> 
> > (*=
> > > adjust result to reflect "d.offset" *)

> 
> >   EVAL Utime.time(ADR(now))=3B=
> > >
    local_now :=3D
> > Utime.localtime(ADR(now))=3B
   
> IF
> > local_now.tm_=
> > >isdst >=3B 0 THEN
 
>    
> > (* decrement the local time zone by one hour i=
> > >f DST is in effect *)
 
>    
> > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> > >br>   
> > END=3B

 =3B- > > Jay

>=3B To: > m3devel at elegosof= > > >t.com=3B jay.krell at cornell.edu
>=3B > > Date: Sun=2C 10 Apr 2011 15:45:16 = > > >-0700
>=3B From: mika at async.caltech.edu
>=3B > > Subject: [M3devel] t= > > >ime conversion problems : Date.ToTime on > > AMD64_LINUX
>=3B
>=3B > H= > > >ello m3devel (especially > Jay)=2C
>=3B > >
>=3B I'm having some time = > > >conversion problems. I *believe* (a bit > hard > > to
>=3B verify because m= > > >3gdb isn't working!!) that Date.ToTime called > > on
>=3B Jan 1 1970 in ti= > > >mezone "UTC" or "GMT" is returning -3600 for > > me.
>=3B
>=3B I not= > > >e the following (fairly recent) code in > > DatePosixC.c:
>=3B
>=3B > = > > >#ifdef _TIME64_T
>=3B > > t =3D > > mktime64(&=3Btm)=3B
>=3B #else= > > >
>=3B t > =3D > > mktime(&=3Btm)=3B
>=3B > > #endif
>=3B #ifdef= > > > DATE_BSD
>=3B > > if (t =3D=3D -1)
>=3B > > goto Exit=3B
= > > >>=3B
>=3B > > /* adjust result to reflect > > "date->=3Boffset" */ > >>>=3B #ifdef > > _TIME64_T
>=3B > > > time64(&=3Bnow)=3B
>=3B > > = > > >local_now =3D > > localtime64(&=3Bnow)=3B
>=3B > > #else
>=3B > time= > > >(&=3Bnow)=3B
>=3B > > local_now =3D > > localtime(&=3Bnow)=3B
>= > > >=3B #endif
>=3B > > assert(local_now !=3D > > NULL)=3B
>=3B > if (l= > > >ocal_now->=3Btm_isdst >=3B > > 0)
>=3B > /* > > decrement the local ti= > > >me zone by one hour if DST is in effect > > */
>=3B > > local_now->=3B= > > >m3_tm_gmtoff -=3D > SecsPerHour=3B
>=3B > >
>=3B /* As > above=2C > > we= > > > must negate "date->=3Boffset" to account > for > > the
>=3B > opposi= > > >te sense of that field compared to Unix. > > */
>=3B t -=3D > > ((-date-&g= > > >t=3Boffset) - > > > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > > Exit:
>=3B #e= > > >ndif
>=3B > return > > t=3B
>=3B
>=3B > >
>=3B What I don't= > > > understand is the subtracting off of SecsPerHour > > if
>=3B "now" is DST= > > >. Yeah=2C now is DST (it's April)=2C but the > time > > I'm
>=3B converting= > > > was in the winter=2C and besides GMT and UTC > don't > > even
>=3B have DST= > > >.
>=3B
>=3B I > don't > > understand the intent of the code.. can som= > > >eone explain why
>=3B we're > subtracting > > off an hour? And why is it ba= > > >sed on the value of
>=3B tm_isdst > of > > the *current* time? Some additi= > > >onal documentation within
>=3B the > file > > would be helpful too.
>= > > >=3B
>=3B Note my conversion > returns the > > correct 0 on various other M3= > > >
>=3B > > implementations.
>=3B >
>=3B > > Note the system time on my= > > > system is in a timezone that has DST > >
>=3B (namely=2C BST) but I have= > > > changed the TZ environment variable to UTC or > > GMT
>=3B before calling= > > > this code.
>=3B > >
>=3B > Mika
> > > > > > > > > > > >= > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > > > From rodney_bates at lcwb.coop Thu Apr 14 17:45:06 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 14 Apr 2011 10:45:06 -0500 Subject: [M3devel] Thread local storage? Message-ID: <4DA71682.9040709@lcwb.coop> I have never understood what "thread local storage" is all about, in particular, what need it addresses. All local variables and parameters are already private to a thread, as consequence of semantics going back as far as Algol-60. They are allocated space when and only when a call is executed. That can't happen until there is a thread to execute the call and it has to have its own private stack where the space is allocated, or it wouldn't work as a thread, (OK leaving out languages like very old Fortrans that don't support recursion either.) That leaves global variables. At least some of these have to be shared by all the threads of a process, or the threads could have no shared memory interaction, and thus no function that would not be served by just separate processes running separate main programs. If you need variables that are private to a thread but in some sense global, just put them in the Thread.Closure. They would hardly make sense as declared lexically global variables anyway, since, except for the initial thread provided by the OS/RTS, their lifetime does not cover the entire execution of the main program. So what are they really, and what are they for? And do we even have or use them in Modula-3? It is looking to me like some sort of accommodation for them in cm3cg may be what has undermined m3gdb in the CVS head. Rodney Bates From hosking at cs.purdue.edu Thu Apr 14 23:05:27 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 17:05:27 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: <4DA71682.9040709@lcwb.coop> References: <4DA71682.9040709@lcwb.coop> Message-ID: I think that these were introduced into the C libraries by Jay. On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > I have never understood what "thread local storage" is all about, in particular, > what need it addresses. > > All local variables and parameters are already private to a thread, as consequence > of semantics going back as far as Algol-60. They are allocated space when and > only when a call is executed. That can't happen until there is a thread to > execute the call and it has to have its own private stack where the space is > allocated, or it wouldn't work as a thread, (OK leaving out languages like > very old Fortrans that don't support recursion either.) > > That leaves global variables. At least some of these have to be shared by all > the threads of a process, or the threads could have no shared memory interaction, > and thus no function that would not be served by just separate processes running > separate main programs. > > If you need variables that are private to a thread but in some sense global, > just put them in the Thread.Closure. They would hardly make sense as declared > lexically global variables anyway, since, except for the initial thread provided > by the OS/RTS, their lifetime does not cover the entire execution of the main > program. > > So what are they really, and what are they for? And do we even have or use them in > Modula-3? > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > undermined m3gdb in the CVS head. > > Rodney Bates From jay.krell at cornell.edu Thu Apr 14 23:27:10 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:27:10 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, Message-ID: [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 14 23:32:22 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:32:22 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Apr 15 02:28:12 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 20:28:12 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend. I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > > We've always had thread locals > > ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? > > - Jay > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > Date: Thu, 14 Apr 2011 21:27:10 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > [I wrote a long answer here but it seemed to have gotten lost.] > > We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. > On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. > There is nothing in cm3cg about this. > > http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c > Revision 1.155: download - view: text, markup, annotated - select for diffs > Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell > Branches: MAIN > Diff to: previous 1.154: preferred, unified > Changes since revision 1.154: +2 -1 lines > Don't use __thread on Solaris. > It failed to link. > See: > http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console > Revision 1.151: download - view: text, markup, annotated - select for diffs > Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell > Branches: MAIN > Diff to: previous 1.150: preferred, unified > Changes since revision 1.150: +30 -0 lines > Use __thread on Linux and Solaris. > Doesn't compile on Apple. > Segfaults on OpenBSD and NetBSD. > Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. > In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker > or C++ exceptions (C++ backend). > > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Thu, 14 Apr 2011 17:05:27 -0400 > > To: rodney_bates at lcwb.coop > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Thread local storage? > > > > I think that these were introduced into the C libraries by Jay. > > > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > > > I have never understood what "thread local storage" is all about, in particular, > > > what need it addresses. > > > > > > All local variables and parameters are already private to a thread, as consequence > > > of semantics going back as far as Algol-60. They are allocated space when and > > > only when a call is executed. That can't happen until there is a thread to > > > execute the call and it has to have its own private stack where the space is > > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > > very old Fortrans that don't support recursion either.) > > > > > > That leaves global variables. At least some of these have to be shared by all > > > the threads of a process, or the threads could have no shared memory interaction, > > > and thus no function that would not be served by just separate processes running > > > separate main programs. > > > > > > If you need variables that are private to a thread but in some sense global, > > > just put them in the Thread.Closure. They would hardly make sense as declared > > > lexically global variables anyway, since, except for the initial thread provided > > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > > program. > > > > > > So what are they really, and what are they for? And do we even have or use them in > > > Modula-3? > > > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > > undermined m3gdb in the CVS head. > > > > > > Rodney Bates > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 15 04:46:46 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 15 Apr 2011 02:46:46 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , , , , , Message-ID: __thread is only used in C code. The difference is in the gcc backend within the gcc C/C++ compiler. Not cm3cg. I put in using __thread for Linux. There is some potential for using it elsewhere, but that's all I did. e.g. Solaris sometimes, AIX. All other non-Windows targets use pthread_getspecific/setspecific. It is hidden from any Modula-3 code. It is all in C. m3gdb might might might need to know about it. It could also be that m3gdb gets confused by any C code linked in. Or that m3gdb is old (it is) and is confused by __thread (but again, m3gdb might or might not have a builtin need to often look at these, or maybe just the human debugger wants to see them). Currently no targets support stack walking. I couldn't get SOLgnu/SOLsun's stack walker to work with the gcc 4.5 backend. I really really tried. I tried not optimizing at all. And/or putting volatile everywhere. Tru64 also didn't work despite a bit of an attempt. I probably also tried not optimizing there. I probably didn't try as much as for SOLgnu/SOLsun due to the machine being slower and the platform being in less use. - Jay From: hosking at cs.purdue.edu Date: Thu, 14 Apr 2011 20:28:12 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend.I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAOffice +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 15 12:37:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:37:37 -0700 Subject: [M3devel] malloc/free from Modula-3? Message-ID: <20110415103737.E73B31A2078@async.async.caltech.edu> Hi m3devel, I'm debugging a problem with some code I have that calls out to C code. It's calling pqlib (PostgreSQL library) and you might expect things to go wrong there, but as it turns out it's dying in CM3's code. The system is AMD64_LINUX (user threading because of the problems with pthreads) and the program is just freezing up hard. No segfault, bus error or anything. It locks up a couple of times a day. What are the rules on calling malloc/free from Modula-3? Stack trace: (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) Here is the code that causes the problem: VAR s := CopyTtoS(query); BEGIN TRY t.res := t.myExec(s,busyWait) FINALLY FreeCopiedS(s) (* stuck here *) END END; The query is a SQL query---t.myExec eventually calls the libpq SQL exec routine (PQexec I think it's called). Since this is already an unsafe module I suppose I could just make an array of CHARs on the stack and pass a pointer to that... Regards, Mika From mika at async.caltech.edu Fri Apr 15 12:46:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:46:47 -0700 Subject: [M3devel] malloc/free from Modula-3 (more details) Message-ID: <20110415104647.6AFF81A2078@async.async.caltech.edu> Here is some more info. Note that all my access to the pq library is inside a single giant lock (I've run this code for years and years but not under CM3 and not under AMD64_LINUX or for that matter any kind of Linux). The locking is not likely to be obviously wrong, but of course there could be something subtly wrong. In gdb the program is completely frozen: (gdb) stepi ^C Program received signal SIGINT, Interrupt. 0x00002ac519a9202e in ?? () from /lib/libc.so.6 (gdb) info registers rax 0xfffffffffffffe00 -512 rbx 0x2ac519d059e0 47026030008800 rcx 0xffffffffffffffff -1 rdx 0x2 2 rsi 0x80 128 rdi 0x2ac519d059e0 47026030008800 rbp 0x2ac63b0ef0e0 0x2ac63b0ef0e0 rsp 0x2ac63b0ef028 0x2ac63b0ef028 r8 0x2fe2620 50210336 r9 0x2ac6400008ef 47030965635311 r10 0x0 0 r11 0x302 770 r12 0x411858 4266072 r13 0x1 1 r14 0x2f5b6d0 49657552 r15 0x0 0 rip 0x2ac519a9202e 0x2ac519a9202e eflags 0x202 [ IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 fctrl 0x37f 895 fstat 0x0 0 ftag 0xffff 65535 fiseg 0x0 0 fioff 0x0 0 foseg 0x0 0 fooff 0x0 0 fop 0x0 0 mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ] (gdb) (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) disassemble 0x00002ac519a9202e No function contains specified address. (gdb) disassemble 0x00002ac519a2caab Dump of assembler code for function free: 0x00002ac519a2ca40 : mov 0x2d7519(%rip),%rax # 0x2ac519d03f60 0x00002ac519a2ca47 : push %rbx 0x00002ac519a2ca48 : mov %rdi,%r8 0x00002ac519a2ca4b : mov (%rax),%rax 0x00002ac519a2ca4e : test %rax,%rax 0x00002ac519a2ca51 : je 0x2ac519a2ca60 0x00002ac519a2ca53 : mov 0x8(%rsp),%rsi 0x00002ac519a2ca58 : mov %rax,%r11 0x00002ac519a2ca5b : pop %rbx 0x00002ac519a2ca5c : jmpq *%r11 0x00002ac519a2ca5f : nop 0x00002ac519a2ca60 : test %rdi,%rdi 0x00002ac519a2ca63 : je 0x2ac519a2cad2 0x00002ac519a2ca65 : mov -0x8(%rdi),%rax 0x00002ac519a2ca69 : lea -0x10(%rdi),%rdx 0x00002ac519a2ca6d : test $0x2,%al 0x00002ac519a2ca6f : jne 0x2ac519a2cad8 0x00002ac519a2ca71 : test $0x4,%al 0x00002ac519a2ca73 : lea 0x2d8f66(%rip),%rbx # 0x2ac519d059e0 0x00002ac519a2ca7a : je 0x2ac519a2ca86 0x00002ac519a2ca7c : and $0xfffffffffc000000,%rdx 0x00002ac519a2ca83 : mov (%rdx),%rbx 0x00002ac519a2ca86 : mov $0x1,%esi 0x00002ac519a2ca8b : xor %eax,%eax 0x00002ac519a2ca8d : cmpl $0x0,0x2dc594(%rip) # 0x2ac519d09028 0x00002ac519a2ca94 : je 0x2ac519a2caa2 0x00002ac519a2ca96 : lock cmpxchg %esi,(%rbx) 0x00002ac519a2ca9a : jne 0x2ac519a2fb34 0x00002ac519a2caa0 : jmp 0x2ac519a2caab 0x00002ac519a2caa2 : cmpxchg %esi,(%rbx) 0x00002ac519a2caa5 : jne 0x2ac519a2fb34 0x00002ac519a2caab : mov %r8,%rsi 0x00002ac519a2caae : mov %rbx,%rdi 0x00002ac519a2cab1 : callq 0x2ac519a2c2b0 0x00002ac519a2cab6 : cmpl $0x0,0x2dc56b(%rip) # 0x2ac519d09028 0x00002ac519a2cabd : je 0x2ac519a2caca 0x00002ac519a2cabf : lock decl (%rbx) 0x00002ac519a2cac2 : jne 0x2ac519a2fb4f 0x00002ac519a2cac8 : jmp 0x2ac519a2cad2 0x00002ac519a2caca : decl (%rbx) 0x00002ac519a2cacc : jne 0x2ac519a2fb4f 0x00002ac519a2cad2 : pop %rbx 0x00002ac519a2cad3 : retq 0x00002ac519a2cad4 : nopl 0x0(%rax) 0x00002ac519a2cad8 : mov 0x2d97c6(%rip),%ecx # 0x2ac519d062a4 0x00002ac519a2cade : test %ecx,%ecx 0x00002ac519a2cae0 : jne 0x2ac519a2cb10 0x00002ac519a2cae2 : cmp 0x2d97a7(%rip),%rax # 0x2ac519d06290 0x00002ac519a2cae9 : jbe 0x2ac519a2cb10 0x00002ac519a2caeb : cmp $0x2000000,%rax 0x00002ac519a2caf1 : ja 0x2ac519a2cb10 0x00002ac519a2caf3 : and $0xfffffffffffffff8,%rax 0x00002ac519a2caf7 : mov %rax,0x2d9792(%rip) # 0x2ac519d06290 0x00002ac519a2cafe : add %rax,%rax 0x00002ac519a2cb01 : mov %rax,0x2d9778(%rip) # 0x2ac519d06280 0x00002ac519a2cb08 : nopl 0x0(%rax,%rax,1) ---Type to continue, or q to quit--- 0x00002ac519a2cb10 : pop %rbx 0x00002ac519a2cb11 : mov %rdx,%rdi 0x00002ac519a2cb14 : jmpq 0x2ac519a2bc40 End of assembler dump. Mika From hosking at cs.purdue.edu Fri Apr 15 17:26:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Apr 2011 11:26:15 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110415103737.E73B31A2078@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > Hi m3devel, > > I'm debugging a problem with some code I have that calls out to C > code. It's calling pqlib (PostgreSQL library) and you might expect > things to go wrong there, but as it turns out it's dying in CM3's code. > > The system is AMD64_LINUX (user threading because of the problems with > pthreads) and the program is just freezing up hard. No segfault, > bus error or anything. It locks up a couple of times a day. > > What are the rules on calling malloc/free from Modula-3? > > Stack trace: > > (gdb) where > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > ) at ../src/C/Common/CstdlibC.c:26 > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > ) at ../src/C/Common/M3toC.m3:46 > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > ) at ../src/UnsafeDatabase.m3:230 > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > ) at ../src/UnsafeDatabase.m3:228 > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > ) at ../src/UnsafeDatabase.m3:258 > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > ) at ../src/DesynchronizedDB.m3:126 > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > ) at ../src/DesynchronizedDB.m3:293 > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > ) at ../src/GCOMSOrderManager.m3:596 > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > ) at ../src/GCOMSOrderManager.m3:310 > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > ) at ../src/EuroDriver.m3:1108 > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > ) at ../src/EuroDriver.m3:1545 > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > #17 0x0000000000000000 in ?? () > (gdb) > > Here is the code that causes the problem: > > VAR > s := CopyTtoS(query); > BEGIN > TRY > t.res := t.myExec(s,busyWait) > FINALLY > FreeCopiedS(s) (* stuck here *) > END > END; > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > routine (PQexec I think it's called). > > Since this is already an unsafe module I suppose I could just make an > array of CHARs on the stack and pass a pointer to that... > > > Regards, > Mika From mika at async.caltech.edu Fri Apr 15 21:03:49 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 12:03:49 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> References: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <20110415190349.E4F531A2078@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >although my understanding of the problem is not that good, I think there ar= >e two routes for this abstract way one could try to prove the correctness o= >f some implementation by setting directly instrumention code to try to see = >for your self dynamically if you may that or what you want to see: >http://www.youtube.com/watch?v=3DzIgu9q0vVc0#t=3D2m06s > >This could would run with just 3 missing modules from this sources besides = >their examples, maybe another time the HP guys allow us to see their works = >on that: >http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D= >/ > >This package is missing from public repositories Elego too as I know, thoug= >h they were in (see bottom links dead as of now, but, ... maybe you can loo= >k for yourselves): >http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > >I know some ESC/Modula-3 tries to make some inferencing of invariants, but = >as I said it dones't mean this is totally computable, im fact in general it= >s isn't, but it would be good to try. > >Essentially where are the codes for showing that, or are you implementing t= >his as private code? The problem is in DatePosix.m3 . To reproduce, set your TZ environment variable to Europe/London and try to convert a Date of 1970-01-01 @ 00:00:00 in the time zone UTC to a Time.T. You should get 0; you will get -3600. Mika > >Thanks in advance > >--- El lun, 11/4/11, Mika Nystrom escribi=F3: > From dragisha at m3w.org Sun Apr 17 16:05:58 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 16:05:58 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? Message-ID: I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? TIA, dd From hosking at cs.purdue.edu Sun Apr 17 17:25:04 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 11:25:04 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: Message-ID: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? > > TIA, > dd > From dabenavidesd at yahoo.es Sun Apr 17 17:31:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 16:31:54 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: Message-ID: <274332.77724.qm@web29720.mail.ird.yahoo.com> Hi all: it's possible to get yet another implementation of it, perhaps when we have the ESC we will be able to check it for Lock level annotations, earlier there was an implementation: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 Is important to keep in mind this await construct annotation may be verified as the one in Locking Level, it would be interesting to see whether this means one can execute commands without locking, is in some way specifying locks underneath. If you want to ask, if their statements is sound it would be good to ask if they might have a copy of the implementation If this is better to check for checks under Pthreads module would be a good idea to use directly their semantics. Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" > Fecha: domingo, 17 de abril, 2011 09:05 > I am still puzzled with this... What > exactly do we have now? Do we have a version where things > work, or something? Or we don't have threads for cm3 > anymore? > > TIA, > dd > > From dragisha at m3w.org Sun Apr 17 18:10:33 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 18:10:33 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. > > On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > >> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >> >> TIA, >> dd >> > From dabenavidesd at yahoo.es Sun Apr 17 18:04:08 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 17:04:08 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <274332.77724.qm@web29720.mail.ird.yahoo.com> Message-ID: <617521.89071.qm@web29714.mail.ird.yahoo.com> Hi all: And yet to more on this, the first on the implementation of mutex abstraction theory based on await construct: http://www.cs.rug.nl/~wim/pub/whh233.ps.gz And better than that I guess, this with Useful Guarded Commands of Dijkstra's language, this is really a good idea to use it inside the ESC checker since it's its internal AST rep: http://www.cs.rug.nl/~wim/onderwijs/concurrency/dikt.pdf.gz Thanks in advance: --- El dom, 17/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" , "Dragi?a Duri?" > Fecha: domingo, 17 de abril, 2011 10:31 > Hi all: > it's possible to get yet another implementation of it, > perhaps when we have the ESC we will be able to check it for > Lock level annotations, earlier there was an > implementation: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 > > Is important to keep in mind this await construct > annotation may be verified as the one in Locking Level, it > would be interesting to see whether this means one can > execute commands without locking, is in some way specifying > locks underneath. > > If you want to ask, if their statements is sound it would > be good to ask if they might have a copy of the > implementation > > If this is better to check for checks under Pthreads module > would be a good idea to use directly their semantics. > > Thanks in advance > > --- El dom, 17/4/11, Dragi?a Duri? > escribi?: > > > De: Dragi?a Duri? > > Asunto: [M3devel] State of threads? pthreads not safe > to use on some platforms? > > Para: "m3devel" > > Fecha: domingo, 17 de abril, 2011 09:05 > > I am still puzzled with this... What > > exactly do we have now? Do we have a version where > things > > work, or something? Or we don't have threads for cm3 > > anymore? > > > > TIA, > > dd > > > > > From hosking at cs.purdue.edu Sun Apr 17 18:43:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 12:43:16 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> So the question is what has changed. I know threads were functional and stable at one point. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 17, 2011, at 12:10 PM, Dragi?a Duri? wrote: > Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... > > On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > >> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. >> >> On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: >> >>> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >>> >>> TIA, >>> dd >>> >> > From hendrik at topoi.pooq.com Sun Apr 17 19:34:55 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 17 Apr 2011 13:34:55 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> Message-ID: <20110417173455.GB11082@topoi.pooq.com> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: > So the question is what has changed. > I know threads were functional and stable at one point. Was that back when they were green threads? Has our use of posix threads ever been stable? Are green threads still available? -- hendrik From hosking at cs.purdue.edu Sun Apr 17 20:13:01 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 14:13:01 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <20110417173455.GB11082@topoi.pooq.com> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >> So the question is what has changed. >> I know threads were functional and stable at one point. > > Was that back when they were green threads? User threads have historically been pretty stable, and I believe still are. Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > Has our use of posix threads ever been stable? > Are green threads still available? > > -- hendrik From dragisha at m3w.org Sun Apr 17 20:18:04 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 20:18:04 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> But, what recent change when my now problematic platform is on 5.8.6....? On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From dabenavidesd at yahoo.es Sun Apr 17 23:21:47 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 22:21:47 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <946881.5570.qm@web29717.mail.ird.yahoo.com> Hi all: it's good to check if at all the platform per target and per threading too, SMP/HT, x86 or AMD64. I'm suspecting more than anything else this really affects specially SMP/HT, but specially not missing that information, perhaps even historically we can't assure that, at least that's how I see it without it, what do you think? Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: domingo, 17 de abril, 2011 13:18 > But, what recent change when my now > problematic platform is on 5.8.6....? > > On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > > > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > > > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony > Hosking wrote: > >>> So the question is what has changed. > >>> I know threads were functional and stable at > one point. > >> > >> Was that back when they were green threads? > > > > User threads have historically been pretty stable, and > I believe still are. > > Similarly, our use of pthreads was stable at one point > but there have been recent changes that appear to have > degraded them. > > > >> Has our use of posix threads ever been stable? > >> Are green threads still available? > >> > >> -- hendrik > > > > From dragisha at m3w.org Mon Apr 18 14:07:40 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 18 Apr 2011 14:07:40 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <7D82E5E5-EC36-4254-A4A7-782A230320FD@m3w.org> User threads were stable, but they had problems of their own... I met some of them when I implemented first pthread support - user threads switching is in order, and pthread switching is out of order. m3pp was failing because of that. On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From mika at async.caltech.edu Mon Apr 18 18:10:28 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:10:28 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Tony Hosking writes: >I suspect the problem is re-entrance to the library. Should you be = >disabling user thread switching somewhere? I suspect the same. That's why I asked what the "rules" are for calling malloc and free. I notice that the m3core code has Scheduler.DisableSwitching around all calls to malloc but not around calls to free. Why would one be unsafe and the other safe? It is always free that my program hangs up on by the way. Surrounding all "suspicious" calls in my code with Scheduler.DisableSwitching doesn't solve the problem, as far as I can tell. I am further suspecting that my problem is related to my having linked the program with -pthread. I am also guessing that this is the same reason you can't link FreeBSD user threads programs with -pthread: namely that if you go reentrantly into the pthread malloc library some code assumes the two invocations are from separate threads and locks up if they are from the same thread. That's my guess anyhow. But that doesn't explain why I've never seen any malloc/free problems on FreeBSD without -pthread. (I don't think I have, anyhow.) Mika From hosking at cs.purdue.edu Mon Apr 18 18:30:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 12:30:45 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Yes, I think you should avoid -pthread. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > Tony Hosking writes: >> I suspect the problem is re-entrance to the library. Should you be = >> disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the "rules" are > for calling malloc and free. I notice that the m3core code has > Scheduler.DisableSwitching around all calls to malloc but not around calls > to free. Why would one be unsafe and the other safe? It is always free > that my program hangs up on by the way. Surrounding all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my having linked the > program with -pthread. I am also guessing that this is the same reason > you can't link FreeBSD user threads programs with -pthread: namely that if > you go reentrantly into the pthread malloc library some code assumes > the two invocations are from separate threads and locks up if they are from > the same thread. That's my guess anyhow. But that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without -pthread. > (I don't think I have, anyhow.) > > Mika From mika at async.caltech.edu Mon Apr 18 18:42:39 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:42:39 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Message-ID: <20110418164239.84CEB1A207A@async.async.caltech.edu> Tony Hosking writes: >Well, that is reassuring news. >If the problems are isolated to threading then they can be resolved easily. > >On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > >> >> Remember that for mission-critical applications there's always user >> threading.. I find that the system (up to and including the CVS head >> and on both amd64 and i386 flavors of Linux as well as every other OS I >> have tried) is very very stable with user threading. I'm eager to use >> pthreads again of course, for all sorts of reasons... >> >> Mika Actually I'm afraid I have to take this back. After digging a bit here, I *think* user threads are OK with the proviso that one not link with -pthread. But I'm not sure. Unfortunately, user threads require -pthread on most systems because of pthread_atfork. I asked Jay what pthread_atfork is used for *under user threads* but he never answered my question. I had understood that pthread_atfork is used in the pthreads implementation to simulate the old behavior of user threads and that it wasn't needed under user threads... I'm also not sure how to change RTProcessC.c so that it picks up pthread_atfork exactly when user threading has been specified. (Programs won't link with pthread_atfork in them without -pthread.) In any case I am pretty sure that linking with -pthread picks up versions of malloc and free that really don't like being called reentrantly---and by "don't like" I mean the worst kind of crash: the application locks up hard in some sort of spin lock. A segfault would be preferable by far. And m3core has a lot of unprotected calls to free in it. There may be some really nasty bugs here. They are nasty because they kill reliability, kill liveness, and can't really be programmed around because they are in such a vital subsystem (namely threading). I would go back to PM3 if I could but I think I'm stuck with CM3 now. Mika P.S. as far as the pthreads bugs go I think it's still a possibility the bugs are in the garbage collector? The user threads m3core disables the garbage collector at different points from where the pthreads m3core does, no? From hosking at cs.purdue.edu Mon Apr 18 19:05:46 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 13:05:46 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418164239.84CEB1A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> Message-ID: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> On Apr 18, 2011, at 12:42 PM, Mika Nystrom wrote: > Tony Hosking writes: >> Well, that is reassuring news. >> If the problems are isolated to threading then they can be resolved easily. >> >> On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: >> >>> >>> Remember that for mission-critical applications there's always user >>> threading.. I find that the system (up to and including the CVS head >>> and on both amd64 and i386 flavors of Linux as well as every other OS I >>> have tried) is very very stable with user threading. I'm eager to use >>> pthreads again of course, for all sorts of reasons... >>> >>> Mika > > > Actually I'm afraid I have to take this back. After digging a bit here, > I *think* user threads are OK with the proviso that one not link with > -pthread. But I'm not sure. > > Unfortunately, user threads require -pthread on most systems because > of pthread_atfork. I asked Jay what pthread_atfork is used for *under > user threads* but he never answered my question. I had understood > that pthread_atfork is used in the pthreads implementation to simulate > the old behavior of user threads and that it wasn't needed under user > threads... I'm also not sure how to change RTProcessC.c so that it > picks up pthread_atfork exactly when user threading has been specified. > (Programs won't link with pthread_atfork in them without -pthread.) pthread_atfork should not be needed under user threads. > > In any case I am pretty sure that linking with -pthread picks up versions > of malloc and free that really don't like being called reentrantly---and > by "don't like" I mean the worst kind of crash: the application locks up > hard in some sort of spin lock. A segfault would be preferable by far. > And m3core has a lot of unprotected calls to free in it. > > There may be some really nasty bugs here. They are nasty because they > kill reliability, kill liveness, and can't really be programmed around > because they are in such a vital subsystem (namely threading). I would > go back to PM3 if I could but I think I'm stuck with CM3 now. > > Mika > > P.S. as far as the pthreads bugs go I think it's still a possibility the > bugs are in the garbage collector? Not so much bugs in GC as bugs in thread interactions with GC perhaps. The GC itself has been stable for a long time. > The user threads m3core disables > the garbage collector at different points from where the pthreads m3core > does, no? From dabenavidesd at yahoo.es Mon Apr 18 19:17:35 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 18:17:35 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <886452.94715.qm@web29705.mail.ird.yahoo.com> Hi all: the only thing you need to keep control as of CM3 is concerned is the modula-3 free calls equals the number of news, that is, if not is runtime error, but it isn't vice versa, if you broke this then you are obviously unsafely corrupted, but as far as the process is concerned it will be growing in time the heap until termination. If this is exacerbated then you will run out of memory either in Modula-3 or C which is dangerous, so this tells you why the scheduler must be aware of this calls, at least that's my rationale. You are right I think there is a unixdb client program that crashes that can't be build in cm3 as it asks to provide their external implementation function calls so you can't declare external calls on your own, you need like compile it too, which is very strange, though my memories about this are very diffuse. Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 11:10 > > Tony Hosking writes: > >I suspect the problem is re-entrance to the library. > Should you be = > >disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the > "rules" are > for calling malloc and free. I notice that the m3core > code has > Scheduler.DisableSwitching around all calls to malloc but > not around calls > to free. Why would one be unsafe and the other > safe? It is always free > that my program hangs up on by the way. Surrounding > all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve > the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my > having linked the > program with -pthread. I am also guessing that this > is the same reason > you can't link FreeBSD user threads programs with -pthread: > namely that if > you go reentrantly into the pthread malloc library some > code assumes > the two invocations are from separate threads and locks up > if they are from > the same thread. That's my guess anyhow. But > that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without > -pthread. > (I don't think I have, anyhow.) > > Mika > > From mika at async.caltech.edu Mon Apr 18 20:37:20 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:37:20 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <20110418183720.E4C311A207A@async.async.caltech.edu> I don't think the pthreads implementation has ever been 100% correct. If it has, please let me know which CVS date to check out!!! (I use a PM3 that is almost ten years old simply because this stuff works and am happy to work with ANYTHING newer that works.) As far as user ("green") threads goes, I think they *may* be right if you are careful not to link with -pthread. This means you have to change RTThreadC.c (remove pthread_atfork) or your program won't link. (Please don't call pthreads "Posix" threads on this list: I know it is correct but it is confusing because Modula-3's user threads implementation for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 usage predates IEEE/ISO standardization of threads so there was no confusion at the time the name was introduced.) Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >But, what recent change when my now problematic platform is on = >5.8.6....? > >On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > >>=20 >> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>=20 >>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>> So the question is what has changed. >>>> I know threads were functional and stable at one point. >>>=20 >>> Was that back when they were green threads? >>=20 >> User threads have historically been pretty stable, and I believe still = >are. >> Similarly, our use of pthreads was stable at one point but there have = >been recent changes that appear to have degraded them. >>=20 >>> Has our use of posix threads ever been stable? >>> Are green threads still available? >>>=20 >>> -- hendrik >>=20 From mika at async.caltech.edu Mon Apr 18 20:45:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:45:40 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> Message-ID: <20110418184540.C2D801A2078@async.async.caltech.edu> Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From mika at async.caltech.edu Mon Apr 18 20:47:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:47:40 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <886452.94715.qm@web29705.mail.ird.yahoo.com> References: <886452.94715.qm@web29705.mail.ird.yahoo.com> Message-ID: <20110418184740.D20261A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >the only thing you need to keep control as of CM3 is concerned is the modul= >a-3 free calls equals the number of news, that is, if not is runtime error,= Not exactly. You can't necessarily call malloc/free from multiple threads. This is a big problem for Modula-3 programs that call out to nontrivial C code. (Why don't we wrap malloc/free ourselves in a special library? Linker problems I suppose... and I also suppose the effort should rather be on getting pthreads working again than messing around with problems with user threads...) Mika From mika at async.caltech.edu Mon Apr 18 21:01:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:01:44 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> <20110418183720.E4C311A207A@async.async.caltech.edu> <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> Message-ID: <20110418190144.406E41A207A@async.async.caltech.edu> If you have a copy, can you try the thread tester program on it? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >First implementation of pthreads for m3 was made for pm3 in December of = >2004... To the best of my knowledge, it was correct. As long as you do = >not define correct like "repetitive behaviour of green threads". I used = >it to run DC++ hub I wrote as a pet project for stress testign it. = >Worked well. > >On Apr 18, 2011, at 8:37 PM, Mika Nystrom wrote: > >> I don't think the pthreads implementation has ever been 100% correct. >>=20 >> If it has, please let me know which CVS date to check out!!! (I use a >> PM3 that is almost ten years old simply because this stuff works and = >am >> happy to work with ANYTHING newer that works.) >>=20 >> As far as user ("green") threads goes, I think they *may* be right if = >you >> are careful not to link with -pthread. This means you have to change = >RTThreadC.c >> (remove pthread_atfork) or your program won't link. >>=20 >> (Please don't call pthreads "Posix" threads on this list: I know it is >> correct but it is confusing because Modula-3's user threads = >implementation >> for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 = >usage=20 >> predates IEEE/ISO standardization of threads so there was no confusion = >at=20 >> the time the name was introduced.) >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>> But, what recent change when my now problematic platform is on =3D >>> 5.8.6....? >>>=20 >>> On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: >>>=20 >>>> =3D20 >>>> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>>> =3D20 >>>>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>>>> So the question is what has changed. >>>>>> I know threads were functional and stable at one point. >>>>> =3D20 >>>>> Was that back when they were green threads? >>>> =3D20 >>>> User threads have historically been pretty stable, and I believe = >still =3D >>> are. >>>> Similarly, our use of pthreads was stable at one point but there = >have =3D >>> been recent changes that appear to have degraded them. >>>> =3D20 >>>>> Has our use of posix threads ever been stable? >>>>> Are green threads still available? >>>>> =3D20 >>>>> -- hendrik >>>> =3D20 From dabenavidesd at yahoo.es Mon Apr 18 20:57:01 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 19:57:01 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418184740.D20261A207A@async.async.caltech.edu> Message-ID: <564805.90054.qm@web29714.mail.ird.yahoo.com> Hi all: ok, but are you telling this about in user threads or working ill or all threads and pthreads? Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 13:47 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >the only thing you need to keep control as of CM3 is > concerned is the modul= > >a-3 free calls equals the number of news, that is, if > not is runtime error,= > > Not exactly. You can't necessarily call malloc/free > from multiple threads. > This is a big problem for Modula-3 programs that call out > to nontrivial C code. > > (Why don't we wrap malloc/free ourselves in a special > library? Linker problems > I suppose... and I also suppose the effort should rather be > on getting pthreads > working again than messing around with problems with user > threads...) > > Mika > From mika at async.caltech.edu Mon Apr 18 21:05:55 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:05:55 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <564805.90054.qm@web29714.mail.ird.yahoo.com> References: <564805.90054.qm@web29714.mail.ird.yahoo.com> Message-ID: <20110418190555.526411A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >ok, but are you telling this about in user threads or working ill or all th= >reads and pthreads? >Thanks in advance > I don't know exactly. There are certainly bad combinations of threading implementations and memory allocators. And I'm sure it may be system dependent as well. In any case, linking user threads with the -pthreads version of libc is probably bad. Calling malloc willy-nilly from multiple user threads is probably bad. Calling free, I don't know. Pthreads currently don't work so maybe malloc works there but that doesn't help very much if the system is anyhow unstable. Mika >--- El lun, 18/4/11, Mika Nystrom escribi=F3: > >> De: Mika Nystrom >> Asunto: Re: [M3devel] malloc/free from Modula-3? >> Para: "Daniel Alejandro Benavides D." >> CC: m3devel at elegosoft.com >> Fecha: lunes, 18 de abril, 2011 13:47 >> "Daniel Alejandro Benavides D." >> writes: >> >Hi all: >> >the only thing you need to keep control as of CM3 is >> concerned is the modul=3D >> >a-3 free calls equals the number of news, that is, if >> not is runtime error,=3D >>=20 >> Not exactly. You can't necessarily call malloc/free >> from multiple threads. >> This is a big problem for Modula-3 programs that call out >> to nontrivial C code. >>=20 >> (Why don't we wrap malloc/free ourselves in a special >> library? Linker problems >> I suppose... and I also suppose the effort should rather be >> on getting pthreads >> working again than messing around with problems with user >> threads...) >>=20 >> Mika >> From dabenavidesd at yahoo.es Mon Apr 18 22:18:49 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 21:18:49 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418190555.526411A207A@async.async.caltech.edu> Message-ID: <967344.66270.qm@web29718.mail.ird.yahoo.com> Hi all: yes pthread allocation must keep their pthread under system constraints, but per thread user space the outer space is limited, then that's why perhaps RT is breaking due system behaviour not by its own consideration, if I may say so, so perhaps this is just the unsafe pattern break codes showed, since the implementation is responsible for avoiding this flaws but assuming they satisfies their restrictions, which is modularly sound, but if not then somebody either doesn't guarantee our assumptions, or can't meet their specifications and ours too so don't assume they satisfy theirs own ones too. Then in fact would explain why there are many crash patterns around which is very system dependent not us (that's why I favour one implementation per target source, mixing both is like meeting their specifications and so like if it would be our goal, correct if I'm wrong here please). Anything else is just falling apart of breakages that somehow rather than literally must be addressed by their own, if its so breaking the code they should, or us adapt to it literally. --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 14:05 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >ok, but are you telling this about in user threads or > working ill or all th= > >reads and pthreads? > >Thanks in advance > > > > I don't know exactly. There are certainly bad > combinations of threading > implementations and memory allocators. And I'm sure > it may be system > dependent as well. In any case, linking user threads > with the -pthreads > version of libc is probably bad. Calling malloc > willy-nilly from multiple > user threads is probably bad. Calling free, I don't > know. Pthreads currently > don't work so maybe malloc works there but that doesn't > help very much if > the system is anyhow unstable. > > Mika > > >--- El lun, 18/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] malloc/free from Modula-3? > >> Para: "Daniel Alejandro Benavides D." > >> CC: m3devel at elegosoft.com > >> Fecha: lunes, 18 de abril, 2011 13:47 > >> "Daniel Alejandro Benavides D." > >> writes: > >> >Hi all: > >> >the only thing you need to keep control as of > CM3 is > >> concerned is the modul=3D > >> >a-3 free calls equals the number of news, that > is, if > >> not is runtime error,=3D > >>=20 > >> Not exactly. You can't necessarily call > malloc/free > >> from multiple threads. > >> This is a big problem for Modula-3 programs that > call out > >> to nontrivial C code. > >>=20 > >> (Why don't we wrap malloc/free ourselves in a > special > >> library? Linker problems > >> I suppose... and I also suppose the effort should > rather be > >> on getting pthreads > >> working again than messing around with problems > with user > >> threads...) > >>=20 > >> Mika > >> > From dabenavidesd at yahoo.es Mon Apr 18 23:01:10 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 22:01:10 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <967344.66270.qm@web29718.mail.ird.yahoo.com> Message-ID: <411737.17355.qm@web29709.mail.ird.yahoo.com> Hi all: this unsound trend is what it makes software development uncontrolled, so many unstable codes, it has an effect as Dr Nelson said software quality and perhaps more dangerous ever more, at least with Modula-3 you can keep it under certain control you know are restricted but this is the result of that, their codes are broken then why one need to fix them if it's worse than before. What a mess if so. Thanks in advance --- El lun, 18/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 15:18 > Hi all: > yes pthread allocation must keep their pthread under system > constraints, but per thread user space the outer space is > limited, then that's why perhaps RT is breaking due system > behaviour not by its own consideration, if I may say so, so > perhaps this is just the unsafe pattern break codes showed, > since the implementation is responsible for avoiding this > flaws but assuming they satisfies their restrictions, which > is modularly sound, but if not then somebody either doesn't > guarantee our assumptions, or can't meet their > specifications and ours too so don't assume they satisfy > theirs own ones too. > Then in fact would explain why there are many crash > patterns around which is very system dependent not us > (that's why I favour one implementation per target source, > mixing both is like meeting their specifications and so like > if it would be our goal, correct if I'm wrong here please). > Anything else is just falling apart of breakages that > somehow rather than literally must be addressed by their > own, if its so breaking the code they should, or us adapt to > it literally. > > --- El lun, 18/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] malloc/free from Modula-3? > > Para: "Daniel Alejandro Benavides D." > > CC: m3devel at elegosoft.com > > Fecha: lunes, 18 de abril, 2011 14:05 > > "Daniel Alejandro Benavides D." > > writes: > > >Hi all: > > >ok, but are you telling this about in user threads > or > > working ill or all th= > > >reads and pthreads? > > >Thanks in advance > > > > > > > I don't know exactly. There are certainly bad > > combinations of threading > > implementations and memory allocators. And I'm > sure > > it may be system > > dependent as well. In any case, linking user > threads > > with the -pthreads > > version of libc is probably bad. Calling malloc > > willy-nilly from multiple > > user threads is probably bad. Calling free, I > don't > > know. Pthreads currently > > don't work so maybe malloc works there but that > doesn't > > help very much if > > the system is anyhow unstable. > > > > Mika > > > > >--- El lun, 18/4/11, Mika Nystrom > > escribi=F3: > > > > > >> De: Mika Nystrom > > >> Asunto: Re: [M3devel] malloc/free from > Modula-3? > > >> Para: "Daniel Alejandro Benavides D." > > >> CC: m3devel at elegosoft.com > > >> Fecha: lunes, 18 de abril, 2011 13:47 > > >> "Daniel Alejandro Benavides D." > > >> writes: > > >> >Hi all: > > >> >the only thing you need to keep control > as of > > CM3 is > > >> concerned is the modul=3D > > >> >a-3 free calls equals the number of news, > that > > is, if > > >> not is runtime error,=3D > > >>=20 > > >> Not exactly. You can't necessarily > call > > malloc/free > > >> from multiple threads. > > >> This is a big problem for Modula-3 programs > that > > call out > > >> to nontrivial C code. > > >>=20 > > >> (Why don't we wrap malloc/free ourselves in > a > > special > > >> library? Linker problems > > >> I suppose... and I also suppose the effort > should > > rather be > > >> on getting pthreads > > >> working again than messing around with > problems > > with user > > >> threads...) > > >>=20 > > >> Mika > > >> > > > From rcolebur at SCIRES.COM Mon Apr 18 23:35:40 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 18 Apr 2011 17:35:40 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: -pthreads -user threads -Win32 native threads -green threads -etc. I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Monday, April 18, 2011 2:46 PM To: Tony Hosking Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From hosking at cs.purdue.edu Tue Apr 19 00:11:26 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 18:11:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > Tony Hosking writes: > ... >> pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } From dabenavidesd at yahoo.es Tue Apr 19 00:46:52 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 23:46:52 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <937894.40594.qm@web29704.mail.ird.yahoo.com> Hi all: I think if each thread handle is RT specific form then you need to be aware of that before wanting to change system and platform, rather than of implementation, so each specific module procedure must be not UNSAFE but platform ready for its available platforms, now if something is specified implementation dependent must have a CM3 suffix for that purpose so everybody keeps and eye on its implementation aspects, so each RTThreadCM3.i3 is clearer to group as they are intended to serve as are RTCollectorSRC.i3 to RTCollector.i3 to mean is not specific to CM3 so to have implementations e.g SOLsun user threads, LINUXLIBC6 x86 user threads (implemented for Multimedia Modula-3, called MM3, etc) but anything else like for unique for Windows, SPIN, is like normal like RTException.i3 and if something is supported just for DEC Unix for instance, then it is better to say RTExceptionOSF.i3 etc in that way if so and not CM3 or anything else. Thanks in advance --- El lun, 18/4/11, Coleburn, Randy escribi?: > De: Coleburn, Randy > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "m3devel at elegosoft.com" > Fecha: lunes, 18 de abril, 2011 16:35 > I think it would be good at this > point to clarify for folks on the m3devel list what is meant > by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" > threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still > using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not > behaving well. This situation concerns me because > thread safety is something I've always relied on Modula-3 to > provide. > > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel > 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user > threads. > ... > > The following code from RTProcessC.c ensures that it is > neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in > source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to > use autoconf. > * VMS lacks pthread_atfork? Would be good to use > autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, > parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > > From jay.krell at cornell.edu Tue Apr 19 01:20:57 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:20:57 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: And "yes" you need to disable switching around free, the same as malloc. And calloc. And realloc. The disable switching *might* be at a lower level than you realize, so *might* not be missing. That is, it might be in our C wrappers. I can check later. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 12:30:45 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Yes, I think you should avoid -pthread. > > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > > > > Tony Hosking writes: > >> I suspect the problem is re-entrance to the library. Should you be = > >> disabling user thread switching somewhere? > > > > I suspect the same. That's why I asked what the "rules" are > > for calling malloc and free. I notice that the m3core code has > > Scheduler.DisableSwitching around all calls to malloc but not around calls > > to free. Why would one be unsafe and the other safe? It is always free > > that my program hangs up on by the way. Surrounding all "suspicious" calls > > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > > far as I can tell. > > > > I am further suspecting that my problem is related to my having linked the > > program with -pthread. I am also guessing that this is the same reason > > you can't link FreeBSD user threads programs with -pthread: namely that if > > you go reentrantly into the pthread malloc library some code assumes > > the two invocations are from separate threads and locks up if they are from > > the same thread. That's my guess anyhow. But that doesn't explain why > > I've never seen any malloc/free problems on FreeBSD without -pthread. > > (I don't think I have, anyhow.) > > > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:26:26 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:26:26 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ Ok, "we" should probably go ahead and fix that. I'll try to, but no promises, sorry. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 18:11:26 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? > > On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > > > Tony Hosking writes: > > ... > >> pthread_atfork should not be needed under user threads. > > ... > > > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > FreeBSD before 6. The comment is from the checked-in source file. > > > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ > > > > INTEGER > > __cdecl > > RTProcess__RegisterForkHandlers( > > ForkHandler prepare, > > ForkHandler parent, > > ForkHandler child) > > { > > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > > * VMS lacks pthread_atfork? Would be good to use autoconf. > > * Win32 lacks pthread_atfork and fork. OK. > > * > > * As well, for all Posix systems, we could implement > > * atfork ourselves, as long as we provide a fork() > > * wrapper that code uses. > > */ > > #if defined(_WIN32) \ > > || defined(__vms) \ > > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > > return 0; > > #else > > while (1) > > { > > int i = pthread_atfork(prepare, parent, child); > > if (i != EAGAIN) > > return i; > > sleep(0); > > } > > #endif > > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:28:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:28:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, Message-ID: "Green" threads are probably another name for user threads. Java used to use something with this name. > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? Certainly. Has anything ever given you a different impression? Cygwin maybe uses pthreads, but that'd still use Win32 threads under the covers. It *might* be nice to have a user thread option that used fibers or Win7's UMS (64-bit only!) but don't hold your breath. Native Win32 threads have been around forever and will be a good option forever and will be in heavy use forever. - Jay > From: rcolebur at SCIRES.COM > To: m3devel at elegosoft.com > Date: Mon, 18 Apr 2011 17:35:40 -0400 > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. > > I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 04:27:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:27:04 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: <20110419022704.6EFB71A207A@async.async.caltech.edu> Jay K writes: >--_f28739e3-2465-450f-b041-5795980d5bfe_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >And "yes" you need to disable switching around free=2C the same as malloc. = >And calloc. And realloc. >The disable switching *might* be at a lower level than you realize=2C so *m= >ight* not be missing. >That is=2C it might be in our C wrappers. I can check later. >=20 > - Jay Well in that case someone should remove the SchedulerPosix.DisableSwitching (etc) from around the mallocs. Or are you telling me that perhaps free() is wrapped but malloc() isn't? That would be... confusing. Mika From mika at async.caltech.edu Tue Apr 19 04:30:58 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:30:58 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: <20110419023058.4FBB11A207A@async.async.caltech.edu> If you or Tony could describe roughly what you think needs to be done I'd be happy to look into it. The basic problem is that a decision that's made/described in m3core/thread/m3makefile needs to someone find its way to controlling what C code gets compiled elsewhere in the system. Or maybe there should be an indirection into the thread library to pick up pthread_atfork (or not). But then again you still haven't explained why you made the user threads use pthread_atfork. I just remove it from my installations, but then again I'm not trying to run CVSup so I don't know if it breaks that program to do so. Mika Jay K writes: >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> The following code from RTProcessC.c ensures that it is neeeded on every = >Unix except >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ > >=20 >Ok=2C "we" should probably go ahead and fix that. >I'll try to=2C but no promises=2C sorry. >=20 > - Jay >=20 >> From: hosking at cs.purdue.edu >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >> Probably unnecessary=2C given that I think there is another entry point t= >o forking a process (I forget where) in the thread-specific portion of m3co= >re. In which case the necessary work might be done there? >>=20 >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: >>=20 >> > Tony Hosking writes: >> > ... >> >> pthread_atfork should not be needed under user threads. >> > ... >> >=20 >> > The following code from RTProcessC.c ensures that it is neeeded on ever= >y Unix except >> > FreeBSD before 6. The comment is from the checked-in source file. >> >=20 >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ >> >=20 >> > INTEGER >> > __cdecl >> > RTProcess__RegisterForkHandlers( >> > ForkHandler prepare=2C >> > ForkHandler parent=2C >> > ForkHandler child) >> > { >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> > * Win32 lacks pthread_atfork and fork. OK. >> > * >> > * As well=2C for all Posix systems=2C we could implement >> > * atfork ourselves=2C as long as we provide a fork() >> > * wrapper that code uses. >> > */ >> > #if defined(_WIN32) \ >> > || defined(__vms) \ >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> > return 0=3B >> > #else >> > while (1) >> > { >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B >> > if (i !=3D EAGAIN) >> > return i=3B >> > sleep(0)=3B >> > } >> > #endif >> > } >>=20 > = > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > fixed if need be.
>=3B >=3B */

> =3B
>Ok=2C "we" should probably go ahead and fix that.
>I'll try to=2C but no =3Bpromises=2C sorry.
> =3B
> =3B- Jay
 =3B
>>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = >that I think there is another entry point to forking a process (I forget wh= >ere) in the thread-specific portion of m3core. In which case the necessary = >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child)>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = >as long as we provide a fork()
>=3B >=3B * wrapper that code uses.R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > >= > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- From jay.krell at cornell.edu Tue Apr 19 08:07:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:07:01 +0000 Subject: [M3devel] pthread_atfork Message-ID: I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: no-op for Win32: just return 0 It is tempting to have this assert(0) but that is wrong. The function will be called and should just do nothing. as-is for pthread: call pthread_atfork manually for userthreads: FreeBSD < 6: does it have pthreads? Can we declare such systems use user threads? take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 VMS: Punt for now. Maybe newer versions than I was using have it. And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible RTProcessPthread__Fork: return fork(); RTProcessUserThreads__Fork: wrapper around fork() that handles the callbacks from RTProcess__RegisterForkHandlers. RTProcessVms__Fork: assert(false) I think, unless newer VMS has fork(). One thing not stated here, is that I believe something "like" pthread_atfork is somewhere between reasonable and actually needed, on all Posix/Unix systems. Because, what isn't well known here, but I think is true, is that we depend on pthread_atfork (or a reimplementation thereof) as described here, so that fork + do more work with user threads behaves like pthreads, and doesn't have all the threads survive. Or..and I assume the answer is no: Do we want fork + do more work to keep all threads for user threads and keep only one thread for pthreads? How it was for a long time, but we didn't discover until fairly recently. I'm 99% sure this is all exactly correct. Possibly this is enough explanation for others to do it. Otherwise, maybe I'll get to it "soon". Maybe. This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:15:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:15:49 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110419022704.6EFB71A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , , , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, , <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu>, , <20110419022704.6EFB71A207A@async.async.caltech.edu> Message-ID: Here is where malloc/calloc/free are handled, in a new, somewhat redundant way, that reduces/eliminates the burden at higher levels: jbook2:src jay$ pwd /dev2/cm3/m3-libs/m3core/src jbook2:src jay$ edit C/Common/CstdlibC.c M3WRAP_NO_SWITCHING(void*, Cstdlib__malloc, malloc, (WORD_T a), (a)) M3WRAP_NO_SWITCHING(void*, Cstdlib__calloc, calloc, (WORD_T a, WORD_T b), (a, b)) M3WRAP_RETURN_VOID_NO_SWITCHING(Cstdlib__free, free, (void* a), (a)) Now, if you have other C code that is calling malloc/free, you still need to disable switching around that. Perhaps all calls to <*extern*> functions should disable/enableswitching. Perhaps. The real answer is though, make sure pthreads and Win32 threads work, and then don't do much to make user threads faster or easier to maintain. see m3core/src/m3core.h for definitions of those macros. They should be pretty clear. If not, run the C preprocessor over those files and look at the results. -- The traditional correct thing is to wrap Cstdlib.malloc/Cstdlib.free whereever they are called. But I kept finding this missing, got fed up with it, fixed it at a lower level. Is that bad? Maybe. Maybe not. -- I wouldn't argue against wrapping all <*extern*> calls with disable/enableswitching. Given that you have to recompile everything to switch between user threads and kernel threads anyway, the compiler could insert these calls only for user threads, perhaps. - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:27:04 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Jay K writes: > >--_f28739e3-2465-450f-b041-5795980d5bfe_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >And "yes" you need to disable switching around free=2C the same as malloc. = > >And calloc. And realloc. > >The disable switching *might* be at a lower level than you realize=2C so *m= > >ight* not be missing. > >That is=2C it might be in our C wrappers. I can check later. > >=20 > > - Jay > > Well in that case someone should remove the > SchedulerPosix.DisableSwitching (etc) from around the mallocs. > > Or are you telling me that perhaps free() is wrapped but malloc() isn't? > That would be... confusing. > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:23:55 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <937894.40594.qm@web29704.mail.ird.yahoo.com> References: , <937894.40594.qm@web29704.mail.ird.yahoo.com> Message-ID: > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:33:31 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:33:31 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: , , <937894.40594.qm@web29704.mail.ird.yahoo.com>, Message-ID: Caveat: OpenBSD now only offers pthreads. Modula-3 user threads require either get/set/make/swapcontext, or sigaltstack. OpenBSD offers neither. There were implementations using setjmp, but they were most unportable, only remained for OpenBSD, and I removed them. - Jay From: jay.krell at cornell.edu To: dabenavidesd at yahoo.es; m3devel at elegosoft.com; rcolebur at scires.com Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 13:51:06 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 04:51:06 -0700 Subject: [M3devel] LONGINT -> more bugs in m3tk Message-ID: <20110419115106.7E6A31A207A@async.async.caltech.edu> Hello m3devel, If anyone knows how to fix the following trivially, please go ahead. "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal (errors reported by m3tk). The offending line is: INTERFACE BasicCtypes; IMPORT Word, Long; TYPE (* the four signed integer types *) signed_char = [-16_7f-1 .. 16_7f]; short_int = [-16_7fff-1 .. 16_7fff]; int = [-16_7fffffff-1 .. 16_7fffffff]; long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) (* the four unsigned integer types *) unsigned_char = [16_0 .. 16_ff]; unsigned_short_int = [16_0 .. 16_ffff]; unsigned_int = [16_0 .. 16_ffffffff]; unsigned_long_int = Word.T; unsigned_long_long = Long.T; (* the three floating types *) float = REAL; double = LONGREAL; long_double = EXTENDED; (* char *) char = signed_char; END BasicCtypes. I'll look at it myself, but not immediately (no time). Workaround for now. Mika From jay.krell at cornell.edu Tue Apr 19 17:06:38 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 15:06:38 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419023058.4FBB11A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Apr 19 17:13:55 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 16:13:55 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <768127.64883.qm@web29706.mail.ird.yahoo.com> Hi all: but I think there should be a way with the first implementation of MM3 if so, what is the risk of not having it? Can you see it, please: https://mail.elegosoft.com/pipermail/m3devel/2007-July/000272.html Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 10:06 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: ? user thread/pthread inconsistency ? cvsupd incompatibility with pthreads ? user threads using pthread_atfork/-pthread ? fork() calls having to go through a wrapper (ok -- you could miss this and ?? not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) ?- Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 17:29:24 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 08:29:24 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: <20110419152924.22C411A207A@async.async.caltech.edu> Never explained this clearly before. Thanks, Jay. I suppose the reason CVSup forks and does work is to save copying overhead because the two forked processes can share read-only pages without any copying? I remember that the last time I looked at this my reaction was "fork() is not part of Modula-3 : people who use it are on their own". In a sense this is still true as FreeBSD simply will not work with -pthread. I'm afraid that, however, that removing -pthread didn't fix my Linux lockup problems. Turns out the library I was having trouble with is a PostgreSQL library written by Critical Mass. To fix it, I wrote a program using m3tk and some Scheme code that will take any M3 interface and wrap every routine in that interface in the following code structure: PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = BEGIN SchedulerIndirection.DisableSwitching(); TRY RETURN PQ.PQconnectPoll(conn) FINALLY SchedulerIndirection.EnableSwitching() END END PQconnectPoll; which then gave rise to my previous bug report about BasicCTypes, which m3tk cannot process a.t.m., owing to LONGINT breakage. Also I have to call "SchedulerIndirection" since the routines to turn off switching are named differently on CM3 (Scheduler.DisableSwitching) and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous to me but oh well I must have a dozen of these CM3/PM3 differences to deal with, what's another one? Mika Jay K writes: >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- From dabenavidesd at yahoo.es Tue Apr 19 18:08:43 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 17:08:43 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: <86414.26723.qm@web29716.mail.ird.yahoo.com> Hi all: I think the worst case scenario is to adapt changes to modification of such bugs, or UNSAFE parts, if it just gets to pick of a portion of the system well stabilized then all we can that depended on that, I hope this stabilizes soon, then we can all share the rest of the things. Thanks in advance --- El mar, 19/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: martes, 19 de abril, 2011 10:29 > Never explained this clearly > before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save > copying > overhead because the two forked processes can share > read-only pages > without any copying? > > I remember that the last time I looked at this my reaction > was "fork() > is not part of Modula-3 : people who use it are on their > own". In a sense > this is still true as FreeBSD simply will not work with > -pthread. > > I'm afraid that, however, that removing -pthread didn't fix > my Linux > lockup problems. Turns out the library I was having > trouble with is > a PostgreSQL library written by Critical Mass. To fix > it, I wrote a > program using m3tk and some Scheme code that will take any > M3 interface > and wrap every routine in that interface in the following > code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : > PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about > BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the > routines to turn > off switching are named differently on CM3 > (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just > seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 > differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() > survived fork() > >Now: user threads and pthreads: only the thread calling > fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user > threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads > and user threads. > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost > immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to > fork(). > >Each module with a need to do something before/after > fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is > needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must > implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own > fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this > approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now > become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok > -- you could miss this an= > >d=20 > > not likely notice -- only fork() > calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but > people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same > thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you > think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's > made/described in > >> m3core/thread/m3makefile needs to someone find its > way to controlling > >> what C code gets compiled elsewhere in the > system. Or maybe there should > >> be an indirection into the thread library to pick > up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you > made the user threads = > >use > >> pthread_atfork. I just remove it from my > installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it > breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c > ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix > that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I > think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the > thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be > done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C > Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be > needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c > ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is > from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks > pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be > good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and > fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix > systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as > we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > > > > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of > pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B > >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix > that.
> >> >I'll try to=3D2C but > no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- > Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B > Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B > CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C > glibc-2.6-4
>=3D3B
>=3D3B > Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to > forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. > In which case the necessa= > >ry =3D > >> >work might be done > there?
>=3D3B
>=3D3B On Apr > 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom > wrote:
>=3D3B
>=3D3B > >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B > ...
>=3D3B >=3D3B>=3D3B > pthread_atfork sh= > >ould not be n=3D > >> >eeded under user > threads.
>=3D3B >=3D3B > ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B > >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B > >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if > need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B > >=3D3B INTEGER
>=3D3B >=3D3B > __cdecl
&= > >gt=3D3B >=3D3B =3D > >> > >RTProcess__RegisterForkHandlers(
>=3D3B > >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler > parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B > {
>=3D3B >=3D3B /* FreeBSD <=3D3B > 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use > autoconf.
>=3D3B >=3D3B * VMS lacks > pthread= > >_atfork=3D > >> >? Would be good to use > autoconf.
>=3D3B >=3D3B * Win32 lacks > pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B > >=3D3B *
>=3D3B >=3D3B * As > well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could > implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a > fork()
>=3D3B >=3D3B * wrapper that > code = > >uses. >> >R>>=3D3B >=3D3B > */
>=3D3B >=3D3B #if defined(_WIN32) > \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B > >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ > )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B > while (1)
>=3D3B >=3D3B > {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C > parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B > >=3D3B return i=3D3B
>=3D3B > >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B > }
>=3D3B >=3D3B > #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > > > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > > > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived > fork()
Previously: pthrea= > >ds: only the thread calling fork() survived > fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives > fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and > pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads > and user threads.
= > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or > exec almost immediately a= > >fter fork.


pthread_atfork > offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and > change all the calls to fork()= > >.
Each module with a need to do something > before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and > pthread_atfork cooperate to do w= > >hat is needed.


No function > pointer is needed.
Instead move th= > >e code to > m3core/thread.


If you really > must not use -pthread=2C = > >then you must implement pthread_atfork > functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that > cooperates
with = > >your pthread_atfork replacement.
There is no > free lunch -- there is a do= > >wnside to this approach=2C as plain fork() > calls
are ok and correct if p= > >thread_atfork is used=2C but now become > incorrect.
Pick your poison:
= > > =3B user thread/pthread > inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads > using pthread_atfork/-pthread >r> =3B fork() calls having to go through a > wrapper (ok -- you could mis= > >s this and
 =3B =3B not > likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper). >


(Most of this has = > >been explained multiple times=2C but people only pay > attention
when they= > > think it affects them. I'm guilty of the same > thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B > Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B > CC= > >: m3devel at elegosoft.com
>=3B > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C > glibc-2.6-4
>=3B
>=3B >
>=3B If = > >you or Tony could describe roughly what you think needs > to be done
>= > >=3B I'd be happy to look into it.
>=3B >
>=3B The basic problem is= > > that a decision that's made/described > in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to > controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there > should
>=3B be an ind= > >irection into the thread library to pick up > pthread_atfork (or not).
>= > >=3B
>=3B But then again you still > haven't explained why you made the = > >user threads use
>=3B > pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not > trying to run CVSup so I don't = > >know if it breaks that program to do > so.
>=3B
>=3B > Mika
= > >>=3B
>=3B Jay K > writes:
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B > >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B= > > >=3B
>=3B > >=3B
>=3B >=3B>=3B The > following code from RTP= > >rocessC.c ensures that it is neeeded on every > =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B > /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability > of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need > be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B > >=3B=3D20
>=3B >=3BOk=3D2C "we" > shoul= > >d probably go ahead and fix that.
>=3B > >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B > >=3B=3D20
>=3B >=3B - > Jay
>=3B = > >>=3B=3D20
>=3B > >=3B>=3B From: hosking at cs.purdue.edu
>=3B > &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B > >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B > >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B > Probably unnecessary=3D2C given that I thi= > >nk there is another entry point > t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of > m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done > there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On > Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B > >=3B>=3B=3D20
>=3B > >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B > >=3B>=3B >=3B ...
>=3B > >=3B>= > >=3B >=3B>=3B pthread_atfork should not be > needed under user threads. >>>=3B >=3B>=3B >=3B > ...
>=3B >=3B>=3B > >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code > from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By > Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from > the checked-in source file= > >.
>=3B >=3B>=3B > >=3B=3D20
>=3B >=3B>=3B > >=3B /* NOTE:= > > Even userthreads now depends
>=3B > >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B > >=3B>=3B >=3B * This can be fixed if need > b= > >e.
>=3B >=3B>=3B >=3B > */
>=3B >=3B>=3B > >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B > INTEGER
>=3B >=3B>=3B >=3B > __cdecl
= > >>=3B >=3B>=3B >=3B > RTProcess__RegisterForkHandlers(
>=3B > >= > >=3B>=3B >=3B ForkHandler > prepare=3D2C
>=3B >=3B>=3B > >=3B For= > >kHandler parent=3D2C
>=3B > >=3B>=3B >=3B ForkHandler > child)
&g= > >t=3B >=3B>=3B >=3B > {
>=3B >=3B>=3B >=3B /* > FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use > autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be > good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks > pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B > *
>=3B >=3B>=3B >=3B * As > well=3D2C for = > >all Posix systems=3D2C we could > implement
>=3B >=3B>=3B > >=3B * a= > >tfork ourselves=3D2C as long as we provide a > fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code > uses.
>=3B >=3B>=3B >=3B > */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) > \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B > >=3B>=3B >=3B || (defined(__FreeBSD__) > /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ > )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B > >=3B #else
>=3B >=3B>=3B > >= > >=3B while (1)
>=3B >=3B>=3B > >=3B {
>=3B >=3B>=3B > >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C > child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D > EAGAIN)
>=3B >=3B>=3B >=3B > retu= > >rn i=3D3B
>=3B >=3B>=3B > >=3B sleep(0)=3D3B
>=3B > >=3B>= > >=3B >=3B }
>=3B > >=3B>=3B >=3B #endif
>=3B > >=3B>=3B &g= > >t=3B }
>=3B > >=3B>=3B=3D20
>=3B >=3B > > > > =3D
>= > >=3B >=3B
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B > charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B > >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B > >=3B<=3Bhead>=3B
>=3B > >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B > >=3B.hmmessage P
>=3B > >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B > >=3Bpadding:0px
>=3B >=3B} >r>>=3B > >=3Bbody.hmmessage
>=3B > >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B > >=3Bfont-family:Tahoma
>=3B > >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B > >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody > class=3D3D'hmmessage'>=3B
>=3B > >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it > is neeeded on e=3D >>>=3B >=3Bvery Unix > except<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /= > >* NOTE: Even userthreads now > depends<=3BBR>=3B=3D
>=3B > >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of > pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can > be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > */<=3BBR>=3B<=3BBR>= > >=3B
>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3BOk=3D2C = > >"we" should probably go ahead and fix > that.<=3BBR>=3B
>=3B > >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C > sorry.<=3BBR>=3B >>>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3Bnbsp=3D= > >3B- > Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B > Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B > >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B > CC: m3devel at el= > >egos=3D
>=3B > >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B > Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C > kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B > >=3Bthat I think there is anot= > >her entry point to forking a process (I forget > wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case > the necessary =3D >>>=3B >=3Bwork might be done > there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C > at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom > wrote:<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking > writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B > ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be > n=3D
>=3B >=3Beede= > >d under user > threads.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B > =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code > from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B > >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The > comment is from the=3D
&= > >gt=3B >=3B checked-in source > file.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now > depends<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B= > > * on availability of > pthreads.<=3BBR>=3B&=3B=3D
>=3B > >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need > be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B > __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B > >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler > prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 > lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use > autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks > pthread_atfork=3D
>=3B >=3B? Would be > good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. > OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix > systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork > ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a > fork()<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3= > >B * wrapper that code > uses.<=3BB=3D
>=3B > >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) > \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > || (defined(__F= > >reeBSD__) /* > &=3Bamp=3D3B&=3Bam=3D
>=3B > >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ > )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D
>=3B >=3B=3D3B > #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > in=3D
= > >>=3B >=3Bt i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B > sleep(0)=3D3B<=3BBR=3D
>=3B > >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B > }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B > =3D
>=3B >=3B > > > <=3B/body>=3B<= > >br>>=3B > >=3B<=3B/html>=3B=3D
>=3B > >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> > > > > >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- > From jay.krell at cornell.edu Tue Apr 19 20:50:16 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 18:50:16 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: It's easy enough for us to support either name. - Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 22:32:54 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 20:32:54 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: > This change depends on pthread_atfork, for pthreads and user threads. Clarification: There's no change described there for pthreads, and indeed, getting this behavior with pthreads doesn't depend on using pthread_atfork. That is just how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs. fork()). The Solaris default used to be fork => forkall(). The current Solaris default is fork => fork1(). There really do exist on Solaris to functions you can call, fork1() and forkall(). All other systems and the Posix specification have the "fork1" semantic for fork. Therefore "fork1" is what we should provide unconditionally. If you really want "forkall", then you write: #ifdef __sun forkall() #else #error #endif and/or you go and hack up m3core/src/thread/POSIX, branch off m3core/src/thread/POSIX_FORKALL or such. I don't expect anyone to ever do that. The need for pthread_atfork is sort of different for user threads vs. pthreads, but it is needed either way. For pthreads, there is this problem -- at the time a thread calls fork(), what are other threads doing? Answer: Anything. Follow up question: What locks might they hold? Answer: Any. Follow up: So, if arbitrary locks are held, and you fork, and don't exec, and the child process attempts to use some of the locks that were held in the parent, and only the forking thread survived, then the other threads will never leave their locks, and later attempts to acquire them in the children will deadlock. Therefore, what pthread_atfork provides for, what you are supposed to use it for, is roughly speaking, right before fork(), in the parent, acquire all locks, and then fork(), and then in the parent and children, release all locks. Therefore, you have to come up with a locking order. Things are a little different for user threads. In user threads, the existance of threads is established by virtue of a global array or list of data describing the threads. And a timer to occasionally preempt. When the timer interrupts, the scheduler picks another thread from the global array/list. Previously, the data went unchanged when fork() was called. Therefore all threads survived. What we do now is that in the child "handler" for pthread_atfork, is reinitialize the various data, so as to forget about all but the current thread. Something like that. I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork, then ForkBeforeExec might get away with just fork() and ignore all this. So far we have not drawn this line -- there is just the one fork and it is used prior to exec or do-more-work and does the same thing w/o knowing what is to follow. The Posix documentation I think describes this all fairly well. Bing for "posix opengroup pthread_atfork". Really. I think it is saying much of the same thing I am saying, but in language that has received much more thought. Be sure to read the "rationale" part. - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Date: Tue, 19 Apr 2011 15:06:38 +0000 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 20 00:10:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 23:10:45 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <718150.90366.qm@web29720.mail.ird.yahoo.com> Hi all: since there was just that interface in *nix platforms, then it must be called like that , but another name will be just create SchedulerCM3, thus suggests SchedulerPosix would be called rather than nothing suffixing it SchedulerSRC and the other one Scheduler alone be the default like to say SchedulerCM3 AS Scheduler? so to switch to SchedulerSRC AS Scheduler provided that both are implementation ready so one can select SchedulerCM3 as SchedulerSRC or Scheduler so one name comes after each rep rather than changing every other place each time, even of Internal Nodes as in m3tk which specifies its members as? SRC_NODE <: AST_NODE then something like CM3_NODE likiwise subtyped if so should be created to hold the extensions (can somebody name them WIDECHAR, LONGINT, LONGCARD, what else am I missing if so?),? like we could come up with something cleaner than just put all there. I will try to se a proposal for that. Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 13:50 It's easy enough for us to support either name. ?- Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Apr 20 04:18:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 20 Apr 2011 02:18:25 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: Can you by chance see the other threads? I suspect not -- user threads I presume. Can you try sprinking a lot more Disable/EnableSwitching around? I guess you tried that..and..? - Jay > From: hosking at cs.purdue.edu > Date: Fri, 15 Apr 2011 11:26:15 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > > > Hi m3devel, > > > > I'm debugging a problem with some code I have that calls out to C > > code. It's calling pqlib (PostgreSQL library) and you might expect > > things to go wrong there, but as it turns out it's dying in CM3's code. > > > > The system is AMD64_LINUX (user threading because of the problems with > > pthreads) and the program is just freezing up hard. No segfault, > > bus error or anything. It locks up a couple of times a day. > > > > What are the rules on calling malloc/free from Modula-3? > > > > Stack trace: > > > > (gdb) where > > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > > ) at ../src/C/Common/CstdlibC.c:26 > > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > > ) at ../src/C/Common/M3toC.m3:46 > > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > > ) at ../src/UnsafeDatabase.m3:230 > > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > > ) at ../src/UnsafeDatabase.m3:228 > > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > > ) at ../src/UnsafeDatabase.m3:258 > > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > > ) at ../src/DesynchronizedDB.m3:126 > > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > > ) at ../src/DesynchronizedDB.m3:293 > > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > > ) at ../src/GCOMSOrderManager.m3:596 > > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > > ) at ../src/GCOMSOrderManager.m3:310 > > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > > ) at ../src/EuroDriver.m3:1108 > > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > > ) at ../src/EuroDriver.m3:1545 > > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > > #17 0x0000000000000000 in ?? () > > (gdb) > > > > Here is the code that causes the problem: > > > > VAR > > s := CopyTtoS(query); > > BEGIN > > TRY > > t.res := t.myExec(s,busyWait) > > FINALLY > > FreeCopiedS(s) (* stuck here *) > > END > > END; > > > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > > routine (PQexec I think it's called). > > > > Since this is already an unsafe module I suppose I could just make an > > array of CHARs on the stack and pass a pointer to that... > > > > > > Regards, > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Apr 20 07:55:57 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 22:55:57 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: <20110420055557.089101A207A@async.async.caltech.edu> The rationale you directed us to says: "It is suggested that programs that use fork() call an exec function very soon afterwards in the child process, thus resetting all states. In the meantime, only a short list of async-signal-safe library routines are promised to be available." and later "Application programs may not be aware that a multi-threaded library is in use, and they feel free to call any number of library routines between the fork() and exec calls, just as they always have. Indeed, they may be extant single-threaded programs and cannot, therefore, be expected to obey new restrictions imposed by the threads library." In C maybe that is true. In Modula-3, every application has *always* had to assume that there are other threads floating around. There is no such thing as a "single-threaded" Modula-3 program---or at least, the distinction doesn't matter. I really think fork-and-do-more-work is just a bug. Maybe it is a special pattern used by some special program---maybe. In any case support for it I do not think belongs in m3core. Mika Jay K writes: >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > This change depends on pthread_atfork=2C for pthreads and user threads. > > >Clarification: There's no change described there for pthreads=2C and indeed= >=2C getting >this behavior with pthreads doesn't depend on using pthread_atfork. That is= > just >how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > fork()). >The Solaris default used to be fork =3D> forkall(). >The current Solaris default is fork =3D> fork1(). >There really do exist on Solaris to functions you can call=2C fork1() and f= >orkall(). >All other systems and the Posix specification have the "fork1" semantic for= > fork. >Therefore "fork1" is what we should provide unconditionally. >If you really want "forkall"=2C then you write: >#ifdef __sun > forkall() >#else > #error >#endif > > > >and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >thread/POSIX_FORKALL or such. >I don't expect anyone to ever do that. > > > >The need for pthread_atfork is sort of different for user threads vs. pthre= >ads=2C but it is needed either way. >For pthreads=2C there is this problem -- at the time a thread calls fork()= >=2C what are other threads doing? >Answer: Anything. >Follow up question: What locks might they hold? >Answer: Any. >Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >xec=2C and the child process >attempts to use some of the locks that were held in the parent=2C and only = >the forking thread survived=2C >then the other threads will never leave their locks=2C and later attempts t= >o acquire them in the children >will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >e supposed to use it for=2C >is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >locks=2C and then fork()=2C and then in the >parent and children=2C release all locks. Therefore=2C you have to come up = >with a locking order. > > > >Things are a little different for user threads. >In user threads=2C the existance of threads is established by virtue of a g= >lobal array or list of data describing the threads. >And a timer to occasionally preempt. When the timer interrupts=2C the sched= >uler picks another thread from the global >array/list. Previously=2C the data went unchanged when fork() was called. T= >herefore all threads survived. >What we do now is that in the child "handler" for pthread_atfork=2C is rein= >itialize the various data=2C so as to forget about >all but the current thread. Something like that. > > > >I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >rk=2C then ForkBeforeExec >might get away with just fork() and ignore all this. >So far we have not drawn this line -- there is just the one fork and it is = >used prior to exec or do-more-work >and does the same thing w/o knowing what is to follow. > > >The Posix documentation I think describes this all fairly well. >Bing for "posix opengroup pthread_atfork". >Really. I think it is saying much of the same thing I am saying=2C but in l= >anguage that has received >much more thought. Be sure to read the "rationale" part. > > > - Jay > > >From: jay.krell at cornell.edu >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >2.6-4 >Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > > > > > > > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > > =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >er threads.


Clarification: There's no change described there for= > pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >epend on using pthread_atfork. That is just
how pthreads are -- except o= >ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >ault used to be fork =3D>=3B forkall().
The current Solaris default is= > fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >t we should provide unconditionally.
If you really want "forkall"=2C the= >n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >ror
#endif



and/or you go and hack up m3core/src/thread/PO= >SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >t anyone to ever do that.



The need for pthread_atfork is sor= >t of different for user threads vs. pthreads=2C but it is needed either way= >.
For pthreads=2C there is this problem -- at the time a thread calls fo= >rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >process
attempts to use some of the locks that were held in the parent= >=2C and only the forking thread survived=2C
then the other threads will = >never leave their locks=2C and later attempts to acquire them in the childr= >en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >you are supposed to use it for=2C
is roughly speaking=2C right before fo= >rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >n the
parent and children=2C release all locks. Therefore=2C you have to= > come up with a locking order.



Things are a little different= > for user threads.
In user threads=2C the existance of threads is establ= >ished by virtue of a global array or list of data describing the threads.r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >eduler picks another thread from the global
array/list. Previously=2C th= >e data went unchanged when fork() was called. Therefore all threads survive= >d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >s reinitialize the various data=2C so as to forget about
all but the cur= >rent thread. Something like that.



I suspect if we had RTProc= >ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >br>might get away with just fork() and ignore all this.
So far we have n= >ot drawn this line -- there is just the one fork and it is used prior to ex= >ec or do-more-work
and does the same thing w/o knowing what is to follow= >.


The Posix documentation I think describes this all fairly well= >.
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >ying much of the same thing I am saying=2C but in language that has receive= >d
much more thought. Be sure to read the "rationale" part.


&n= >bsp=3B- Jay


From: jay.krell at cornell.edu<= >br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >[M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >: Tue=2C 19 Apr 2011 15:06:38 +0000

> >"> > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From mika at async.caltech.edu Wed Apr 20 08:12:17 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 23:12:17 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: <20110420061217.C539A1A207A@async.async.caltech.edu> Jay K writes: >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume. >Can you try sprinking a lot more Disable/EnableSwitching around? >I guess you tried that..and..? So far so good. I'll let you know tomorrow. (The application runs 8 1/2 hours a day and has been crashing about twice a day.) The annoying thing is that when it does lock up, it locks up in one of the wrapped free()s, not in the unwrapped one. Mika > > - Jay > > >> From: hosking at cs.purdue.edu >> Date: Fri=2C 15 Apr 2011 11:26:15 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] malloc/free from Modula-3? >>=20 >> I suspect the problem is re-entrance to the library. Should you be disabl= >ing user thread switching somewhere? >>=20 >> Antony Hosking | Associate Professor | Computer Science | Purdue Universi= >ty >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Office +1 765 494 6001 | Mobile +1 765 427 5484 >>=20 >>=20 >>=20 >>=20 >> On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote: >>=20 >> > Hi m3devel=2C >> >=20 >> > I'm debugging a problem with some code I have that calls out to C >> > code. It's calling pqlib (PostgreSQL library) and you might expect >> > things to go wrong there=2C but as it turns out it's dying in CM3's cod= >e. >> >=20 >> > The system is AMD64_LINUX (user threading because of the problems with >> > pthreads) and the program is just freezing up hard. No segfault=2C >> > bus error or anything. It locks up a couple of times a day. >> >=20 >> > What are the rules on calling malloc/free from Modula-3? >> >=20 >> > Stack trace: >> >=20 >> > (gdb) where >> > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 >> > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 >> > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 >> > #3 0x000000000091462f in Cstdlib__free (a=3DCannot access memory at ad= >dress 0x2ac73b0ef0d8 >> > ) at ../src/C/Common/CstdlibC.c:26 >> > #4 0x000000000091477d in M3toC__FreeCopiedS (s=3DCannot access memory = >at address 0x2ac73b0ef0f8 >> > ) at ../src/C/Common/M3toC.m3:46 >> > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot= > access memory at address 0x2ac73b0ef118 >> > ) at ../src/UnsafeDatabase.m3:230 >> > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access memo= >ry at address 0x2ac73b0ef148 >> > ) at ../src/UnsafeDatabase.m3:228 >> > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef268 >> > ) at ../src/UnsafeDatabase.m3:258 >> > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=3DCannot access me= >mory at address 0x2ac73b0ef328 >> > ) at ../src/DesynchronizedDB.m3:126 >> > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=3DCannot access m= >emory at address 0x2ac73b0ef3e8 >> > ) at ../src/DesynchronizedDB.m3:293 >> > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=3DCannot a= >ccess memory at address 0x2ac73b0ef5a8 >> > ) at ../src/GCOMSOrderManager.m3:596 >> > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=3DCann= >ot access memory at address 0x2ac73b0ef668 >> > ) at ../src/GCOMSOrderManager.m3:310 >> > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price=3D= >{}=2C count=3DCannot access memory at address 0x2ac73b0ef7b= >8 >> > ) at ../src/EuroDriver.m3:1108 >> > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../sr= >c/EuroDriver.m3:1401 >> > #14 0x00000000004197a1 in EuroDriver__OApply (cl=3DCannot access memory= > at address 0x2ac73b0efd38 >> > ) at ../src/EuroDriver.m3:1545 >> > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/PO= >SIX/ThreadPosix.m3:1174 >> > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 >> > #17 0x0000000000000000 in ?? () >> > (gdb)=20 >> >=20 >> > Here is the code that causes the problem: >> >=20 >> > VAR >> > s :=3D CopyTtoS(query)=3B >> > BEGIN >> > TRY >> > t.res :=3D t.myExec(s=2CbusyWait) >> > FINALLY >> > FreeCopiedS(s) (* stuck here *) >> > END >> > END=3B >> >=20 >> > The query is a SQL query---t.myExec eventually calls the libpq SQL exec >> > routine (PQexec I think it's called). >> >=20 >> > Since this is already an unsafe module I suppose I could just make an >> > array of CHARs on the stack and pass a pointer to that... >> >=20 >> >=20 >> > Regards=2C >> > Mika >>=20 > = > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume.
Can you try sprinking a lot more Disable/EnableSwitching around?<= >br>I guess you tried that..and..?

 =3B- Jay


>=3B Fr= >om: hosking at cs.purdue.edu
>=3B Date: Fri=2C 15 Apr 2011 11:26:15 -0400= >
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegosoft.comr>>=3B Subject: Re: [M3devel] malloc/free from Modula-3?
>=3B
&g= >t=3B I suspect the problem is re-entrance to the library. Should you be dis= >abling user thread switching somewhere?
>=3B
>=3B Antony Hosking= > | Associate Professor | Computer Science | Purdue University
>=3B 305= > N. University Street | West Lafayette | IN 47907 | USA
>=3B Office +1= > 765 494 6001 | Mobile +1 765 427 5484
>=3B
>=3B
>=3B
= >>=3B
>=3B On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote:>>=3B
>=3B >=3B Hi m3devel=2C
>=3B >=3B
>=3B >=3B = >I'm debugging a problem with some code I have that calls out to C
>=3B= > >=3B code. It's calling pqlib (PostgreSQL library) and you might expect= >
>=3B >=3B things to go wrong there=2C but as it turns out it's dyin= >g in CM3's code.
>=3B >=3B
>=3B >=3B The system is AMD64_LIN= >UX (user threading because of the problems with
>=3B >=3B pthreads) = >and the program is just freezing up hard. No segfault=2C
>=3B >=3B = >bus error or anything. It locks up a couple of times a day.
>=3B >= >=3B
>=3B >=3B What are the rules on calling malloc/free from Modula= >-3?
>=3B >=3B
>=3B >=3B Stack trace:
>=3B >=3B
&g= >t=3B >=3B (gdb) where
>=3B >=3B #0 0x00002ac519a9202e in ?? () fr= >om /lib/libc.so.6
>=3B >=3B #1 0x00002ac519a2fb43 in ?? () from /li= >b/libc.so.6
>=3B >=3B #2 0x00002ac519a2caab in free () from /lib/li= >bc.so.6
>=3B >=3B #3 0x000000000091462f in Cstdlib__free (a=3DCanno= >t access memory at address 0x2ac73b0ef0d8
>=3B >=3B ) at ../src/C/Co= >mmon/CstdlibC.c:26
>=3B >=3B #4 0x000000000091477d in M3toC__FreeCo= >piedS (s=3DCannot access memory at address 0x2ac73b0ef0f8
>=3B >=3B = >) at ../src/C/Common/M3toC.m3:46
>=3B >=3B #5 0x00000000007884f0 in= > UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot access memory at address 0x2= >ac73b0ef118
>=3B >=3B ) at ../src/UnsafeDatabase.m3:230
>=3B &g= >t=3B #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef148
>=3B >=3B ) at ../src/UnsafeDatabase.m= >3:228
>=3B >=3B #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t= >=3DCannot access memory at address 0x2ac73b0ef268
>=3B >=3B ) at ../= >src/UnsafeDatabase.m3:258
>=3B >=3B #8 0x000000000077ad5b in Desync= >hronizedDB__SExec (t=3DCannot access memory at address 0x2ac73b0ef328
&g= >t=3B >=3B ) at ../src/DesynchronizedDB.m3:126
>=3B >=3B #9 0x0000= >00000077c190 in DesynchronizedDB__SMExec (t=3DCannot access memory at addre= >ss 0x2ac73b0ef3e8
>=3B >=3B ) at ../src/DesynchronizedDB.m3:293
&= >gt=3B >=3B #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t= >=3DCannot access memory at address 0x2ac73b0ef5a8
>=3B >=3B ) at ../= >src/GCOMSOrderManager.m3:596
>=3B >=3B #11 0x0000000000566b95 in GCO= >MSOrderManager__PlaceSingleOrder (t=3DCannot access memory at address 0x2ac= >73b0ef668
>=3B >=3B ) at ../src/GCOMSOrderManager.m3:310
>=3B &= >gt=3B #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price= >=3D{<=3BNo data fields>=3B}=2C count=3DCannot access memory at address = >0x2ac73b0ef7b8
>=3B >=3B ) at ../src/EuroDriver.m3:1108
>=3B &g= >t=3B #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../s= >rc/EuroDriver.m3:1401
>=3B >=3B #14 0x00000000004197a1 in EuroDriver= >__OApply (cl=3DCannot access memory at address 0x2ac73b0efd38
>=3B >= >=3B ) at ../src/EuroDriver.m3:1545
>=3B >=3B #15 0x00000000009112c1 = >in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174
= >>=3B >=3B #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6
>=3B= > >=3B #17 0x0000000000000000 in ?? ()
>=3B >=3B (gdb)
>=3B &= >gt=3B
>=3B >=3B Here is the code that causes the problem:
>=3B= > >=3B
>=3B >=3B VAR
>=3B >=3B s :=3D CopyTtoS(quer= >y)=3B
>=3B >=3B BEGIN
>=3B >=3B TRY
>=3B >=3B = > t.res :=3D t.myExec(s=2CbusyWait)
>=3B >=3B FINALLY
&= >gt=3B >=3B FreeCopiedS(s) (* stuck here *)
>=3B >=3B E= >ND
>=3B >=3B END=3B
>=3B >=3B
>=3B >=3B The query = >is a SQL query---t.myExec eventually calls the libpq SQL exec
>=3B >= >=3B routine (PQexec I think it's called).
>=3B >=3B
>=3B >= >=3B Since this is already an unsafe module I suppose I could just make anr>>=3B >=3B array of CHARs on the stack and pass a pointer to that...r>>=3B >=3B
>=3B >=3B
>=3B >=3B Regards=2C
>=3B = >>=3B Mika
>=3B
>= > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_-- From hosking at cs.purdue.edu Wed Apr 20 16:58:06 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 10:58:06 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110420055557.089101A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: Agreed. But CVSup used it and we were trying to be supportive. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > The rationale you directed us to says: > > "It is suggested that programs that use fork() call an exec function very > soon afterwards in the child process, thus resetting all states. In the > meantime, only a short list of async-signal-safe library routines are > promised to be available." > > and later > > "Application programs may not be aware that a multi-threaded library > is in use, and they feel free to call any number of library routines > between the fork() and exec calls, just as they always have. Indeed, > they may be extant single-threaded programs and cannot, therefore, > be expected to obey new restrictions imposed by the threads library." > > In C maybe that is true. In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is > no such thing as a "single-threaded" Modula-3 program---or at least, > the distinction doesn't matter. > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > pattern used by some special program---maybe. In any case support for it > I do not think belongs in m3core. > > Mika > > > > Jay K writes: >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >>> This change depends on pthread_atfork=2C for pthreads and user threads. >> >> >> Clarification: There's no change described there for pthreads=2C and indeed= >> =2C getting >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= >> just >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= >> fork()). >> The Solaris default used to be fork =3D> forkall(). >> The current Solaris default is fork =3D> fork1(). >> There really do exist on Solaris to functions you can call=2C fork1() and f= >> orkall(). >> All other systems and the Posix specification have the "fork1" semantic for= >> fork. >> Therefore "fork1" is what we should provide unconditionally. >> If you really want "forkall"=2C then you write: >> #ifdef __sun >> forkall() >> #else >> #error >> #endif >> >> >> >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >> thread/POSIX_FORKALL or such. >> I don't expect anyone to ever do that. >> >> >> >> The need for pthread_atfork is sort of different for user threads vs. pthre= >> ads=2C but it is needed either way. >> For pthreads=2C there is this problem -- at the time a thread calls fork()= >> =2C what are other threads doing? >> Answer: Anything. >> Follow up question: What locks might they hold? >> Answer: Any. >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >> xec=2C and the child process >> attempts to use some of the locks that were held in the parent=2C and only = >> the forking thread survived=2C >> then the other threads will never leave their locks=2C and later attempts t= >> o acquire them in the children >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >> e supposed to use it for=2C >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >> locks=2C and then fork()=2C and then in the >> parent and children=2C release all locks. Therefore=2C you have to come up = >> with a locking order. >> >> >> >> Things are a little different for user threads. >> In user threads=2C the existance of threads is established by virtue of a g= >> lobal array or list of data describing the threads. >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= >> uler picks another thread from the global >> array/list. Previously=2C the data went unchanged when fork() was called. T= >> herefore all threads survived. >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= >> itialize the various data=2C so as to forget about >> all but the current thread. Something like that. >> >> >> >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >> rk=2C then ForkBeforeExec >> might get away with just fork() and ignore all this. >> So far we have not drawn this line -- there is just the one fork and it is = >> used prior to exec or do-more-work >> and does the same thing w/o knowing what is to follow. >> >> >> The Posix documentation I think describes this all fairly well. >> Bing for "posix opengroup pthread_atfork". >> Really. I think it is saying much of the same thing I am saying=2C but in l= >> anguage that has received >> much more thought. Be sure to read the "rationale" part. >> >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >> 2.6-4 >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 >> >> >> >> >> >> >> >> >> Previously: user threads: all threads survived fork() >> Previously: pthreads: only the thread calling fork() survived fork() >> Now: user threads and pthreads: only the thread calling fork() survives for= >> k() >> This must be how pthreads behave=2C and this makes user threads and pthread= >> s consistent. >> >> This change depends on pthread_atfork=2C for pthreads and user threads. >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd. >> Most programs either never fork=2C or exec almost immediately after fork. >> >> >> pthread_atfork offers a good model. >> A sort of "distributed" model. >> You don't have to go and change all the calls to fork(). >> Each module with a need to do something before/after fork=2C calls the cent= >> ral pthread_atfork=2C >> and fork and pthread_atfork cooperate to do what is needed. >> >> >> No function pointer is needed. >> Instead move the code to m3core/thread. >> >> >> If you really must not use -pthread=2C then you must implement pthread_atfo= >> rk functionality >> yourself and have all fork() calls go through your own fork() wrapper that = >> cooperates >> with your pthread_atfork replacement. >> There is no free lunch -- there is a downside to this approach=2C as plain = >> fork() calls >> are ok and correct if pthread_atfork is used=2C but now become incorrect. >> Pick your poison: >> user thread/pthread inconsistency >> cvsupd incompatibility with pthreads=20 >> user threads using pthread_atfork/-pthread=20 >> fork() calls having to go through a wrapper (ok -- you could miss this an= >> d=20 >> not likely notice -- only fork() calls in fork+do-more-work programs nee= >> d the wrapper).=20 >> >> >> (Most of this has been explained multiple times=2C but people only pay atte= >> ntion >> when they think it affects them. I'm guilty of the same thing.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >>> From: mika at async.caltech.edu >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >> c-2.6-4 >>> =20 >>> =20 >>> If you or Tony could describe roughly what you think needs to be done >>> I'd be happy to look into it. >>> =20 >>> The basic problem is that a decision that's made/described in >>> m3core/thread/m3makefile needs to someone find its way to controlling >>> what C code gets compiled elsewhere in the system. Or maybe there should >>> be an indirection into the thread library to pick up pthread_atfork (or n= >> ot). >>> =20 >>> But then again you still haven't explained why you made the user threads = >> use >>> pthread_atfork. I just remove it from my installations=2C but then again= >> I'm >>> not trying to run CVSup so I don't know if it breaks that program to do s= >> o. >>> =20 >>> Mika >>> =20 >>> Jay K writes: >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= >> ry =3D >>>> Unix except >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>> >>>> =3D20 >>>> Ok=3D2C "we" should probably go ahead and fix that. >>>> I'll try to=3D2C but no promises=3D2C sorry. >>>> =3D20 >>>> - Jay >>>> =3D20 >>>>> From: hosking at cs.purdue.edu >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >>>>> To: mika at async.caltech.edu >>>>> CC: m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >> 2C glib=3D >>>> c-2.6-4 >>>>> =3D20 >>>>> Probably unnecessary=3D2C given that I think there is another entry po= >> int t=3D >>>> o forking a process (I forget where) in the thread-specific portion of m= >> 3co=3D >>>> re. In which case the necessary work might be done there? >>>>> =3D20 >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >>>>> =3D20 >>>>>> Tony Hosking writes: >>>>>> ... >>>>>>> pthread_atfork should not be needed under user threads. >>>>>> ... >>>>>> =3D20 >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= >> ver=3D >>>> y Unix except >>>>>> FreeBSD before 6. The comment is from the checked-in source file. >>>>>> =3D20 >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>>>> =3D20 >>>>>> INTEGER >>>>>> __cdecl >>>>>> RTProcess__RegisterForkHandlers( >>>>>> ForkHandler prepare=3D2C >>>>>> ForkHandler parent=3D2C >>>>>> ForkHandler child) >>>>>> { >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. >>>>>> * Win32 lacks pthread_atfork and fork. OK. >>>>>> * >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement >>>>>> * atfork ourselves=3D2C as long as we provide a fork() >>>>>> * wrapper that code uses. >>>>>> */ >>>>>> #if defined(_WIN32) \ >>>>>> || defined(__vms) \ >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >>>>>> return 0=3D3B >>>>>> #else >>>>>> while (1) >>>>>> { >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >>>>>> if (i !=3D3D EAGAIN) >>>>>> return i=3D3B >>>>>> sleep(0)=3D3B >>>>>> } >>>>>> #endif >>>>>> } >>>>> =3D20 >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>> >>>> >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >> nds
=3D >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >> This can be=3D >>>> fixed if need be.
>=3D3B >=3D3B */

>>>>  =3D3B
>>>> Ok=3D2C "we" should probably go ahead and fix that.
>>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>>>>  =3D3B
>>>>  =3D3B- Jay
 =3D3B
>>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >> 011 18:11=3D >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >> @elegos=3D >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >> C kerne=3D >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >> =3D2C given =3D >>>> that I think there is another entry point to forking a process (I forget= >> wh=3D >>>> ere) in the thread-specific portion of m3core. In which case the necessa= >> ry =3D >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >> =3D2C at 2:45=3D >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >> ing writes:=3D >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >> ould not be n=3D >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >>
>=3D3B =3D >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >> rom the=3D >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >> OTE: Even u=3D >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >> s.
&=3D >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >>
>=3D3B =3D >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >> gt=3D3B >=3D3B =3D >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >> e=3D2C>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >> ler child)>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >> hread_atfork.=3D >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >> _atfork=3D >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >> ead_atf=3D >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >> =3D2C for all =3D >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >> elves=3D2C =3D >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >> uses.>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >> =3D3B >=3D3B =3D >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >> p=3D3B&am=3D >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B> R>>=3D3B >=3D >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >> =3D3B >=3D3B in=3D >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >> B >=3D3B if (=3D >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= >> sleep(0)=3D3B>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >> BR>>=3D3B
=3D >>>> >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- >> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >> er threads.


Clarification: There's no change described there for= >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >> epend on using pthread_atfork. That is just
how pthreads are -- except o= >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >> t we should provide unconditionally.
If you really want "forkall"=2C the= >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >> t anyone to ever do that.



The need for pthread_atfork is sor= >> t of different for user threads vs. pthreads=2C but it is needed either way= >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >> process
attempts to use some of the locks that were held in the parent= >> =2C and only the forking thread survived=2C
then the other threads will = >> never leave their locks=2C and later attempts to acquire them in the childr= >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >> n the
parent and children=2C release all locks. Therefore=2C you have to= >> come up with a locking order.



Things are a little different= >> for user threads.
In user threads=2C the existance of threads is establ= >> ished by virtue of a global array or list of data describing the threads.> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >> eduler picks another thread from the global
array/list. Previously=2C th= >> e data went unchanged when fork() was called. Therefore all threads survive= >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >> s reinitialize the various data=2C so as to forget about
all but the cur= >> rent thread. Something like that.



I suspect if we had RTProc= >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >> br>might get away with just fork() and ignore all this.
So far we have n= >> ot drawn this line -- there is just the one fork and it is used prior to ex= >> ec or do-more-work
and does the same thing w/o knowing what is to follow= >> .


The Posix documentation I think describes this all fairly well= >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >> ying much of the same thing I am saying=2C but in language that has receive= >> d
much more thought. Be sure to read the "rationale" part.


&n= >> bsp=3B- Jay


From: jay.krell at cornell.edu<= >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >> : Tue=2C 19 Apr 2011 15:06:38 +0000

>> >> > "> >> >> >> >> >> Previously: user threads: all threads survived fork()
Previously: pthrea= >> ds: only the thread calling fork() survived fork()
Now: user threads and= >> pthreads: only the thread calling fork() survives fork()
This must be h= >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >> br> >> This change depends on pthread_atfork=2C for pthreads and user threads.
= >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= >> tributed" model.
You don't have to go and change all the calls to fork()= >> .
Each module with a need to do something before/after fork=2C calls the= >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >> hat is needed.


No function pointer is needed.
Instead move th= >> e code to m3core/thread.


If you really must not use -pthread=2C = >> then you must implement pthread_atfork functionality
yourself and have a= >> ll fork() calls go through your own fork() wrapper that cooperates
with = >> your pthread_atfork replacement.
There is no free lunch -- there is a do= >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread > r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >> s this and
 =3B =3B not likely notice -- only fork() calls in f= >> ork+do-more-work programs need the wrapper).


(Most of this has = >> been explained multiple times=2C but people only pay attention
when they= >> think it affects them. I'm guilty of the same thing.)


 =3B-= >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >> you or Tony could describe roughly what you think needs to be done
>= >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= >> e needs to someone find its way to controlling
>=3B what C code gets c= >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >> irection into the thread library to pick up pthread_atfork (or not).
>= >> =3B
>=3B But then again you still haven't explained why you made the = >> user threads use
>=3B pthread_atfork. I just remove it from my instal= >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >> know if it breaks that program to do so.
>=3B
>=3B Mika
= >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >> . In which case the necessary work might be done there?
>=3B >=3B>= >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >> =3B >=3B>=3B pthread_atfork should not be needed under user threads.>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= >> The following code from RTProcessC.c ensures that it is neeeded on e=3D>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >> n the thread-specific portion of m3core. In which case the necessary =3D>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
>> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From jay.krell at cornell.edu Thu Apr 21 02:27:15 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 00:27:15 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." Any idea why? I don't know. Is this such a burden? We need a global lock order for it. Unreasonable? I understand pthread_atfork is causing a problem with user threads, but - It really ought not. The libc vs. libpthread design is broken. I believe many systems don't have this broken design, but I guess some do. The broken design is that libc and libpthread implement some functions with the same name, and you get to chose which one. And some of them might be thread-safe. > In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is This is the case in C too. I can't assume any library I link to doesn't create some helper threads with pthreads. pthreads should be assumed always in use, and there should be no ramifications of this. I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread safe vs. ones that are. We really need pthreads to work and to worry less about user threads imho. I believe we have seen the same problems with Win32 threads as with pthreads. (Yes, I know, some people might protest that Win32 threads work fine, but pthreads work very well too. It is only the new stress testing that is turning up problems as I understand, and it turns up problems in pthreads and Win32 threads. Ok, there is now some new mention of problems with other programs?) I suspect the problem is in the allocator/garbage collector, but that user threads don't stress it enough. I suspect I suspect I suspect...whatever.. real debugging is needed, (my own) suspicion and speculation isn't going to help much. It would of course be good if someone anyone can find a point in the history where pthreads and Win32 threads are really solid, with these new one or two test programs, if such a point exists -- I'm not sure one does, or that recent changes are the problem. But, again, I don't know. Time, debugging, investigation is needed. - Jay > From: hosking at cs.purdue.edu > Date: Wed, 20 Apr 2011 10:58:06 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Agreed. But CVSup used it and we were trying to be supportive. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > > > > The rationale you directed us to says: > > > > "It is suggested that programs that use fork() call an exec function very > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." > > > > and later > > > > "Application programs may not be aware that a multi-threaded library > > is in use, and they feel free to call any number of library routines > > between the fork() and exec calls, just as they always have. Indeed, > > they may be extant single-threaded programs and cannot, therefore, > > be expected to obey new restrictions imposed by the threads library." > > > > In C maybe that is true. In Modula-3, every application has *always* > > had to assume that there are other threads floating around. There is > > no such thing as a "single-threaded" Modula-3 program---or at least, > > the distinction doesn't matter. > > > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > > pattern used by some special program---maybe. In any case support for it > > I do not think belongs in m3core. > > > > Mika > > > > > > > > Jay K writes: > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >>> This change depends on pthread_atfork=2C for pthreads and user threads. > >> > >> > >> Clarification: There's no change described there for pthreads=2C and indeed= > >> =2C getting > >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= > >> just > >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > >> fork()). > >> The Solaris default used to be fork =3D> forkall(). > >> The current Solaris default is fork =3D> fork1(). > >> There really do exist on Solaris to functions you can call=2C fork1() and f= > >> orkall(). > >> All other systems and the Posix specification have the "fork1" semantic for= > >> fork. > >> Therefore "fork1" is what we should provide unconditionally. > >> If you really want "forkall"=2C then you write: > >> #ifdef __sun > >> forkall() > >> #else > >> #error > >> #endif > >> > >> > >> > >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= > >> thread/POSIX_FORKALL or such. > >> I don't expect anyone to ever do that. > >> > >> > >> > >> The need for pthread_atfork is sort of different for user threads vs. pthre= > >> ads=2C but it is needed either way. > >> For pthreads=2C there is this problem -- at the time a thread calls fork()= > >> =2C what are other threads doing? > >> Answer: Anything. > >> Follow up question: What locks might they hold? > >> Answer: Any. > >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= > >> xec=2C and the child process > >> attempts to use some of the locks that were held in the parent=2C and only = > >> the forking thread survived=2C > >> then the other threads will never leave their locks=2C and later attempts t= > >> o acquire them in the children > >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= > >> e supposed to use it for=2C > >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = > >> locks=2C and then fork()=2C and then in the > >> parent and children=2C release all locks. Therefore=2C you have to come up = > >> with a locking order. > >> > >> > >> > >> Things are a little different for user threads. > >> In user threads=2C the existance of threads is established by virtue of a g= > >> lobal array or list of data describing the threads. > >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= > >> uler picks another thread from the global > >> array/list. Previously=2C the data went unchanged when fork() was called. T= > >> herefore all threads survived. > >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= > >> itialize the various data=2C so as to forget about > >> all but the current thread. Something like that. > >> > >> > >> > >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= > >> rk=2C then ForkBeforeExec > >> might get away with just fork() and ignore all this. > >> So far we have not drawn this line -- there is just the one fork and it is = > >> used prior to exec or do-more-work > >> and does the same thing w/o knowing what is to follow. > >> > >> > >> The Posix documentation I think describes this all fairly well. > >> Bing for "posix opengroup pthread_atfork". > >> Really. I think it is saying much of the same thing I am saying=2C but in l= > >> anguage that has received > >> much more thought. Be sure to read the "rationale" part. > >> > >> > >> - Jay > >> > >> > >> From: jay.krell at cornell.edu > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= > >> 2.6-4 > >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > >> > >> > >> > >> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork() > >> Previously: pthreads: only the thread calling fork() survived fork() > >> Now: user threads and pthreads: only the thread calling fork() survives for= > >> k() > >> This must be how pthreads behave=2C and this makes user threads and pthread= > >> s consistent. > >> > >> This change depends on pthread_atfork=2C for pthreads and user threads. > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd. > >> Most programs either never fork=2C or exec almost immediately after fork. > >> > >> > >> pthread_atfork offers a good model. > >> A sort of "distributed" model. > >> You don't have to go and change all the calls to fork(). > >> Each module with a need to do something before/after fork=2C calls the cent= > >> ral pthread_atfork=2C > >> and fork and pthread_atfork cooperate to do what is needed. > >> > >> > >> No function pointer is needed. > >> Instead move the code to m3core/thread. > >> > >> > >> If you really must not use -pthread=2C then you must implement pthread_atfo= > >> rk functionality > >> yourself and have all fork() calls go through your own fork() wrapper that = > >> cooperates > >> with your pthread_atfork replacement. > >> There is no free lunch -- there is a downside to this approach=2C as plain = > >> fork() calls > >> are ok and correct if pthread_atfork is used=2C but now become incorrect. > >> Pick your poison: > >> user thread/pthread inconsistency > >> cvsupd incompatibility with pthreads=20 > >> user threads using pthread_atfork/-pthread=20 > >> fork() calls having to go through a wrapper (ok -- you could miss this an= > >> d=20 > >> not likely notice -- only fork() calls in fork+do-more-work programs nee= > >> d the wrapper).=20 > >> > >> > >> (Most of this has been explained multiple times=2C but people only pay atte= > >> ntion > >> when they think it affects them. I'm guilty of the same thing.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu > >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >>> From: mika at async.caltech.edu > >>> CC: m3devel at elegosoft.com > >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >> c-2.6-4 > >>> =20 > >>> =20 > >>> If you or Tony could describe roughly what you think needs to be done > >>> I'd be happy to look into it. > >>> =20 > >>> The basic problem is that a decision that's made/described in > >>> m3core/thread/m3makefile needs to someone find its way to controlling > >>> what C code gets compiled elsewhere in the system. Or maybe there should > >>> be an indirection into the thread library to pick up pthread_atfork (or n= > >> ot). > >>> =20 > >>> But then again you still haven't explained why you made the user threads = > >> use > >>> pthread_atfork. I just remove it from my installations=2C but then again= > >> I'm > >>> not trying to run CVSup so I don't know if it breaks that program to do s= > >> o. > >>> =20 > >>> Mika > >>> =20 > >>> Jay K writes: > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= > >> ry =3D > >>>> Unix except > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>> > >>>> =3D20 > >>>> Ok=3D2C "we" should probably go ahead and fix that. > >>>> I'll try to=3D2C but no promises=3D2C sorry. > >>>> =3D20 > >>>> - Jay > >>>> =3D20 > >>>>> From: hosking at cs.purdue.edu > >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >>>>> To: mika at async.caltech.edu > >>>>> CC: m3devel at elegosoft.com > >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >> 2C glib=3D > >>>> c-2.6-4 > >>>>> =3D20 > >>>>> Probably unnecessary=3D2C given that I think there is another entry po= > >> int t=3D > >>>> o forking a process (I forget where) in the thread-specific portion of m= > >> 3co=3D > >>>> re. In which case the necessary work might be done there? > >>>>> =3D20 > >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >>>>> =3D20 > >>>>>> Tony Hosking writes: > >>>>>> ... > >>>>>>> pthread_atfork should not be needed under user threads. > >>>>>> ... > >>>>>> =3D20 > >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= > >> ver=3D > >>>> y Unix except > >>>>>> FreeBSD before 6. The comment is from the checked-in source file. > >>>>>> =3D20 > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>>>> =3D20 > >>>>>> INTEGER > >>>>>> __cdecl > >>>>>> RTProcess__RegisterForkHandlers( > >>>>>> ForkHandler prepare=3D2C > >>>>>> ForkHandler parent=3D2C > >>>>>> ForkHandler child) > >>>>>> { > >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. > >>>>>> * Win32 lacks pthread_atfork and fork. OK. > >>>>>> * > >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement > >>>>>> * atfork ourselves=3D2C as long as we provide a fork() > >>>>>> * wrapper that code uses. > >>>>>> */ > >>>>>> #if defined(_WIN32) \ > >>>>>> || defined(__vms) \ > >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >>>>>> return 0=3D3B > >>>>>> #else > >>>>>> while (1) > >>>>>> { > >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >>>>>> if (i !=3D3D EAGAIN) > >>>>>> return i=3D3B > >>>>>> sleep(0)=3D3B > >>>>>> } > >>>>>> #endif > >>>>>> } > >>>>> =3D20 > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >> nds
=3D > >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >> This can be=3D > >>>> fixed if need be.
>=3D3B >=3D3B */

> >>>>  =3D3B
> >>>> Ok=3D2C "we" should probably go ahead and fix that.
> >>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >>>>  =3D3B
> >>>>  =3D3B- Jay
 =3D3B
> >>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >> 011 18:11=3D > >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >> @elegos=3D > >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >> C kerne=3D > >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >> =3D2C given =3D > >>>> that I think there is another entry point to forking a process (I forget= > >> wh=3D > >>>> ere) in the thread-specific portion of m3core. In which case the necessa= > >> ry =3D > >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >> =3D2C at 2:45=3D > >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >> ing writes:=3D > >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >> ould not be n=3D > >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >>
>=3D3B =3D > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >> rom the=3D > >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >> OTE: Even u=3D > >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >> s.
&=3D > >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >>
>=3D3B =3D > >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >> gt=3D3B >=3D3B =3D > >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >> e=3D2C >>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >> ler child) >>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >> hread_atfork.=3D > >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >> _atfork=3D > >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >> ead_atf=3D > >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >> =3D2C for all =3D > >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >> elves=3D2C =3D > >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >> uses. >>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >> =3D3B >=3D3B =3D > >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >> p=3D3B&am=3D > >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >> R>>=3D3B >=3D > >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >> =3D3B >=3D3B in=3D > >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >> B >=3D3B if (=3D > >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > >> sleep(0)=3D3B >>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >> BR>>=3D3B
=3D > >>>> > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >> > >> > >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= > >> er threads.


Clarification: There's no change described there for= > >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= > >> epend on using pthread_atfork. That is just
how pthreads are -- except o= > >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= > >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= > >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= > >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= > >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= > >> t we should provide unconditionally.
If you really want "forkall"=2C the= > >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= > >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= > >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= > >> t anyone to ever do that.



The need for pthread_atfork is sor= > >> t of different for user threads vs. pthreads=2C but it is needed either way= > >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= > >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= > >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= > >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = > >> process
attempts to use some of the locks that were held in the parent= > >> =2C and only the forking thread survived=2C
then the other threads will = > >> never leave their locks=2C and later attempts to acquire them in the childr= > >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = > >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= > >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= > >> n the
parent and children=2C release all locks. Therefore=2C you have to= > >> come up with a locking order.



Things are a little different= > >> for user threads.
In user threads=2C the existance of threads is establ= > >> ished by virtue of a global array or list of data describing the threads. >> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= > >> eduler picks another thread from the global
array/list. Previously=2C th= > >> e data went unchanged when fork() was called. Therefore all threads survive= > >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= > >> s reinitialize the various data=2C so as to forget about
all but the cur= > >> rent thread. Something like that.



I suspect if we had RTProc= > >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= > >> br>might get away with just fork() and ignore all this.
So far we have n= > >> ot drawn this line -- there is just the one fork and it is used prior to ex= > >> ec or do-more-work
and does the same thing w/o knowing what is to follow= > >> .


The Posix documentation I think describes this all fairly well= > >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= > >> ying much of the same thing I am saying=2C but in language that has receive= > >> d
much more thought. Be sure to read the "rationale" part.


&n= > >> bsp=3B- Jay


From: jay.krell at cornell.edu<= > >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = > >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= > >> : Tue=2C 19 Apr 2011 15:06:38 +0000

> >> > >> >> "> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork()
Previously: pthrea= > >> ds: only the thread calling fork() survived fork()
Now: user threads and= > >> pthreads: only the thread calling fork() survives fork()
This must be h= > >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >> br> > >> This change depends on pthread_atfork=2C for pthreads and user threads.
= > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >> tributed" model.
You don't have to go and change all the calls to fork()= > >> .
Each module with a need to do something before/after fork=2C calls the= > >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >> hat is needed.


No function pointer is needed.
Instead move th= > >> e code to m3core/thread.


If you really must not use -pthread=2C = > >> then you must implement pthread_atfork functionality
yourself and have a= > >> ll fork() calls go through your own fork() wrapper that cooperates
with = > >> your pthread_atfork replacement.
There is no free lunch -- there is a do= > >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread >> r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >> s this and
 =3B =3B not likely notice -- only fork() calls in f= > >> ork+do-more-work programs need the wrapper).


(Most of this has = > >> been explained multiple times=2C but people only pay attention
when they= > >> think it affects them. I'm guilty of the same thing.)


 =3B-= > >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >> you or Tony could describe roughly what you think needs to be done
>= > >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >> e needs to someone find its way to controlling
>=3B what C code gets c= > >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >> irection into the thread library to pick up pthread_atfork (or not).
>= > >> =3B
>=3B But then again you still haven't explained why you made the = > >> user threads use
>=3B pthread_atfork. I just remove it from my instal= > >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >> know if it breaks that program to do so.
>=3B
>=3B Mika
= > >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >> . In which case the necessary work might be done there?
>=3B >=3B>= > >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >> =3B >=3B>=3B pthread_atfork should not be needed under user threads. >>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > >> The following code from RTProcessC.c ensures that it is neeeded on e=3D >>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >> n the thread-specific portion of m3core. In which case the necessary =3D >>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Apr 21 04:02:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:02:47 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: <20110421020247.31A0A1A2078@async.async.caltech.edu> Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From mika at async.caltech.edu Thu Apr 21 04:53:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:53:04 -0700 Subject: [M3devel] wrapping postgresql library not enough Message-ID: <20110421025304.685501A2078@async.async.caltech.edu> Hi again, So, as I described before, I wrote a program that wraps all of Critical Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency at which my application locks up, but it has not eliminated the lockups. User threads, AMD64_LINUX. I'm not quite sure what to try next. How can I find out where malloc and free might be used reentrantly, assuming that is the problem? There may not be any reliable threading in CM3 anymore. Possibly it's reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. Mika (gdb) where #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 #3 0x000000000077b08d in PQsendQuery () #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 ) at ../AMD64_LINUX/PQSchedulerWrap.m3:210 #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 ) at ../src/UnsafeDatabase.m3:92 #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 ) at ../src/UnsafeDatabase.m3:239 #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 ) at ../src/UnsafeDatabase.m3:269 #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 ) at ../src/DesynchronizedDB.m3:74 #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () (gdb) From hosking at cs.purdue.edu Thu Apr 21 05:12:53 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:12:53 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421025304.685501A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> Message-ID: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Can you dump all the user threads at this point? ThreadPosix__DumpEverybody() should do it. We want to see where the other threads might be. On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > Hi again, > > So, as I described before, I wrote a program that wraps all of Critical > Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / > Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency > at which my application locks up, but it has not eliminated the lockups. > > User threads, AMD64_LINUX. > > I'm not quite sure what to try next. How can I find out where malloc > and free might be used reentrantly, assuming that is the problem? > > There may not be any reliable threading in CM3 anymore. Possibly it's > reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. > > Mika > > (gdb) where > #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 > #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 > #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 > #3 0x000000000077b08d in PQsendQuery () > #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 > ) > at ../AMD64_LINUX/PQSchedulerWrap.m3:210 > #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 > ) at ../src/UnsafeDatabase.m3:92 > #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 > ) at ../src/UnsafeDatabase.m3:239 > #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 > ) at ../src/UnsafeDatabase.m3:269 > #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 > ) at ../src/DesynchronizedDB.m3:74 > #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > (gdb) From mika at async.caltech.edu Thu Apr 21 05:20:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 20:20:44 -0700 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Message-ID: <20110421032044.9BDAA1A2078@async.async.caltech.edu> Aha, I will try that if it happens again. I wrapped more things that I think might be iffy in my program. But I have a feeling ThreadPosix__DumpEverybody won't work... the application is probably deadlocked by having a single thread locking against itself with some sort of pthread lock. I should look at the source code of free. Mika Tony Hosking writes: >Can you dump all the user threads at this point? = >ThreadPosix__DumpEverybody() should do it. We want to see where the = >other threads might be. > >On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > >> Hi again, >>=20 >> So, as I described before, I wrote a program that wraps all of = >Critical >> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >frequency >> at which my application locks up, but it has not eliminated the = >lockups. >>=20 >> User threads, AMD64_LINUX. >>=20 >> I'm not quite sure what to try next. How can I find out where malloc >> and free might be used reentrantly, assuming that is the problem? >>=20 >> There may not be any reliable threading in CM3 anymore. Possibly it's >> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>=20 >> Mika >>=20 >> (gdb) where >> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >> #3 0x000000000077b08d in PQsendQuery () >> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >access memory at address 0x7f7b848a4298 >> ) >> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >memory at address 0x7f7b848a4328 >> ) at ../src/UnsafeDatabase.m3:92 >> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >memory at address 0x7f7b848a43f8 >> ) at ../src/UnsafeDatabase.m3:239 >> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >memory at address 0x7f7b848a4528 >> ) at ../src/UnsafeDatabase.m3:269 >> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >access memory at address 0x7f7b848a45e8 >> ) at ../src/DesynchronizedDB.m3:74 >> #9 0x00000000005aeac6 in = >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >(finalAttempt=3DCannot access memory at address 0x7f7b848a463f >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >(cl=3DCannot access memory at address 0x7f7b848a4778 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >> #11 0x00000000005b171e in = >DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >at address 0x7f7b848a4958 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >(cl=3DCannot access memory at address 0x7f7b848a4ba8 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >(cl=3DCannot access memory at address 0x7f7b848a4d58 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >../src/thread/POSIX/ThreadPosix.m3:1174 >> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >> #16 0x0000000000000000 in ?? () >> (gdb)=20 From hosking at cs.purdue.edu Thu Apr 21 05:46:37 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:46:37 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421032044.9BDAA1A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> <20110421032044.9BDAA1A2078@async.async.caltech.edu> Message-ID: <2EDBC39D-053F-4C7A-AD64-7D202E080581@cs.purdue.edu> If you are able to get the debugger to stop there then you should still be able to dump the threads from the debugger: call ThreadPosix__DumpEverybody() Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 11:20 PM, Mika Nystrom wrote: > Aha, I will try that if it happens again. I wrapped more things that I think > might be iffy in my program. > > But I have a feeling ThreadPosix__DumpEverybody won't work... the > application is probably deadlocked by having a single thread locking > against itself with some sort of pthread lock. I should look at the > source code of free. > > Mika > > Tony Hosking writes: >> Can you dump all the user threads at this point? = >> ThreadPosix__DumpEverybody() should do it. We want to see where the = >> other threads might be. >> >> On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: >> >>> Hi again, >>> =20 >>> So, as I described before, I wrote a program that wraps all of = >> Critical >>> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >>> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >> frequency >>> at which my application locks up, but it has not eliminated the = >> lockups. >>> =20 >>> User threads, AMD64_LINUX. >>> =20 >>> I'm not quite sure what to try next. How can I find out where malloc >>> and free might be used reentrantly, assuming that is the problem? >>> =20 >>> There may not be any reliable threading in CM3 anymore. Possibly it's >>> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>> =20 >>> Mika >>> =20 >>> (gdb) where >>> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >>> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >>> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >>> #3 0x000000000077b08d in PQsendQuery () >>> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >> access memory at address 0x7f7b848a4298 >>> ) >>> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >>> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >> memory at address 0x7f7b848a4328 >>> ) at ../src/UnsafeDatabase.m3:92 >>> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >> memory at address 0x7f7b848a43f8 >>> ) at ../src/UnsafeDatabase.m3:239 >>> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >> memory at address 0x7f7b848a4528 >>> ) at ../src/UnsafeDatabase.m3:269 >>> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >> access memory at address 0x7f7b848a45e8 >>> ) at ../src/DesynchronizedDB.m3:74 >>> #9 0x00000000005aeac6 in = >> DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >> (finalAttempt=3DCannot access memory at address 0x7f7b848a463f >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >>> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >> (cl=3DCannot access memory at address 0x7f7b848a4778 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >>> #11 0x00000000005b171e in = >> DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >> at address 0x7f7b848a4958 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >>> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >> (cl=3DCannot access memory at address 0x7f7b848a4ba8 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >>> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >> (cl=3DCannot access memory at address 0x7f7b848a4d58 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >>> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >> ../src/thread/POSIX/ThreadPosix.m3:1174 >>> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >>> #16 0x0000000000000000 in ?? () >>> (gdb)=20 From hendrik at topoi.pooq.com Thu Apr 21 15:19:04 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 09:19:04 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: <20110421131904.GB22222@topoi.pooq.com> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > Agreed. But CVSup used it and we were trying to be supportive. So, at present, CVSup presumably works with user threads. Maybe the cure is to fix CVSup instead of fork(). Why does CVSup needs a fork() that preserves all threads? Is it essential to its design, or incidental? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:20:54 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:20:54 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421131904.GB22222@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> Message-ID: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >> Agreed. But CVSup used it and we were trying to be supportive. > > So, at present, CVSup presumably works with user threads. > Maybe the cure is to fix CVSup instead of fork(). > > Why does CVSup needs a fork() that preserves all threads? Is > it essential to its design, or incidental? CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > -- hendrik From hendrik at topoi.pooq.com Thu Apr 21 16:28:26 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:28:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> Message-ID: <20110421142826.GA23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: > > On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > > > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > >> Agreed. But CVSup used it and we were trying to be supportive. > > > > So, at present, CVSup presumably works with user threads. > > Maybe the cure is to fix CVSup instead of fork(). > > > > Why does CVSup needs a fork() that preserves all threads? Is > > it essential to its design, or incidental? > > CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. Does it need to use fork() at all? Wuld it suffice to use Modula 3's own thread system? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:30:25 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:30:25 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421142826.GA23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: I'm not familiar with the architecture of CVSup. On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >> >> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >> >>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>> Agreed. But CVSup used it and we were trying to be supportive. >>> >>> So, at present, CVSup presumably works with user threads. >>> Maybe the cure is to fix CVSup instead of fork(). >>> >>> Why does CVSup needs a fork() that preserves all threads? Is >>> it essential to its design, or incidental? >> >> CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > Does it need to use fork() at all? Wuld it suffice to use Modula 3's > own thread system? > > -- hendrik From wagner at elegosoft.com Thu Apr 21 16:47:33 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 16:47:33 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Quoting Tony Hosking : > I'm not familiar with the architecture of CVSup. > > On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > >> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>> >>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>> >>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>> >>>> So, at present, CVSup presumably works with user threads. >>>> Maybe the cure is to fix CVSup instead of fork(). >>>> >>>> Why does CVSup needs a fork() that preserves all threads? Is >>>> it essential to its design, or incidental? >>> >>> CVSup uses fork in an ill-defined way. fork is only well-defined >>> for use as fork+exec. So, the fact that there are continuing >>> threads is immaterial. CVSup wants to continue doing real work >>> (without exec) after fork. >> >> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >> own thread system? Not for real word server applications, where one problem in a user session would crash dozens of other sessions. At least this was what John Polstra reported. The additional safety of separate address spaces _has_ its advantages. I thought CVSup had been changed to work with pthreads though in CM3? Or do I misremember? Anyway, this shouldn't be our most important problem, but rather make System pthreads and also M3 user threads work in a reliable and fault-free way again. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hendrik at topoi.pooq.com Thu Apr 21 16:54:27 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:54:27 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Message-ID: <20110421145427.GC23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: > Quoting Tony Hosking : > >> I'm not familiar with the architecture of CVSup. >> >> On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: >> >>> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>>> >>>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>>> >>>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>>> >>>>> So, at present, CVSup presumably works with user threads. >>>>> Maybe the cure is to fix CVSup instead of fork(). >>>>> >>>>> Why does CVSup needs a fork() that preserves all threads? Is >>>>> it essential to its design, or incidental? >>>> >>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>> for use as fork+exec. So, the fact that there are continuing >>>> threads is immaterial. CVSup wants to continue doing real work >>>> (without exec) after fork. >>> >>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>> own thread system? > > Not for real word server applications, where one problem in a user > session would crash dozens of other sessions. At least this was what > John Polstra reported. The additional safety of separate address > spaces _has_ its advantages. > > I thought CVSup had been changed to work with pthreads though in CM3? > Or do I misremember? > > Anyway, this shouldn't be our most important problem, but rather make > System pthreads and also M3 user threads work in a reliable and > fault-free way again. Once our only problems are with CVSup's abuse of fork(), I'd consider that done. -- hendrik From mika at async.caltech.edu Thu Apr 21 17:07:48 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 21 Apr 2011 08:07:48 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421150748.62E2B1A2078@async.async.caltech.edu> Hendrik Boom writes: ... >>>>> >>>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>>> for use as fork+exec. So, the fact that there are continuing >>>>> threads is immaterial. CVSup wants to continue doing real work >>>>> (without exec) after fork. >>>> >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real word server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > >Once our only problems are with CVSup's abuse of fork(), I'd >consider that done. Unfortunately, no. CVSup's ab/use of fork() only affects some software-engineering issues at the margin of the threading problem. It does do bad things to portability to have to support it. The Modula-3 way of doing what CVSup does is to use Process.Create to create a new process. If there is state in the current process that needs to be transmitted to the new process, Pickle the state and send it over a Pipe into the new process. Using fork() to do a kind of pseudo-shared-memory should not be necessary. I think. Mika From wagner at elegosoft.com Thu Apr 21 17:14:24 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 17:14:24 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Quoting Hendrik Boom : > On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: [...] >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real world server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > > Once our only problems are with CVSup's abuse of fork(), I'd > consider that done. I think Mika has pointed out quite a few problems that haven't been solved yet (or I missed something, which may well be); he even contributed a sophisticated test program for validation of threads. If it hasn't been done yet, CVSup can probably easily be adapted to other thread-survival-during-fork patterns; we just should be able to provide a consistent specification and implementation for all platforms, for both threads and process semantics. However, I'll leave the office now for Easter holidays, and wish you all sunny and restful Easter days! Olaf PS: I'm still surprised that none of those recent problems has found its way into our fault tracker... -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 21 18:27:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 21 Apr 2011 17:27:45 +0100 (BST) Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421151025.A8AC31A2078@async.async.caltech.edu> Message-ID: <980045.69138.qm@web29713.mail.ird.yahoo.com> Hi all: sure I can actually it work ?s with WeakRef, somehow interesting, is there a m3-db/m3postgresql or ? Besides the compilation problems, zeus3D package misses sources of at least 4 interface files and its not public dependences: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/src/ Aux, Camera, G, Light perhaps they were exported as hide_Interface() with its own modules if so. They are a kind of helper or friend modules ti it, so they are missing in public three there. There is also a local copy of Obliq and Modula-3 packages of modules of models of lego tools, so it might be interesting for somebody to play with them, it's fairly a good size library written by an old M3 friend :), as are the algorithm animations done for zeus3D at DEC SRC. There is another one built package to animate distributed protocols called Zada, it might be interesting to plug it all, even there is a formal specification of the same problem, a Manufacturing Cell, local projects name was Didactic Manufacturing Cell thesis (all sources available for free just that Obliq is not working as far I can tell) Zada related relevant links (missing main web page is not under development any more, maybe we should ask them what they have about it and perhaps even their thesis) they had even audio: http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-agarwal.html http://ls4-www.cs.tu-dortmund.de/RVS/Exports/EB_PG225.ps.Z http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-kukasch.html So its would a good package animation example of lego tools, specifications, and protocols of all the framework, don't you think? Thanks in advance PD: just if you care of and have at 2 PM ET today there is a conference on embedded software verification, so it might be good to see what this guys are looking for, maybe we can give them a look of it? http://www.eetimes.com/design/eda-design/4215099/Using-verification-coverage-with-formal-analysis --- El jue, 21/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] wrapping postgresql library not enough > Para: "Daniel Alejandro Benavides D." > Fecha: jueves, 21 de abril, 2011 10:10 > Hi Daniel, > > Sure I'd be happy to look at it (although I *may* already > have fixed my > problems, I don't know). Would it be OK to add the > PSQL driver you are talking > about to CM3? > > Can you give me a filename of some file you think would be > in zeus3d and I can > go looking in all my files for it? > > Mika > > "Daniel Alejandro Benavides D." writes: > >Hi Mika: > >I have a local copy of Daniel Solaz's 2001 PSQL driver, > if needed, just in = > >case you want it I can send right away to you. > >Besides: Did you know or know how to find out zeus3d > sources besides the pu= > >blic package? > > > >Thanks in advance > > > >--- El mi=E9, 20/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] wrapping postgresql library > not enough > >> Para: "Tony Hosking" > >> CC: m3devel at elegosoft.com > >> Fecha: mi=E9rcoles, 20 de abril, 2011 22:20 > >> Aha, I will try that if it happens > >> again. I wrapped more things that I think > >> might be iffy in my program. > >>=20 > >> But I have a feeling ThreadPosix__DumpEverybody > won't > >> work... the > >> application is probably deadlocked by having a > single > >> thread locking > >> against itself with some sort of pthread > lock. I > >> should look at the > >> source code of free. > >>=20 > >> Mika > >>=20 > >> Tony Hosking writes: > >> >Can you dump all the user threads at this > point?=20 > >> =3D > >> >ThreadPosix__DumpEverybody() should do > it. We > >> want to see where the =3D > >> >other threads might be. > >> > > >> >On Apr 20, 2011, at 10:53 PM, Mika Nystrom > wrote: > >> > > >> >> Hi again, > >> >>=3D20 > >> >> So, as I described before, I wrote a > program that > >> wraps all of =3D > >> >Critical > >> >> Mass's "postgresql" library (PQ.i3) with > >> Scheduler.DisableSwitching / > >> >> Scheduler.EnableSwitching. Yes, it > seems to > >> have reduced the =3D > >> >frequency > >> >> at which my application locks up, but it > has not > >> eliminated the =3D > >> >lockups. > >> >>=3D20 > >> >> User threads, AMD64_LINUX. > >> >>=3D20 > >> >> I'm not quite sure what to try > next. How can > >> I find out where malloc > >> >> and free might be used reentrantly, > assuming that > >> is the problem? > >> >>=3D20 > >> >> There may not be any reliable threading > in CM3 > >> anymore. Possibly it's > >> >> reliable on FreeBSD (without -pthread) > but I > >> wouldn't bet money on it. > >> >>=3D20 > >> >> Mika > >> >>=3D20 > >> >> (gdb) where > >> >> #0 0x00007f7a8da0102e in ?? () > from > >> /lib/libc.so.6 > >> >> #1 0x00007f7a8d99eb43 in ?? () > from > >> /lib/libc.so.6 > >> >> #2 0x00007f7a8d99baab in free () > from > >> /lib/libc.so.6 > >> >> #3 0x000000000077b08d in > PQsendQuery () > >> >> #4 0x0000000000767c12 in > >> PQSchedulerWrap__PQsendQuery (conn=3D3DCannot =3D > >> >access memory at address 0x7f7b848a4298 > >> >> ) > >> >> at > >> ../AMD64_LINUX/PQSchedulerWrap.m3:210 > >> >> #5 0x0000000000763ce0 in > >> UnsafeDatabase__MyExec (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4328 > >> >> ) at ../src/UnsafeDatabase.m3:92 > >> >> #6 0x0000000000764a29 in > >> UnsafeDatabase__ExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a43f8 > >> >> ) at ../src/UnsafeDatabase.m3:239 > >> >> #7 0x0000000000764cee in > >> UnsafeDatabase__TExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4528 > >> >> ) at ../src/UnsafeDatabase.m3:269 > >> >> #8 0x0000000000756638 in > >> DesynchronizedDB__TransSExec (trans=3D3DCannot > =3D > >> >access memory at address 0x7f7b848a45e8 > >> >> ) at ../src/DesynchronizedDB.m3:74 > >> >> #9 0x00000000005aeac6 in =3D > >> > >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 > >> =3D > >> >(finalAttempt=3D3DCannot access memory at > address > >> 0x7f7b848a463f > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > >> >> #10 0x00000000005ae639 in > >> DBTable_gcoms_ordr_statusMonitor__ScanDirty =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4778 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > >> >> #11 0x00000000005b171e in =3D > >> >DBTable_gcoms_ordr_statusMonitor__CheckForNew > >> (cl=3D3DCannot access memory =3D > >> >at address 0x7f7b848a4958 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > >> >> #12 0x00000000005b1382 in > >> DBTable_gcoms_ordr_statusMonitor__Sync =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4ba8 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > >> >> #13 0x00000000005b0e1c in > >> DBTable_gcoms_ordr_statusMonitor__ApplyC =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4d58 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > >> >> #14 0x0000000000922eb5 in > ThreadPosix__RunThread > >> () at =3D > >> >../src/thread/POSIX/ThreadPosix.m3:1174 > >> >> #15 0x00007f7a8d9697b0 in ?? () from > >> /lib/libc.so.6 > >> >> #16 0x0000000000000000 in ?? () > >> >> (gdb)=3D20 > >> > From rcolebur at SCIRES.COM Thu Apr 21 19:48:57 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 13:48:57 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I concur with Mika ! We need to stick to the design tennants of Modula-3. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Wednesday, April 20, 2011 10:03 PM To: Jay K Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From jay.krell at cornell.edu Thu Apr 21 19:47:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 17:47:01 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421150748.62E2B1A2078@async.async.caltech.edu> References: , , <20110420055557.089101A207A@async.async.caltech.edu>, , <20110421131904.GB22222@topoi.pooq.com>, <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu>, <20110421142826.GA23127@topoi.pooq.com>, , <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com>, <20110421145427.GC23127@topoi.pooq.com>, <20110421150748.62E2B1A2078@async.async.caltech.edu> Message-ID: > Unfortunately, no. CVSup's ab/use of fork() only affects some > software-engineering issues at the margin of the threading problem. > It does do bad things to portability to have to support it. > > The Modula-3 way of doing what CVSup does is to use Process.Create to > create a new process. If there is state in the current process that > needs to be transmitted to the new process, Pickle the state and send > it over a Pipe into the new process. Using fork() to do a kind of > pseudo-shared-memory should not be necessary. I think. What cvsup does is more efficient. But I'm not sure if by much. You get a bunch of read-only shared state, including sharing physical memory. Most other forms of inter-process-communication will duplicate the data. What cvsup does is not portable to Windows. And is not efficient on Cygwin. Though it is portable to Interix and efficient there. The NT kernel supports fork() in the manner everyone is familiar with, but Win32 doesn't expose it. Anyway, we still don't know what are the problems currently, and they might exist on Win32 also. Or at least there are problems on Win64. The test program needs to be tried on 32bit Windows. And we need to consider removing use of GetThreadContext, and probably SuspendThread/ResumeThread as well. I understand the -pthread dependency might be undesirable in user threads and it should be easy to remove it, with the tradeoff that direct fork() won't do the right thing (though it will be racy and therefore often appear to work). - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Apr 21 19:35:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 13:35:45 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Message-ID: fork + exec works fine and has always worked fine. fork + carry on is what Jay fixed with pthread_atfork. On Apr 21, 2011, at 11:14 AM, Olaf Wagner wrote: > If it hasn't been done yet, CVSup can probably easily be adapted to > other thread-survival-during-fork patterns; we just should be able > to provide a consistent specification and implementation for all > platforms, for both threads and process semantics. From jay.krell at cornell.edu Thu Apr 21 20:42:21 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 18:42:21 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 21 21:21:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 19:21:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I forgot to reply to this point, which keep coming up: > starts blurring the line between what is and is not Modula-3. This could > turn out to be a problem if Modula-3 is in future again ported to systems > without complete, modern C environments. By "problem" I mean an extra > bifurcation of application programs: some programs for "POSIX Modula-3" Cvsup won't work in such a system without a certain amount of work. Such a system requires a fairly significant amount of work. Nothing discussed here really touches on the work required. Such a system needs threads implemented in *some* fashion. Either pthreads, or sigaltstack, or jmpbuf smashing, or somehow natively in Modula-3, but ultimately, you need assembly and/or processor-specificness *somewhere*. You must save restore registers somewhere. You must setup a stack somewhere. You need preemption somehow (well, maybe not -- "cooperative suspend"). And you need to provide for file I/O somewhere, depending on what sort of program you want to run. And socket I/O, ditto. And keyboard/video/mouse/console functionality, ditto. You can use more or less of the facilities of the underlying system, giving you more or less functionality for free/cheap, obligating you to implement more or less of the functionality you require or desire. Modula-3 layers on top of Posix and Win32, with more or less thickness and state. Modula-3 was kind of interesting in that it provided threads with a certain amount of portability without requiring an underlying threading capability, with just timers for preemption and setjmp/longjmp for context save/restore. This is still somewhat interesting, e.g. maybe for DJGPP, but pthreads and Win32 threads are widespread now, and providing these features isn't particularly valuable or differentiating. You know -- why use Modula-3? One of the reasons in the past was maybe for portable threading. That is still a reason, but portable threading is not as difficult to write for one's self these days, merely by layering over pthreads and Win32. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 00:23:41 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 01:15:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 23:15:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 01:28:59 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 00:28:59 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <284553.92617.qm@web29701.mail.ird.yahoo.com> Hi all: one way of being us safe and capable of doing more than traditional modification of sources each time the system changes (which is basically almost everyone blindly agrees, sorry if I'm mad, but who else aside of us cares about this?) and avoid this dangerous path to near disaster of Modula-3 specifications is to selfhost in our threads. Modula-3 has been almost hard to break system, and even if Modula-3 is modified must remain there, no UNSAFE stuff imploded, no every one in its place, no dangerous threading with locks every 10, 20 30 milliseconds, no more, we are here to say again guys just make things clear and quick and simple, don't make us to change our implementations each time you are making its. I mean Linux KVM, Solaris Zones, Win HV, it isn't that hard to believe it, don't you think? Is about what guys there did with SPIN, I'm going back to that, but is useful for one reason, the only way to keep function stability and I mean that for us in some sense could be that, remember we are able to test stress a VM in such an environment as most as we would want, knowing every fault there is ones besides HyperThreading is making its presence in hardware assisted virtualization? then is a matter of what we want to achieve the next level is up to us. That said originally the DEC Unix already had emulation of threads, of C POSIX threads? (now called pthreads today), of Mach 3.0 layered ones and DEC OSF too and besides and own strends for more than the most of any system can think of. See: http://qstream.org/~krasic/cs508-2006/summaries/paper17/SpinOS.ppt Thanks in advance --- El jue, 21/4/11, Coleburn, Randy escribi?: De: Coleburn, Randy Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "m3devel" Fecha: jueves, 21 de abril, 2011 17:23 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. ?Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.?? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages.? I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible.? ?I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?.? The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika.? It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform.? And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only.? Everything else is completely portable. ?Regards,Randy Coleburn ?From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 ?> I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 05:24:02 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I'm not advocating we "throw the baby out with the bath water", but if the "fork-and-do-more" pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay ________________________________ From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 08:29:08 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 06:29:08 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , , , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , , , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , , , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , , , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , , , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , , , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , ,,, , , ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , ,,, , , , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , , , , , , Message-ID: > As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Exchanging Modula-3 for C has tremendously improved the maintainability, portability, correctness, size-in-source, and correctness-by-inspection of the system. Debuggability has improved as well, though hopefully that will "even out" -- hopefully Modula-3 code will become just as debuggable with stock gdb, and on systems without stabs support. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I?m not advocating we ?throw the baby out with the bath water?, but if the ?fork-and-do-more? pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 22 18:25:36 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 22 Apr 2011 09:25:36 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Message-ID: <20110422162536.E63631A2078@async.async.caltech.edu> Jay K writes: >--_f508cc8d-d8d9-47ec-89e0-fcca258f1fc1_ >Content-Type: text/plain; charset="Windows-1252" >Content-Transfer-Encoding: quoted-printable > > > > As for the use of C vs. Modula-3=2C it obvious that you and I have a dif= >ference of opinion. >Exchanging Modula-3 for C has tremendously improved the maintainability=2C = >portability=2C correctness=2C size-in-source=2C and correctness-by-inspecti= >on of the system. >Debuggability has improved as well=2C though hopefully that will "even out"= > -- hopefully Modula-3 code will become just as debuggable with stock gdb= >=2C and on systems without stabs support. > > - Jay In theory you are right. In practice, almost ten years ago now others at Caltech and I started to try to switch from PM3 to CM3 since it was obvious it was CM3 that was going to be maintained in future. The process still isn't over, as my frequent postings to this list bear witness to. My perspective is really very simple. I've been using Modula-3 as a tool to solve almost all programming problems I've had since about 1999, when I gave up C due to my inability to write a correct C program. I'm a user of the system and only reluctantly a peripheral maintainer. A few of my observations as a user are the following. -- Threads work perfectly in PM3. In CM3 it's very much ??? -- It took five years to get TEXT working acceptably (as in, no stack overflows!) in CM3. Now seems more or less OK, but Text.Hash is still possibly an order of magnitude slower than . -- I have no debugger that works for CM3. m3gdb is *very useful* for the old PM3. No it doesn't work perfectly but you can set breakpoints and print variables but that's what at least I use a debugger for. -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE are very slow (like orders of magnitude slower than for PM3). I looked at it once and realized that someone had walked all over RT0 with very heavy feet. Not a pretty sight, on my (long) to-do list. -- m3tk is broken owing to the introduction of LONGINT. This is not a peripheral issue when a lot of the code you produce is generated from other code (a process that improves programming productivity manyfold---when it works). As with all the other items on this list, m3tk has no trouble parsing the Modula-3 accepted by PM3. -- other LONGINT problems... -- It's taken me literally months of struggle to get my application working with CM3. The old PM3-compiled one traded half a billion dollars worth of stocks last year without a single glitch. The CM3-compiled version has been hanging every day except the last (so hopefully it's OK now...???) I'm literally losing sleep over this because it's running during European business hours. Only reason I got sleep last night is because it's Good Friday. Now I'm not blaming anyone in particular for these problems. In fact I think they mostly come from things that were done at Critical Mass, Inc. However there is a common thread. Just about all the issues I've run into have had *something* to do with someone's attempt to make Modula-3 more of "all things to all people" than it was. Its main strength was and is that it is a programming language with almost equal expressibility (both at the machine and data-structure level) as C++ with a *fifty* page definition in reasonably clear English---that's just a few pages longer than R5RS and in a bigger font to boot. Portability, library support, multicore use, comprehensive base datatypes, ... all those things are nice but not part of the "core mission" of the language. It makes sense to try to expand out and be all things to all people if you have the budget and development team for it. As it is now, all the improvements have done is threaten the survival of the programming language for any use whatsoever. Here's an experiment anyone on this list can try. Find a friend who doesn't use Modula-3 but is a computer programmer, software engineer, etc. Ask him to install CM3 and configure it so that the thread testing program works. See how long it takes before he gives up. Mika From jay.krell at cornell.edu Fri Apr 22 21:25:42 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 19:25:42 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: <20110422162536.E63631A2078@async.async.caltech.edu> References: <20110422162536.E63631A2078@async.async.caltech.edu> Message-ID: Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 23:02:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 22:02:28 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: Message-ID: <927697.40402.qm@web29714.mail.ird.yahoo.com> Hi all: if the problems were the thread package then we will be easyly solved for us, but instead the question how much we owe to each platforms depednence part, e.g: Cvsup was created with Pm3 and/or Ezm3, Pm3 with Linux almost perfect coverage of all source of incompatibilities and so, DEC SRC was make it freely available in any site without too much effort, though by doing it In Modula-3 made almost hard to erase fame of hard thing to do. Then Digital split ended with this efforts going on and meanwhile DEC SRC was being distributed as source compatible with pm3, or with cm3 even. Later Cm3 became an open product and everyone kept their objectives so don't think is easy to do that. Instead, there should be an easy way I Think to recover to be in that point. I mean: CM3 -> DEC SRC, DEC SRC -> PM3, and PM3 -> EZM3 -> Cvsup & Spin. I think there should be enough number of similar or almost equal modules same, so most of the time is relinking stuff. I think every one hast its own rights to believe the things they believe. I think what my old M3 Professor told me once before retired, Modula-3 didn't gain more users because the lack of an IDE, today there is not reason to say that, instead the market has grown a lot times faster then why should I believe it doesn't hold true (why we don't have Cm3 running a Cvsup server, why Cm3 is not running a Spin community, why Cm3 is not hosting several projects in web platforms, why Cm3 is not running Obliq3D stuff, and I could say more but but we don't want to be there I guess this is enough too much or little I don't know, we are not there still, Mika's projects, DEC SRC missing packages, zeus3D, etc). Let me know your thoughts and thanks in advance --- El vie, 22/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 22 de abril, 2011 14:25 Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:14:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:14:49 +0000 Subject: [M3devel] pthread_atfork In-Reply-To: References: Message-ID: This is in now. Mika, can you try user threads again, and this time remove -pthread/-pthreads/-lpthread from your config files? Really, if user threads are to be supported, I wish they were easier to use -- no need to rebuild m3core or anything else, and specifiable in link or runtime commads. More details below: From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: pthread_atfork Date: Tue, 19 Apr 2011 06:07:01 +0000 I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: ... manually for userthreads: ... take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 [Jay April 23] done: implemented in C, locking via Disable/EnableSwitching, ok? And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. .. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible [Jay April 23] instead, just not provided This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. [Jay April 23] still a major concern in need of significant investigation - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:19:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:19:25 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? Message-ID: I've asked before. I'm asking again. I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. @M3pthreads @M3userthreads or somesuch. I'd also like to be able to switch the default via a line in a m3makefile that builds a program. I know roughly what implementing this would look like. I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. I'll have to reread. Whatever problem it has, this change probably would too. I'm willing to implement this myself. I'm not asking for anyone else to do it. I understand it makes LOCK harder to efficiently inline. If/when LOCK becomes partly inlined, provide a switch? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sat Apr 23 21:04:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:04:16 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: <13CE2151-2CF8-4773-B32F-91E753256DAA@cs.purdue.edu> The problem is that you are forced to link with pthreads for the whole executable which may cause problems. Not sure why though. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From hosking at cs.purdue.edu Sat Apr 23 21:06:22 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:06:22 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From dabenavidesd at yahoo.es Sun Apr 24 19:51:57 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 18:51:57 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: Message-ID: <423962.44965.qm@web29707.mail.ird.yahoo.com> Hi all: In ref [2] it is claimed implemented stack walking exception handling for Modula-3 in SPINE using GCC 2.96 RT. Can you watch it? http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.8017 As also the strind schedulling for the SMP CPU Thanks in advance [1] D. Gough, S. Drew, K. J. Gough, and J. Ledermann, ?Implementing Zero Overhead Exception Handling,? 1995. [2] M. E. Fiuczynski, ?Safe and efficient resource sharing in component-based systems,? University of Washington, 2004. --- El s?b, 23/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: "Jay K" > CC: "m3devel" > Fecha: s?bado, 23 de abril, 2011 14:06 > Good point about lock. I really think > this is the showstopper because compiled code will be fo a > particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K > wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and > pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a > line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look > like. > > > > > > I am a bit confused as to what problems > -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would > too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently > inline. > > If/when LOCK becomes partly inlined, provide a > switch? > > > > > > Thank you, > > - Jay > > > From dabenavidesd at yahoo.es Sun Apr 24 22:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 21:47:28 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ Message-ID: <319537.44680.qm@web29718.mail.ird.yahoo.com> Hi all: using google found this ftp directory (Modula2+ VAX): ftp://ftp.cisco.com/pub/rfc/DEC If DEC guys allowed to read and copy this is because they worked not just its products, but also their specifications, would be nice to see the Time modules explained in: /TIME_V1_0_0.PS If it's of enterprise and historical (academic) interest language could gain recognition from that, I mean they founded several open source projects back in time, which became later de-facto standards, it would be nice to see if this is of enough interest of their internal sources (can't forget Modula-3 was just one of them). They further developed Modula2+ into Modula2+Epsilon, which could be said to be a Modula3 in house brother if so. Furthermore there is a private source code library of internal design/layout chips components, later things could show they did watch to continue some of their work of prior language(s) or of its younger brother. See: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 Anyway this might be seen as a history on programming languages PhD project or so and computers if so (my CS professor told DEC machines were able to be specified to some large extent by its consumers and it seems by their producers too e.g like VAXstations). I never had the opportunity of doing something like that, perhaps computers have change, or like most people has changed, interesting ... Thanks in advance From dabenavidesd at yahoo.es Mon Apr 25 00:54:26 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 23:54:26 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ In-Reply-To: <319537.44680.qm@web29718.mail.ird.yahoo.com> Message-ID: <34227.88904.qm@web29715.mail.ird.yahoo.com> Hi all: fortunately they had recognition was made by some sue, but for an issue; the claim they created a thing for the development of it: http://www.google.com/patents?id=PY0kAAAAEBAJ&zoom=4&pg=PA1#v=onepage&q&f=false Which is nice given that a patent is hold means that somehow everybody agrees with that I guess, but who *may not* be affected by it, exactly the counterexample of just another read my lips example :) of patents. Thanks in advance --- El dom, 24/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: [M3devel] Architecture descriptions of software/hardware in Modula-2+ > Para: m3devel at elegosoft.com > Fecha: domingo, 24 de abril, 2011 15:47 > Hi all: > using google found this ftp directory (Modula2+ VAX): > > ftp://ftp.cisco.com/pub/rfc/DEC > > If DEC guys allowed to read and copy this is because they > worked not just its products, but also their specifications, > would be nice to see the Time modules explained in: > /TIME_V1_0_0.PS > > If it's of enterprise and historical (academic) interest > language could gain recognition from that, I mean they > founded several open source projects back in time, which > became later de-facto standards, it would be nice to see if > this is of enough interest of their internal sources (can't > forget Modula-3 was just one of them). > > They further developed Modula2+ into Modula2+Epsilon, which > could be said to be a Modula3 in house brother if so. > > Furthermore there is a private source code library of > internal design/layout chips components, later things could > show they did watch to continue some of their work of prior > language(s) or of its younger brother. See: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 > > Anyway this might be seen as a history on programming > languages PhD project or so and computers if so (my CS > professor told DEC machines were able to be specified to > some large extent by its consumers and it seems by their > producers too e.g like VAXstations). I never had the > opportunity of doing something like that, perhaps computers > have change, or like most people has changed, interesting > ... > > Thanks in advance > From rodney_bates at lcwb.coop Tue Apr 26 18:28:34 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 11:28:34 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <20110419115106.7E6A31A207A@async.async.caltech.edu> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> Message-ID: <4DB6F2B2.3090905@lcwb.coop> I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry more quickly than I can. On 04/19/2011 06:51 AM, Mika Nystrom wrote: > Hello m3devel, > > If anyone knows how to fix the following trivially, please go ahead. > > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal > > (errors reported by m3tk). > > The offending line is: > > INTERFACE BasicCtypes; > > IMPORT Word, Long; > > TYPE > (* the four signed integer types *) > signed_char = [-16_7f-1 .. 16_7f]; > short_int = [-16_7fff-1 .. 16_7fff]; > int = [-16_7fffffff-1 .. 16_7fffffff]; > long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; > long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) > > (* the four unsigned integer types *) > unsigned_char = [16_0 .. 16_ff]; > unsigned_short_int = [16_0 .. 16_ffff]; > unsigned_int = [16_0 .. 16_ffffffff]; > unsigned_long_int = Word.T; > unsigned_long_long = Long.T; > > (* the three floating types *) > float = REAL; > double = LONGREAL; > long_double = EXTENDED; > > (* char *) > char = signed_char; > > END BasicCtypes. > > I'll look at it myself, but not immediately (no time). Workaround for now. > > Mika > From jay.krell at cornell.edu Wed Apr 27 09:05:03 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 27 Apr 2011 07:05:03 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: About inlining LOCK I was more echoing your past sentiment than giving my own, and I provided a mitigation: a) specifically the suggestion that the switchability be an option; it'd probably be on by default but people that want to squeeze out the extra performance could enable the inlining and lose the switchability About the need to link to pthread. Good point. I hadn't thought of that. BUT two points in possible rebuttal (not that is necessary an argument! more like a conversation) I'm sure that on some platforms there is no distinction. No separate broken libc.so and working libpthread.so. But yes, I realize the recent talk was about Linux, apparently it is broken. Solaris is probably better. Others I don't know, I'd have to check. Specifically, there should not be two separate fork implementations. Or two separate malloc implementations. The one and only fork must know about pthread_atfork. The one and only malloc must be thread safe, with respect to pthreads. (but not necessarily Modula-3 user-threads, that is our own problem). If that means fork and pthread_atfork have to be in the same .so, and/or that malloc and pthread_* have to be in the same .so, both of which are easy to believe are true, so be it. I shouldn't get one malloc via cc foo.c and a different one via cc foo.c -lpthread. The rare single-threaded process isn't worth optimizing. Isn't worth the headache. As well, though..and this isn't trivial, but I wonder if we should dispense with "static dynamic" linking, where we just call fork, malloc, etc., but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. That might address some of this libc vs. libpthread problem? But then the ABI rather than the source interface becomes needed to be known. Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? Hm. No. Nevermind. What is the problem again with using -pthread on Linux with user threads? So, maybe it pulls in an extra .so and has some small startup cost. Surely it should work? I'll have to dig in to the email and reproduce the bug. I'm of a mind now to remove the recent pthread_atfork stuff. But first I or someone has to understand the problem I was solving. Right now, I don't. The LOCK inlining problem seems very hypothetical right now. We don't have such a feature. Will we really ever? Is pthread_lockmutex partially inlined in C? Or have pthreads generally been implemented well enough that everyone is satisfied with no inlining? Or does Modula-3 sort of promote threads as lighter weight than they are in typical practise? And ditto for locking? You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, I suspect all the commercial Unices, had already presented and reasonably implemented the ideal threading interface, but all the free Unices were still holding out and either not providing pthreads or not providing them well.. at the time Modula-3 was slightly ahead of its time, and one could imagine not using the underlying platform directly because it was too slow or not full featured enough. But hasn't everything just about caught up by now (except OpenBSD with its user pthreads...) (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 years ago circa Solaris 2.10 was pretty modern..) That is -- are pthreads really worth us optimizing beyond? Don't all the C and C++ programmers and systems providers get it by now and have provided almost exactly what we need? Sorry, I'm rambling... Later, - Jay > From: hosking at cs.purdue.edu > Date: Sat, 23 Apr 2011 15:06:22 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look like. > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently inline. > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > Thank you, > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Apr 26 21:50:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 14:50:08 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB6F2B2.3090905@lcwb.coop> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> <4DB6F2B2.3090905@lcwb.coop> Message-ID: <4DB721F0.3010101@lcwb.coop> On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this > bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry > more quickly than I can. > P.S.: Hats off to whoever adapted m3tk for LONGINT. There are places _everywhere_ that needed modification. The whole thing is about as pedantic as code gets. Layers upon layers of linguistic abstractions that do mostly zero semantic abstraction. I traced two declarations through 10 levels each of referrals to other interfaces, all but one of them just renames with the same simple name and different qualifying interfaces, before finally getting to something with some meat. The whole thing smells very strongly of the Ada/Diana culture of ASTs. > On 04/19/2011 06:51 AM, Mika Nystrom wrote: >> Hello m3devel, >> >> If anyone knows how to fix the following trivially, please go ahead. >> >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal >> >> (errors reported by m3tk). >> >> The offending line is: >> >> INTERFACE BasicCtypes; >> >> IMPORT Word, Long; >> >> TYPE >> (* the four signed integer types *) >> signed_char = [-16_7f-1 .. 16_7f]; >> short_int = [-16_7fff-1 .. 16_7fff]; >> int = [-16_7fffffff-1 .. 16_7fffffff]; >> long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; >> long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) >> >> (* the four unsigned integer types *) >> unsigned_char = [16_0 .. 16_ff]; >> unsigned_short_int = [16_0 .. 16_ffff]; >> unsigned_int = [16_0 .. 16_ffffffff]; >> unsigned_long_int = Word.T; >> unsigned_long_long = Long.T; >> >> (* the three floating types *) >> float = REAL; >> double = LONGREAL; >> long_double = EXTENDED; >> >> (* char *) >> char = signed_char; >> >> END BasicCtypes. >> >> I'll look at it myself, but not immediately (no time). Workaround for now. >> >> Mika >> > From hosking at cs.purdue.edu Wed Apr 27 17:20:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 27 Apr 2011 11:20:15 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: On Apr 27, 2011, at 3:05 AM, Jay K wrote: > About inlining LOCK I was more echoing your past sentiment > than giving my own, and I provided a mitigation: > a) specifically the suggestion that the switchability be an option; > it'd probably be on by default but people that want to squeeze > out the extra performance could enable the inlining > and lose the switchability It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > About the need to link to pthread. > Good point. I hadn't thought of that. > BUT two points in possible rebuttal (not that is necessary > an argument! more like a conversation) > I'm sure that on some platforms there is no distinction. > No separate broken libc.so and working libpthread.so. > But yes, I realize the recent talk was about Linux, apparently it is broken. > Solaris is probably better. > Others I don't know, I'd have to check. > > > Specifically, there should not be two separate fork implementations. > Or two separate malloc implementations. > > > The one and only fork must know about pthread_atfork. > The one and only malloc must be thread safe, with respect to pthreads. > (but not necessarily Modula-3 user-threads, that is our own problem). > > > If that means fork and pthread_atfork have to be in the same .so, > and/or that malloc and pthread_* have to be in the same .so, both > of which are easy to believe are true, so be it. > > > I shouldn't get one malloc via cc foo.c and a different > one via cc foo.c -lpthread. > The rare single-threaded process isn't worth optimizing. > Isn't worth the headache. > > > As well, though..and this isn't trivial, but I wonder if we should > dispense with "static dynamic" linking, where we just call fork, malloc, etc., > but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. > That might address some of this libc vs. libpthread problem? > But then the ABI rather than the source interface becomes needed to be known. > Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? > > > Hm. No. Nevermind. > What is the problem again with using -pthread on Linux with user threads? > So, maybe it pulls in an extra .so and has some small startup cost. > Surely it should work? > I'll have to dig in to the email and reproduce the bug. > I'm of a mind now to remove the recent pthread_atfork stuff. > But first I or someone has to understand the problem I was solving. > Right now, I don't. > > > The LOCK inlining problem seems very hypothetical right now. > We don't have such a feature. Will we really ever? > > > Is pthread_lockmutex partially inlined in C? > Or have pthreads generally been implemented well enough that everyone is satisfied > with no inlining? > > > Or does Modula-3 sort of promote threads as lighter weight than they are > in typical practise? And ditto for locking? > > > You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, > I suspect all the commercial Unices, had already presented and reasonably > implemented the ideal threading interface, but all the free Unices were still > holding out and either not providing pthreads or not providing them well.. > at the time Modula-3 was slightly ahead of its time, and one could imagine > not using the underlying platform directly because it was too slow > or not full featured enough. But hasn't everything just about caught up by > now (except OpenBSD with its user pthreads...) > (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 > years ago circa Solaris 2.10 was pretty modern..) > > > That is -- are pthreads really worth us optimizing beyond? > Don't all the C and C++ programmers and systems providers get it by now > and have provided almost exactly what we need? > > > Sorry, I'm rambling... > > > Later, > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Sat, 23 Apr 2011 15:06:22 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > > > Sent from my iPhone > > > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > > > I've asked before. I'm asking again. > > > > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > > @M3pthreads @M3userthreads > > > or somesuch. > > > > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > > > > I know roughly what implementing this would look like. > > > > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > > I'll have to reread. > > > Whatever problem it has, this change probably would too. > > > > > > > > > I'm willing to implement this myself. > > > I'm not asking for anyone else to do it. > > > > > > > > > I understand it makes LOCK harder to efficiently inline. > > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > > > > Thank you, > > > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 27 23:53:31 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 27 Apr 2011 22:53:31 +0100 (BST) Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB721F0.3010101@lcwb.coop> Message-ID: <689426.54766.qm@web29719.mail.ird.yahoo.com> Hi all: how does the arithmetic of LONGREAL affects that of the LONGINT, since both depend on that every BITSIZE should have comparable sizes, shouldn't it? I mean opaque types must be declared in both size LONGINT type and/or INTEGER size(s) to be able to be instantiated from memory in FPU in both cases, I know x87 is bigger size for its integers, besides what happens if the system doesn't provide that, should one consider implementing all (the) type conversions? Here: http://modula3.elegosoft.com/cm3/doc/reference/real-intf.html I think this one lets open the door for such conversions if I may say so: http://modula3.elegosoft.com/cm3/doc/reference/arithmetic.html I know one of the annoyances of old x86 FPU was to make any format conversion to CPU arithmetic thus converting from floats was very onerous in terms of losing your maximized precision in one assignment and bigger performance issues which made impractical for both sides, since initially every cycle was different and so, but I guess all programmer must be aware of that, but if the systems provide better performance like in any reasonable bigger machine, why not? I will give a check of decndumber to seek if they do that at least they had elegant exception handling Thanks in advance --- El mar, 26/4/11, Rodney M. Bates escribi?: > De: Rodney M. Bates > Asunto: Re: [M3devel] LONGINT -> more bugs in m3tk > Para: m3devel at elegosoft.com > Fecha: martes, 26 de abril, 2011 14:50 > > > On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > > I just checked in a patch tp M3CExpValue.m3 in m3tk, > in the head, which I believe will fix this > > bug. I compiled it, but have not tested it, as I am > sure Mika has this case to retry > > more quickly than I can. > > > > P.S.: Hats off to whoever adapted m3tk for > LONGINT. There are places _everywhere_ that needed > modification. The whole thing is about as pedantic as > code gets. Layers upon layers of > linguistic abstractions that do mostly zero semantic > abstraction. I traced two declarations > through 10 levels each of referrals to other interfaces, > all but one of them just renames > with the same simple name and different qualifying > interfaces, before finally getting to > something with some meat. > > The whole thing smells very strongly of the Ada/Diana > culture of ASTs. > > > On 04/19/2011 06:51 AM, Mika Nystrom wrote: > >> Hello m3devel, > >> > >> If anyone knows how to fix the following > trivially, please go ahead. > >> > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,45: type error in arguments to binary operator > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,25: bad numeric literal > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,51: bad numeric literal > >> > >> (errors reported by m3tk). > >> > >> The offending line is: > >> > >> INTERFACE BasicCtypes; > >> > >> IMPORT Word, Long; > >> > >> TYPE > >> (* the four signed integer types *) > >> signed_char = [-16_7f-1 .. 16_7f]; > >> short_int = [-16_7fff-1 .. 16_7fff]; > >> int = [-16_7fffffff-1 .. 16_7fffffff]; > >> long_int = [-16_7fffffffffffffff -1 .. > 16_7fffffffffffffff ]; > >> long_long = [-16_7fffffffffffffffL-1L .. > 16_7fffffffffffffffL]; (** HERE **) > >> > >> (* the four unsigned integer types *) > >> unsigned_char = [16_0 .. 16_ff]; > >> unsigned_short_int = [16_0 .. 16_ffff]; > >> unsigned_int = [16_0 .. 16_ffffffff]; > >> unsigned_long_int = Word.T; > >> unsigned_long_long = Long.T; > >> > >> (* the three floating types *) > >> float = REAL; > >> double = LONGREAL; > >> long_double = EXTENDED; > >> > >> (* char *) > >> char = signed_char; > >> > >> END BasicCtypes. > >> > >> I'll look at it myself, but not immediately (no > time). Workaround for now. > >> > >> Mika > >> > > > From wagner at elegosoft.com Thu Apr 28 09:29:14 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 28 Apr 2011 09:29:14 +0200 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Quoting Tony Hosking : > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > >> About inlining LOCK I was more echoing your past sentiment >> than giving my own, and I provided a mitigation: >> a) specifically the suggestion that the switchability be an option; >> it'd probably be on by default but people that want to squeeze >> out the extra performance could enable the inlining >> and lose the switchability > > It is expected that LOCK will be implemented using CAS (and > equivalents) in the very near future. I have a student working on > it. This will be incompatible with the current user-threads > implementation of LOCK. It will be too painful to support both > using a compiler switch. Couldn't we change the existing user threads to use CAS for locks, too, in order to gain runtime switching of thread implementations? I have neither thought much about it nor looked at the implementation in recent years, so I hope it's no stupid question. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 28 15:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 28 Apr 2011 14:47:28 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: <212562.68766.qm@web29706.mail.ird.yahoo.com> Hi all: yes, that would be awesome ! :) instead of having the threads question of select one be able to switch, we can have sort of multiprocessor implementation capabilities and/or simple multi threading both for free in the RT like for special SPIN, however you should have migration capabilities between both address spaces and sort of a RT executive to control threads of both spaces: http://courses.cs.vt.edu/~cs5204/fall03/Papers/Migration/mathiske95migrating.pdf Similarly it would be good to see how much hard would it be to set distributed applications to get RT easily distributed in m3core, libm3, etc: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.9178 for something like the compiler, type checker, etc So any thread could live in a different machine in desired space --- El jue, 28/4/11, Olaf Wagner escribi?: > De: Olaf Wagner > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: m3devel at elegosoft.com > Fecha: jueves, 28 de abril, 2011 02:29 > Quoting Tony Hosking : > > > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > > > >> About inlining LOCK I was more echoing your past > sentiment > >> than giving my own, and I provided a mitigation: > >> a) specifically the suggestion that the > switchability be an option; > >> it'd probably be on by default > but people that want to squeeze > >> out the extra performance could > enable the inlining > >> and lose the switchability > > > > It is expected that LOCK will be implemented using CAS > (and equivalents) in the very near future. I > have a student working on it. This will be > incompatible with the current user-threads > implementation of LOCK. It will be too painful to > support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for > locks, too, > in order to gain runtime switching of thread > implementations? > I have neither thought much about it nor looked at the > implementation > in recent years, so I hope it's no stupid question. > > Olaf > --Olaf Wagner -- elego Software Solutions GmbH > > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 > 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | > USt-IdNr: DE163214194 > > From hosking at cs.purdue.edu Thu Apr 28 16:22:36 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 28 Apr 2011 10:22:36 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> References: , <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: On Apr 28, 2011, at 3:29 AM, Olaf Wagner wrote: > Quoting Tony Hosking : > >> On Apr 27, 2011, at 3:05 AM, Jay K wrote: >> >>> About inlining LOCK I was more echoing your past sentiment >>> than giving my own, and I provided a mitigation: >>> a) specifically the suggestion that the switchability be an option; >>> it'd probably be on by default but people that want to squeeze >>> out the extra performance could enable the inlining >>> and lose the switchability >> >> It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for locks, too, > in order to gain runtime switching of thread implementations? > I have neither thought much about it nor looked at the implementation > in recent years, so I hope it's no stupid question. Hmm. Yes, I suppose we could. It might degrade performance of the user-level (uniprocessor) threads implementation. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 > From rodney_bates at lcwb.coop Fri Apr 1 19:43:25 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Fri, 01 Apr 2011 12:43:25 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: <20110305180558.451421A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu> Message-ID: <4D960EBD.2090206@lcwb.coop> I have looked at this a bit. I do not get this symptom on LINUXLIBC6, using the Cartesian product of release/head for m3gdb and the compiler. However, I see a lot of broken stuff that used to work. In all the cases I have seen, it correlates to debugging code compiled by the head compiler, while the release and head versions of m3gdb make no difference to the symptoms in anything I have tried. I believe Jay had started updating the m3gdb in head to use a newer gdb. I am not sure how far that got. Its source code is vastly different, and most of the other language support in gdb has been ripped out. There is quite a lot of change in the stabs info generated by the two compiler versions. The existing way for a long time has been that this is constructed during the first pass over the input to cm3cg, before anything other than initial tree building is done. This is a very limited strategy, because cm3cg can change things later, and this will not be reflected in the stabs info. Debugging anything involving static links has been broken for a long time, because tree-nested.c in cm3cg does some very convoluted stuff with the way static ancestor activation records are accessed, and I have not been able to see an easy way to get stabs info to reflect it. I have thought for a long time that we really need to go to a version of dwarf for debug info. stabs is very limited, and there is a lot of info in the stabs output that is not really stabs, it's just kludgily coded values crammed inside various fields of stabs that the compiler and m3gdb both agree on (well, they used to.) Maybe the current head cm3cg can be fixed, but I know Jay has made big changes in where type information is actually developed, so it may not be worth it. After all the work I have done on this to get m3gdb to where it was, I am unenthusiastic about a bunch more work just to get back to where we were. I have long lists of improvements to make to m3gdb, and it's hard to go back that way. And, It's all in C :-(. If I put much effort into fixing m3gdb and its support in cm3cg at this point, it will probably be a rework to use dwarf. That is likely a big job too, but at least more rewarding and likely to lead to better m3gdb function in the end. For now, it you want a working m3gdb, you really have to use the release compiler. On 03/05/2011 12:05 PM, Mika Nystrom wrote: > Hi m3devel, > > I don't know what's happened here. m3gdb used to work, and work fairly > well (never perfectly, but...). Now I'm trying to debug a program and > I get the following: > > (m3gdb) break Main.m3:65 > Debug info for file "Main.mc" not in stabs format > (m3gdb) > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > systems too. > > Everything is from the CVS head. > > Main was compiled as follows, says cm3 -commands: > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ ci > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > I'm not even sure where to begin looking. Is it a configuration issue, > something broken in the compiler, or what? > > I fear m3gdb is essentially useless at the moment. > > Mika > From jay.krell at cornell.edu Sat Apr 2 03:26:39 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 01:26:39 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <4D960EBD.2090206@lcwb.coop> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: I really tried not to break anything in the wierdo stubs/m3gdb stuff. The work I have been doing to properly declare types is I believe disabled currently. As types become more correct, things start breaking. Either because the optimizer is more aggressive, or because more sanity checks can be done by the compiler, and we don't give it particularly good trees. In most things I've looked it, it's really not any mystery what is wrong with our trees btw. It's not really generally the fault of gcc being obscure or changing. Though perhaps to some extent that is true. Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not accessible from C source, and therefore bugs in them go undetected. It's more generally clearly not great stuff in cm3cg. The main bad thing is that field access is done by just adding to base pointers and casting away. Note that stabs is not supported on Darwin. Nor as I recall HP-UX/HPPA64. All the more reason to no do things this way. Note furthermore that discussion here is often confused. Really no particular debug-info generation code should be cm3cg. In our part -- m3cg/parse.c Neither stabs nor dwarf nor coff nor anything else. The right approach is to build an intermediate form with reasonably good types. The backend handles it from there, whatever debug-info format is requested on the command line. This also enables using stock/current gdb, instead of hacked up/old gdb. I made no visible progress upgrading our fork of gdb. I'd rather just have it go away. I understand, that custom information is being tunneled through to m3gdb. But that "custom information" is largely just basic type information, that any decent C compiler/debug-info/debugger can handle. Someone should investigate if we really pass stuff through that can't be well represented in gcc's type information. For example: sub ranges? sets? enums? packed? Some higher level description of "methods" (really -- whatever C++ does should suffice; if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) At some point the new type information was really breaking stuff so I think I disabled it. The system is kind of incomplete and fragile and the way forward is not always easy or possible in small steps. :( - Jay > Date: Fri, 1 Apr 2011 12:43:25 -0500 > From: rodney_bates at lcwb.coop > To: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > using the Cartesian product of release/head for m3gdb and the compiler. > > However, I see a lot of broken stuff that used to work. In all the cases > I have seen, it correlates to debugging code compiled by the head compiler, > while the release and head versions of m3gdb make no difference to the > symptoms in anything I have tried. > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > I am not sure how far that got. Its source code is vastly different, and > most of the other language support in gdb has been ripped out. > > There is quite a lot of change in the stabs info generated by the two > compiler versions. The existing way for a long time has been that this > is constructed during the first pass over the input to cm3cg, before > anything other than initial tree building is done. This is a very > limited strategy, because cm3cg can change things later, and this will > not be reflected in the stabs info. > > Debugging anything involving static links has been broken for a long time, > because tree-nested.c in cm3cg does some very convoluted stuff with the > way static ancestor activation records are accessed, and I have not been > able to see an easy way to get stabs info to reflect it. > > I have thought for a long time that we really need to go to a version > of dwarf for debug info. stabs is very limited, and there is a lot of > info in the stabs output that is not really stabs, it's just kludgily > coded values crammed inside various fields of stabs that the compiler > and m3gdb both agree on (well, they used to.) > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > changes in where type information is actually developed, so it may not > be worth it. After all the work I have done on this to get m3gdb to where > it was, I am unenthusiastic about a bunch more work just to get back to > where we were. I have long lists of improvements to make to m3gdb, and > it's hard to go back that way. And, It's all in C :-(. > > If I put much effort into fixing m3gdb and its support in cm3cg at this > point, it will probably be a rework to use dwarf. That is likely a big > job too, but at least more rewarding and likely to lead to better m3gdb > function in the end. > > For now, it you want a working m3gdb, you really have to use the release > compiler. > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > Hi m3devel, > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > well (never perfectly, but...). Now I'm trying to debug a program and > > I get the following: > > > > (m3gdb) break Main.m3:65 > > Debug info for file "Main.mc" not in stabs format > > (m3gdb) > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > systems too. > > > > Everything is from the CVS head. > > > > Main was compiled as follows, says cm3 -commands: > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika/t/ > ci > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > something broken in the compiler, or what? > > > > I fear m3gdb is essentially useless at the moment. > > > > Mika > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sat Apr 2 04:42:33 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 01 Apr 2011 19:42:33 -0700 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <20110402024233.0BEC11A2078@async.async.caltech.edu> Jay K writes: ... >But that "custom information" is largely just basic type information=2C tha= >t any decent >C compiler/debug-info/debugger can handle. >Someone should investigate if we really pass stuff through that can't be we= >ll represented >in gcc's type information. >For example: sub ranges? sets? enums? packed? Subranges, sets, and enums certainly used to be shown in a "friendly" way by m3gdb. Mika From jay.krell at cornell.edu Sat Apr 2 05:24:02 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 2 Apr 2011 03:24:02 +0000 Subject: [M3devel] m3gdb broken In-Reply-To: <20110402024233.0BEC11A2078@async.async.caltech.edu> References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. - Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Sat Apr 2 17:53:19 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sat, 2 Apr 2011 16:53:19 +0100 (BST) Subject: [M3devel] m3gdb broken In-Reply-To: Message-ID: <196546.21262.qm@web29708.mail.ird.yahoo.com> Hi all: well, all that said, is it possible, but how many checks would be needed to all those errors, I perhaps think the best tool at hand is but to run undependently of the platform ESC, at least to catch the most common RT errors for now, but gcc as a backend is enough optimizable and also enough conservative to catch at compile time warnings you see as possible M3CG errors, in Modula-3 safe code there are not explicity needed in ESC terms but in UNSAFE ones, so the most RTError you want to catch as m3gdb exceptions would be advisable in foreseeable future, perhaps making some of them implementable at ESC level, but to check for those that are that in the RTCollector, RTException and the RTThread specially perhaps simpler ones in ESC terms but important, and in the M3CG memory handling, exception, and threads too, obviously in the RTType, even if those are not safely checked RTError should have some correspondent there and that is it must be caught in the M3CG before it can also generate bad codes, if so, also possible improved UNSAFE RTTypeSRC information to accomplish that perhaps a better but not limited to type checking and exceptions handled of both safe and UNSAFE code. Thinking in what's on the market there are recent reports in Logic Analyzer of embedded software-hardware verification environments for circuit analysis, such kind of automation tests are deployable even at the try and so we may get that, although this Embedded software Arena, but even at that level, cost effective knowing of the logic synthesis theories errors they want to be rid of, that is, is more important to shape some circuit level checks before product is deployed since may things depended on that too. We could seemingly adopt that strategic objective too, detect most of the common errors in those areas, also a nice RTTypeSRC interface for unsafe type code patterns. Perhaps some of this work is already done implemented in Modula-3 VM backend Quest abstract machine, since? it has nice constructs and basically same stack architecture and machine but with additional ability to support for separating sound and unsound modules. I think in this way the margin of RT errors will be diminished by at least by some volume bounded by the amount of each type inference in the checker, specially some of the ones in RTTypeSRC information portion plus desired good to watch at least some code modularity level sound handled in such code, so some of them that must be catched and, something it isn't right now for M3CG will not, but just because of correspondent MODULEs so it can be translated sound and caught in better CG compiler time in other representation and then generate code to its RT binary structures perhaps some may want for that too. Also that could be generous in terms of debugging both Quest level and m3gdb source level. Thanks in advance --- El vie, 1/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] m3gdb broken Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 1 de abril, 2011 22:24 The question is, can they be well represented within gcc's usual trees/types/debug-info for good experience in stock gdb. Enums for example -- yes, I had them working. Subranges -- what should the experience be? For example, when I use debuggers, I use expression evaluation involving assignment to set values. Should the debugger catch attempts to assign out of range values? Or are subranges just integers with a certain number of bits and that is it and that suffices? I understand the runtime does more precise checks than just bit-width. sets -- I can see this being....are they just an integer or an array of integers? Or higher level? I can see this being poor...unless unless.. gcc maybe already had good support for Pascal? Maybe Ada has a similar feature? Ada does actually look like Modula-3 to a large extent! Consider also that on Darwin, none of this stuff works at all. So even sets as integers or arrays of integers would be much better than current. One has to read some of the gcc documentation, browse tree.h, tree.def, etc., learn what is well representable. One shouldn't just completely ignore the significant infrastructure and just tunnel all the information through in a custom way that stock gdb is completely ignorant of. ?- Jay > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] m3gdb broken > Date: Fri, 1 Apr 2011 19:42:33 -0700 > From: mika at async.caltech.edu > > Jay K writes: > ... > >But that "custom information" is largely just basic type information=2C tha= > >t any decent > >C compiler/debug-info/debugger can handle. > >Someone should investigate if we really pass stuff through that can't be we= > >ll represented > >in gcc's type information. > >For example: sub ranges? sets? enums? packed? > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > by m3gdb. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Sun Apr 3 03:28:18 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:28:18 -0700 Subject: [M3devel] floating point : problem with Extended.m3 Message-ID: <20110403012818.3560A1A2078@async.async.caltech.edu> Hello m3devel (mainly Tony), I believe I've found a problem with float/Common/Extended.m3. Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? That does not seem to me to be a safe assumption... in fact I don't think it's even safe to round up. EXTENDED could (and should) be 10 bytes wide on most machines. (I am aware that it is currently the same as LONGREAL, but...) Mika > UNSAFE MODULE Extended; > > IMPORT Word, Int32; > > PROCEDURE Equal(a, b: T): BOOLEAN = > BEGIN RETURN a = b END Equal; > > > CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); > > TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; > > PROCEDURE Hash(a: T): Word.T = > VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN > FOR i := 0 TO LAST(arr) DO > res := Word.Xor(res, Int32.Hash(arr[i])) > END (* FOR *); > RETURN res > END Hash; > > PROCEDURE Compare(a, b: T): [-1..1] = > BEGIN > IF a < b THEN RETURN -1 > ELSIF a > b THEN RETURN 1 > ELSE RETURN 0 > END (* IF *) > END Compare; From hosking at cs.purdue.edu Sun Apr 3 03:40:02 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 2 Apr 2011 21:40:02 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403012818.3560A1A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: Are you saying it is broken with EXTENDED=LONGREAL? That is the current implementation. This code is heritage. Not sure I understand all the implications. On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > Hello m3devel (mainly Tony), > > I believe I've found a problem with float/Common/Extended.m3. > > Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? > > That does not seem to me to be a safe assumption... in fact I don't > think it's even safe to round up. EXTENDED could (and should) be > 10 bytes wide on most machines. (I am aware that it is currently > the same as LONGREAL, but...) > > Mika > >> UNSAFE MODULE Extended; >> >> IMPORT Word, Int32; >> >> PROCEDURE Equal(a, b: T): BOOLEAN = >> BEGIN RETURN a = b END Equal; >> >> >> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >> >> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >> >> PROCEDURE Hash(a: T): Word.T = >> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >> FOR i := 0 TO LAST(arr) DO >> res := Word.Xor(res, Int32.Hash(arr[i])) >> END (* FOR *); >> RETURN res >> END Hash; >> >> PROCEDURE Compare(a, b: T): [-1..1] = >> BEGIN >> IF a < b THEN RETURN -1 >> ELSIF a > b THEN RETURN 1 >> ELSE RETURN 0 >> END (* IF *) >> END Compare; From mika at async.caltech.edu Sun Apr 3 03:52:08 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sat, 02 Apr 2011 18:52:08 -0700 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: References: <20110403012818.3560A1A2078@async.async.caltech.edu> Message-ID: <20110403015208.6B1511A2078@async.async.caltech.edu> No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 bits). I'm really just curious what has to be changed to make EXTENDED not be LONGREAL... probably many more things in the compiler? Mika Tony Hosking writes: >Are you saying it is broken with EXTENDED=LONGREAL? >That is the current implementation. >This code is heritage. Not sure I understand all the implications. > >On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: > >> Hello m3devel (mainly Tony), >> >> I believe I've found a problem with float/Common/Extended.m3. >> >> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >> >> That does not seem to me to be a safe assumption... in fact I don't >> think it's even safe to round up. EXTENDED could (and should) be >> 10 bytes wide on most machines. (I am aware that it is currently >> the same as LONGREAL, but...) >> >> Mika >> >>> UNSAFE MODULE Extended; >>> >>> IMPORT Word, Int32; >>> >>> PROCEDURE Equal(a, b: T): BOOLEAN = >>> BEGIN RETURN a = b END Equal; >>> >>> >>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>> >>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>> >>> PROCEDURE Hash(a: T): Word.T = >>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>> FOR i := 0 TO LAST(arr) DO >>> res := Word.Xor(res, Int32.Hash(arr[i])) >>> END (* FOR *); >>> RETURN res >>> END Hash; >>> >>> PROCEDURE Compare(a, b: T): [-1..1] = >>> BEGIN >>> IF a < b THEN RETURN -1 >>> ELSIF a > b THEN RETURN 1 >>> ELSE RETURN 0 >>> END (* IF *) >>> END Compare; From hosking at cs.purdue.edu Sun Apr 3 22:15:00 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 3 Apr 2011 16:15:00 -0400 Subject: [M3devel] floating point : problem with Extended.m3 In-Reply-To: <20110403015208.6B1511A2078@async.async.caltech.edu> References: <20110403012818.3560A1A2078@async.async.caltech.edu> <20110403015208.6B1511A2078@async.async.caltech.edu> Message-ID: <3523B817-7987-4CA1-8230-0DF99EE76CCE@cs.purdue.edu> On Apr 2, 2011, at 9:52 PM, Mika Nystrom wrote: > No it's fine with EXTENDED=LONGREAL (well as long as LONGREAL is 64 or 32 > bits). > > I'm really just curious what has to be changed to make EXTENDED not be > LONGREAL... probably many more things in the compiler? Yes, definitely. Compiler and run-time. > > Mika > > Tony Hosking writes: >> Are you saying it is broken with EXTENDED=LONGREAL? >> That is the current implementation. >> This code is heritage. Not sure I understand all the implications. >> >> On Apr 2, 2011, at 9:28 PM, Mika Nystrom wrote: >> >>> Hello m3devel (mainly Tony), >>> >>> I believe I've found a problem with float/Common/Extended.m3. >>> >>> Doesn't the code assume that EXTENDED is a multiple of 32 bits wide? >>> >>> That does not seem to me to be a safe assumption... in fact I don't >>> think it's even safe to round up. EXTENDED could (and should) be >>> 10 bytes wide on most machines. (I am aware that it is currently >>> the same as LONGREAL, but...) >>> >>> Mika >>> >>>> UNSAFE MODULE Extended; >>>> >>>> IMPORT Word, Int32; >>>> >>>> PROCEDURE Equal(a, b: T): BOOLEAN = >>>> BEGIN RETURN a = b END Equal; >>>> >>>> >>>> CONST Int32Sz = BYTESIZE(T) DIV BYTESIZE(Int32.T); >>>> >>>> TYPE Int32Arr = ARRAY [0..Int32Sz-1] OF Int32.T; >>>> >>>> PROCEDURE Hash(a: T): Word.T = >>>> VAR arr := LOOPHOLE(a, Int32Arr); res := 0; BEGIN >>>> FOR i := 0 TO LAST(arr) DO >>>> res := Word.Xor(res, Int32.Hash(arr[i])) >>>> END (* FOR *); >>>> RETURN res >>>> END Hash; >>>> >>>> PROCEDURE Compare(a, b: T): [-1..1] = >>>> BEGIN >>>> IF a < b THEN RETURN -1 >>>> ELSIF a > b THEN RETURN 1 >>>> ELSE RETURN 0 >>>> END (* IF *) >>>> END Compare; From rodney_bates at lcwb.coop Sun Apr 3 22:50:35 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:50:35 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> Message-ID: <4D98DD9B.9080003@lcwb.coop> On 04/01/2011 08:26 PM, Jay K wrote: > I really tried not to break anything in the wierdo stubs/m3gdb stuff. > > > The work I have been doing to properly declare types is I believe disabled currently. > > Hmm, I looked at a sample of stabs output for the same program, produced by the release and head compilers. In the head, all of the stabs type information is missing, including the type definitions and the references to them. This may be enough to explain all the breakage in m3gdb I have seen. I see another small change, but it is something that, I think, m3gdb already adapts to, and, as in the release version, was weird anyway. > As types become more correct, things start breaking. > Either because the optimizer is more aggressive, or because more sanity checks > can be done by the compiler, and we don't give it particularly good trees. > > > In most things I've looked it, it's really not any mystery what is wrong with our trees btw. > It's not really generally the fault of gcc being obscure or changing. > Though perhaps to some extent that is true. > Or, there is the rare case, like where Modula-3 uses div/mod primitives that are not > accessible from C source, and therefore bugs in them go undetected. > It's more generally clearly not great stuff in cm3cg. > The main bad thing is that field access is done by just adding to base pointers and casting away. > > > Note that stabs is not supported on Darwin. > Nor as I recall HP-UX/HPPA64. > All the more reason to no do things this way. > > > Note furthermore that discussion here is often confused. > Really no particular debug-info generation code should be cm3cg. > In our part -- m3cg/parse.c > Neither stabs nor dwarf nor coff nor anything else. > > > The right approach is to build an intermediate form with reasonably good types. > The backend handles it from there, whatever debug-info format is requested on the command line. > > > This also enables using stock/current gdb, instead of hacked up/old gdb. > > > I made no visible progress upgrading our fork of gdb. > I'd rather just have it go away. > > > I understand, that custom information is being tunneled through to m3gdb. > But that "custom information" is largely just basic type information, that any decent > C compiler/debug-info/debugger can handle. > Someone should investigate if we really pass stuff through that can't be well represented > in gcc's type information. > For example: sub ranges? sets? enums? packed? > Some higher level description of "methods" (really -- whatever C++ does should suffice; > if it exposes a vtable pointer in the debugger, so be it; if it doesn't, that's perhaps nice too.) > > > At some point the new type information was really breaking stuff so I think I disabled it. > The system is kind of incomplete and fragile and the way forward is not always easy > or possible in small steps. :( > > > - Jay > > > > Date: Fri, 1 Apr 2011 12:43:25 -0500 > > From: rodney_bates at lcwb.coop > > To: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > > > I have looked at this a bit. I do not get this symptom on LINUXLIBC6, > > using the Cartesian product of release/head for m3gdb and the compiler. > > > > However, I see a lot of broken stuff that used to work. In all the cases > > I have seen, it correlates to debugging code compiled by the head compiler, > > while the release and head versions of m3gdb make no difference to the > > symptoms in anything I have tried. > > > > I believe Jay had started updating the m3gdb in head to use a newer gdb. > > I am not sure how far that got. Its source code is vastly different, and > > most of the other language support in gdb has been ripped out. > > > > There is quite a lot of change in the stabs info generated by the two > > compiler versions. The existing way for a long time has been that this > > is constructed during the first pass over the input to cm3cg, before > > anything other than initial tree building is done. This is a very > > limited strategy, because cm3cg can change things later, and this will > > not be reflected in the stabs info. > > > > Debugging anything involving static links has been broken for a long time, > > because tree-nested.c in cm3cg does some very convoluted stuff with the > > way static ancestor activation records are accessed, and I have not been > > able to see an easy way to get stabs info to reflect it. > > > > I have thought for a long time that we really need to go to a version > > of dwarf for debug info. stabs is very limited, and there is a lot of > > info in the stabs output that is not really stabs, it's just kludgily > > coded values crammed inside various fields of stabs that the compiler > > and m3gdb both agree on (well, they used to.) > > > > Maybe the current head cm3cg can be fixed, but I know Jay has made big > > changes in where type information is actually developed, so it may not > > be worth it. After all the work I have done on this to get m3gdb to where > > it was, I am unenthusiastic about a bunch more work just to get back to > > where we were. I have long lists of improvements to make to m3gdb, and > > it's hard to go back that way. And, It's all in C :-(. > > > > If I put much effort into fixing m3gdb and its support in cm3cg at this > > point, it will probably be a rework to use dwarf. That is likely a big > > job too, but at least more rewarding and likely to lead to better m3gdb > > function in the end. > > > > For now, it you want a working m3gdb, you really have to use the release > > compiler. > > > > On 03/05/2011 12:05 PM, Mika Nystrom wrote: > > > Hi m3devel, > > > > > > I don't know what's happened here. m3gdb used to work, and work fairly > > > well (never perfectly, but...). Now I'm trying to debug a program and > > > I get the following: > > > > > > (m3gdb) break Main.m3:65 > > > Debug info for file "Main.mc" not in stabs format > > > (m3gdb) > > > > > > The system/architecture is AMD64_FREEBSD but I've seen this problem on other > > > systems too. > > > > > > Everything is from the CVS head. > > > > > > Main was compiled as follows, says cm3 -commands: > > > > > > gcc -gstabs+ -m64 -fPIC -z origin -Wl,--warn-common -Wl,-rpath,\$ORIGIN -Wl,-rpath,\$ORIGIN/../lib -Wl,--fatal-warnings -z now -Bsymbolic -L/usr/local/lib -o sstubgen _m3main.o Main.mo /home/mika/t/mscheme/sstubgen/AMD64_FREEBSD/libsstubgen.a /home/mika/t/mscheme/schemereadline/AMD64_FREEBSD/libschemereadline.a /home/mika/t/mscheme/schemesig/AMD64_FREEBSD/libschemesig.a /home/mika/t/calarm/m3readline/AMD64_FREEBSD/libm3readline.a /home/mika/t/calarm/sx/AMD64_FREEBSD/libsx.a /home/mika/t/mscheme/modula3scheme/AMD64_FREEBSD/libmodula3scheme.a /home/mika/t/rdwr/AMD64_FREEBSD/librdwr.a /home/mika/t/mscheme/AMD64_FREEBSD/libmscheme.a /home/mika/t/cit_util/AMD64_FREEBSD/libcit_util.a /usr/local/cm3/pkg/netobj/AMD64_FREEBSD/libm3netobj.a /usr/local/cm3/pkg/tcp/AMD64_FREEBSD/libm3tcp.a /usr/local/cm3/pkg/patternmatching/AMD64_FREEBSD/libpatternmatching.a /home/mika/t/mscheme/scheme-lib/AMD64_FREEBSD/libscheme-lib.a /usr/local/cm3/pkg/libbuf/AMD64_FREEBSD/liblibbuf.a /home/mika /t/ > > ci > > > t_common/AMD64_FREEBSD/libcit_common.a /usr/local/cm3/pkg/set/AMD64_FREEBSD/libset.a /home/mika/t/rdwrreset/AMD64_FREEBSD/librdwrreset.a /usr/local/cm3/pkg/m3tk/AMD64_FREEBSD/libm3tk.a /usr/local/cm3/pkg/m3tk-misc/AMD64_FREEBSD/libm3tk-misc.a /usr/local/cm3/pkg/libm3/AMD64_FREEBSD/libm3.a /usr/local/cm3/pkg/m3core/AMD64_FREEBSD/libm3core.a -lm -lcrypt -lssl -L/usr/local/lib -lintl -lreadline > > > > > > I'm not even sure where to begin looking. Is it a configuration issue, > > > something broken in the compiler, or what? > > > > > > I fear m3gdb is essentially useless at the moment. > > > > > > Mika > > > From rodney_bates at lcwb.coop Sun Apr 3 22:46:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Sun, 03 Apr 2011 15:46:08 -0500 Subject: [M3devel] m3gdb broken In-Reply-To: References: <20110305180558.451421A2078@async.async.caltech.edu>, <4D960EBD.2090206@lcwb.coop> , <20110402024233.0BEC11A2078@async.async.caltech.edu> Message-ID: <4D98DC90.3090900@lcwb.coop> On 04/01/2011 10:24 PM, Jay K wrote: > The question is, can they be well represented within gcc's usual trees/types/debug-info > for good experience in stock gdb. > Enums for example -- yes, I had them working. > > Subranges -- what should the experience be? > For example, when I use debuggers, I use expression evaluation involving assignment > to set values. Should the debugger catch attempts to assign out of range values? > Or are subranges just integers with a certain number of bits and that is it and that suffices? > I understand the runtime does more precise checks than just bit-width. > > sets -- I can see this being....are they just an integer or an array of integers? > Or higher level? I can see this being poor...unless unless.. gcc maybe already > had good support for Pascal? Maybe Ada has a similar feature? > Ada does actually look like Modula-3 to a large extent! > > Consider also that on Darwin, none of this stuff works at all. > So even sets as integers or arrays of integers would be much better than current. > > One has to read some of the gcc documentation, browse tree.h, tree.def, etc., > learn what is well representable. > > One shouldn't just completely ignore the significant infrastructure and just > tunnel all the information through in a custom way that stock gdb is completely ignorant of. > There is no way stock gdb can possibly provide anything remotely similar to the support it gives other languages, just by providing it with better debug info. Every one of the languages already supported by stock gdb (C, C++, Objective C, Pascal, Ada, Java, Fortran, and Scheme come to mind) have significant amounts of language-specific code, comparable to what m3gdb adds for Modula-3 support. Some categories of language-specific code include: name demanglers, expression parsers, expression evaluators, identifier lookups, characteristics of builtin types, and value printers. You can find anecdotal cases of types, operators, and their syntax that are equivalent to another language, but this is highly incomplete. Value notations also differ. Semantics of same-looking operators can be significantly different. The dot is a good example. Not even Pascal nor Ada are the same as Modula-3. Interpretation of memory is also highly language dependent. There are significant differences in runtime representation of seemingly equivalent types and other stuff. Modula-3's TEXT, for example, is represented unlike any of the other languages. Heap objects have distinct ways of showing allocation status, GC info, open array sizes for differing dimensionalities, and just the plain allocated object type. The method dispatching mechanism is distinct. Actually, this stuff is really compiler-dependent, not just language-dependent. There are significant differences between pm3/friends and cm3, and occasional smaller differences between versions thereof. m3gdb recognizes many of these and adapts. Ditto for some of the differences in thread and GC implementations. Some procedures have hidden parameters that show up in the type info, but do not appear in source code calls. The rules for these are fairly conditional, on both the procedure signature and on the target OS. It is difficult for a debugger user to even know when, where, and of what type these need to be to supply them explicitly to make such a call, and as I recall, in some cases impossible to supply some of the needed values, even with full knowledge. In contrast, m3gdb allows typing procedure and method calls as they would appear in source code, and supplies the hidden parameters as needed. Assigning procedures to variables and parameters and calling through them also have unique runtime representations. As of the gdb the current m3gdb is based on, no stock gdb language handles up-level addressing. Much of this is working in the current m3gdb, or was until recently. Most of what isn't is documented as work to do. > > - Jay > > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] m3gdb broken > > Date: Fri, 1 Apr 2011 19:42:33 -0700 > > From: mika at async.caltech.edu > > > > Jay K writes: > > ... > > >But that "custom information" is largely just basic type information=2C tha= > > >t any decent > > >C compiler/debug-info/debugger can handle. > > >Someone should investigate if we really pass stuff through that can't be we= > > >ll represented > > >in gcc's type information. > > >For example: sub ranges? sets? enums? packed? > > > > Subranges, sets, and enums certainly used to be shown in a "friendly" way > > by m3gdb. > > > > Mika From dragisha at m3w.org Thu Apr 7 16:34:20 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Thu, 7 Apr 2011 16:34:20 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Message-ID: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> I am getting this, when my program does a Thread.Fork. *** *** runtime error: *** <*ASSERT*> failed. *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 *** Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... TIA, dd From hosking at cs.purdue.edu Thu Apr 7 17:41:07 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 11:41:07 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> Message-ID: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> OK, sounds like the stability of the system is going downhill. I'm not sure when I'll have time to look at this. Maybe soon, maybe in the summer. I am discouraged at all the reports we have been having about these sorts of problems. On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > I am getting this, when my program does a Thread.Fork. > > *** > *** runtime error: > *** <*ASSERT*> failed. > *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > *** > > Any ideas? It worked... And stopped working... Some changes were made to openssl libraries and similar... > > TIA, > dd From dabenavidesd at yahoo.es Thu Apr 7 18:30:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 7 Apr 2011 17:30:54 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <785020.50972.qm@web29715.mail.ird.yahoo.com> Hi all: it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. Thanks in advance PS there is Elego's product wiki article http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System Also related: http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf And also: http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf Perhaps is good to see this issue: http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). --- El jue, 7/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Dragi?a Duri?" > CC: "m3devel" > Fecha: jueves, 7 de abril, 2011 10:41 > OK, sounds like the stability of the > system is going downhill. I'm not sure when I'll have > time to look at this. Maybe soon, maybe in the > summer. I am discouraged at all the reports we have > been having about these sorts of problems. > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > I am getting this, when my program does a > Thread.Fork. > > > > *** > > *** runtime error: > > *** <*ASSERT*> failed. > > *** file > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > *** > > > > Any ideas? It worked... And stopped working... Some > changes were made to openssl libraries and similar... > > > > TIA, > > dd > > From mika at async.caltech.edu Thu Apr 7 21:02:19 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 07 Apr 2011 12:02:19 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> Message-ID: <20110407190219.C00E31A2078@async.async.caltech.edu> Remember that for mission-critical applications there's always user threading.. I find that the system (up to and including the CVS head and on both amd64 and i386 flavors of Linux as well as every other OS I have tried) is very very stable with user threading. I'm eager to use pthreads again of course, for all sorts of reasons... Mika Tony Hosking writes: >OK, sounds like the stability of the system is going downhill. I'm not = >sure when I'll have time to look at this. Maybe soon, maybe in the = >summer. I am discouraged at all the reports we have been having about = >these sorts of problems. > >On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: > >> I am getting this, when my program does a Thread.Fork. >>=20 >> *** >> *** runtime error: >> *** <*ASSERT*> failed. >> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >> *** >>=20 >> Any ideas? It worked... And stopped working... Some changes were made = >to openssl libraries and similar... >>=20 >> TIA, >> dd From hosking at cs.purdue.edu Thu Apr 7 22:07:51 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 7 Apr 2011 16:07:51 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110407190219.C00E31A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> Message-ID: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Well, that is reassuring news. If the problems are isolated to threading then they can be resolved easily. On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > > Remember that for mission-critical applications there's always user > threading.. I find that the system (up to and including the CVS head > and on both amd64 and i386 flavors of Linux as well as every other OS I > have tried) is very very stable with user threading. I'm eager to use > pthreads again of course, for all sorts of reasons... > > Mika > > Tony Hosking writes: >> OK, sounds like the stability of the system is going downhill. I'm not = >> sure when I'll have time to look at this. Maybe soon, maybe in the = >> summer. I am discouraged at all the reports we have been having about = >> these sorts of problems. >> >> On Apr 7, 2011, at 10:34 AM, Dragi=C5=A1a Duri=C4=87 wrote: >> >>> I am getting this, when my program does a Thread.Fork. >>> =20 >>> *** >>> *** runtime error: >>> *** <*ASSERT*> failed. >>> *** file "../src/thread/PTHREAD/ThreadPThread.m3", line 586 >>> *** >>> =20 >>> Any ideas? It worked... And stopped working... Some changes were made = >> to openssl libraries and similar... >>> =20 >>> TIA, >>> dd From jay.krell at cornell.edu Thu Apr 7 23:25:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 7 Apr 2011 21:25:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <785020.50972.qm@web29715.mail.ird.yahoo.com> References: <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <785020.50972.qm@web29715.mail.ird.yahoo.com> Message-ID: Problems with cvsupd are understood and believed fixed. It depended on all threads surviving fork, instead of just the one forking thread. Which is what userthreads did and which is what pthreads never did and never will do. No threads survive fork+exec, which is what most users of fork do. But cvsupd uses the "fork and do more work" pattern. Userthreads have been changed to be like pthreads. Past problems with cvsupd on pthreads should NOT be used to think about problems anywhere else in the system. If cvsupd still doesn't work, then that is possibly relevant to the rest of the system. - Jay > Date: Thu, 7 Apr 2011 17:30:54 +0100 > From: dabenavidesd at yahoo.es > To: dragisha at m3w.org; hosking at cs.purdue.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Hi all: > it's still unclear whether or how much of this is not growing up from api instabilities rather than owned bugs. This is to say, many of this kind of bugs are rather implementation dependent, which is naturally a symptom of UNSAFE clearly, or what do you think if not? > > After that, what is the status on cvsup, I guess this is just another symptom of a bug from the birth of current Thread implementation (non UNSAFE), because as I recall from what it was reported to me in back 2008 fall and later year reproduced after somebody complaint, thing is that problem is cvsupd, not in the client, and so it could go back to some point in that gap [2006..2008], I can remember I compiled the full package several times but I can't say from that I tested the server, just connecting Elego one which I don't know after the big Elego machine's crash it didn't work well for me again that. One thing I do remember is exploring dcvs to make me an impression for a kind of public project but we never managed to start at least I, project partners may got that to work, I never did that part of the job. So this bugs me how we managed to use it if so, when it was not working, I still don't know, I can ask them however if neede. The last thing I > possibly did was to check out private copies of repositories and if so maybe this is what I would expect when it was working, as we got proxied cvsup ports, then we needed to move to local ftp'ing and perhaps but that's something I can't remember doing cvsup (we have had in the past a laboratory to ckeckout asterisk repositories but after they closed the service we didn't anymore and so we and ever looked for something like that to practice its use, never filled our exact expectations). > For now, I would say why if we test automatically if so a dcvs sessions, I will see that for an optional checking. > > Thanks in advance > > PS there is Elego's product wiki article > http://en.wikipedia.org/wiki/Distributed_Concurrent_Versions_System > Also related: > http://dissertations.ub.rug.nl/FILES/faculties/science/2005/h.gao/thesis.pdf > And also: > http://iwi.eldoc.ub.rug.nl/FILES/root/2007/SciCompProgGao/2007SciCompProgGao.pdf > > Perhaps is good to see this issue: > http://www.eetimes.com/design/embedded/4214763/Is-lock-free-programming-practical-for-multicore-?cid=NL_Embedded&Ecosystem=embedded > > This could be a clue in the current times issues, I believe embedded market and everywhere else is hitting hard, so this doesn't come by free (well a wicked C-free, yes why not? :) ). > > --- El jue, 7/4/11, Tony Hosking escribi?: > > > De: Tony Hosking > > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Para: "Dragi?a Duri?" > > CC: "m3devel" > > Fecha: jueves, 7 de abril, 2011 10:41 > > OK, sounds like the stability of the > > system is going downhill. I'm not sure when I'll have > > time to look at this. Maybe soon, maybe in the > > summer. I am discouraged at all the reports we have > > been having about these sorts of problems. > > > > On Apr 7, 2011, at 10:34 AM, Dragi?a Duri? wrote: > > > > > I am getting this, when my program does a > > Thread.Fork. > > > > > > *** > > > *** runtime error: > > > *** <*ASSERT*> failed. > > > *** file > > "../src/thread/PTHREAD/ThreadPThread.m3", line 586 > > > *** > > > > > > Any ideas? It worked... And stopped working... Some > > changes were made to openssl libraries and similar... > > > > > > TIA, > > > dd > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 00:45:16 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Sun, 10 Apr 2011 15:45:16 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX Message-ID: <20110410224516.E72F41A207A@async.async.caltech.edu> Hello m3devel (especially Jay), I'm having some time conversion problems. I *believe* (a bit hard to verify because m3gdb isn't working!!) that Date.ToTime called on Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. I note the following (fairly recent) code in DatePosixC.c: #ifdef _TIME64_T t = mktime64(&tm); #else t = mktime(&tm); #endif #ifdef DATE_BSD if (t == -1) goto Exit; /* adjust result to reflect "date->offset" */ #ifdef _TIME64_T time64(&now); local_now = localtime64(&now); #else time(&now); local_now = localtime(&now); #endif assert(local_now != NULL); if (local_now->tm_isdst > 0) /* decrement the local time zone by one hour if DST is in effect */ local_now->m3_tm_gmtoff -= SecsPerHour; /* As above, we must negate "date->offset" to account for the opposite sense of that field compared to Unix. */ t -= ((-date->offset) - local_now->m3_tm_gmtoff); Exit: #endif return t; What I don't understand is the subtracting off of SecsPerHour if "now" is DST. Yeah, now is DST (it's April), but the time I'm converting was in the winter, and besides GMT and UTC don't even have DST. I don't understand the intent of the code.. can someone explain why we're subtracting off an hour? And why is it based on the value of tm_isdst of the *current* time? Some additional documentation within the file would be helpful too. Note my conversion returns the correct 0 on various other M3 implementations. Note the system time on my system is in a timezone that has DST (namely, BST) but I have changed the TZ environment variable to UTC or GMT before calling this code. Mika From jay.krell at cornell.edu Mon Apr 11 04:00:35 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 11 Apr 2011 02:00:35 +0000 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110410224516.E72F41A207A@async.async.caltech.edu> References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: This stuff is tricky. Maybe impossible. And requires a fair amount of time and attention. Which I might not have right now, but which I definitely tried to apply at the time. Can you compare with some historical versions? e.g. http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=1.1.1.2;content-type=text%2Fplain http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=1.1.1.1;content-type=text%2Fplain and notice: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3 Revision 1.3: download - view: text, markup, annotated - select for diffs Note: This also fixes the UTC timezone on Solaris, Interix, HP-UX, Cygwin, which appear to have been historically broken. I'm usually a bit nervous when I make accusations of long-term breakage. And yeah there is a general problem of determining when daylight savings is in effect. The rules very often change. The operating system needs a lot of data to know this. This stuff should be easy to debug without m3gdb. Just print stuff to stdout liberally. Aha, see here: http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/Attic/DateBsd.m3?rev=1.1.1.2;content-type=text%2Fplain (* adjust result to reflect "d.offset" *) EVAL Utime.time(ADR(now)); local_now := Utime.localtime(ADR(now)); IF local_now.tm_isdst > 0 THEN (* decrement the local time zone by one hour if DST is in effect *) DEC(local_now.tm_gmtoff, 1 * SecsPerHour) END; - Jay > To: m3devel at elegosoft.com; jay.krell at cornell.edu > Date: Sun, 10 Apr 2011 15:45:16 -0700 > From: mika at async.caltech.edu > Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > > Hello m3devel (especially Jay), > > I'm having some time conversion problems. I *believe* (a bit hard to > verify because m3gdb isn't working!!) that Date.ToTime called on > Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. > > I note the following (fairly recent) code in DatePosixC.c: > > #ifdef _TIME64_T > t = mktime64(&tm); > #else > t = mktime(&tm); > #endif > #ifdef DATE_BSD > if (t == -1) > goto Exit; > > /* adjust result to reflect "date->offset" */ > #ifdef _TIME64_T > time64(&now); > local_now = localtime64(&now); > #else > time(&now); > local_now = localtime(&now); > #endif > assert(local_now != NULL); > if (local_now->tm_isdst > 0) > /* decrement the local time zone by one hour if DST is in effect */ > local_now->m3_tm_gmtoff -= SecsPerHour; > > /* As above, we must negate "date->offset" to account for the > opposite sense of that field compared to Unix. */ > t -= ((-date->offset) - local_now->m3_tm_gmtoff); > Exit: > #endif > return t; > > > What I don't understand is the subtracting off of SecsPerHour if > "now" is DST. Yeah, now is DST (it's April), but the time I'm > converting was in the winter, and besides GMT and UTC don't even > have DST. > > I don't understand the intent of the code.. can someone explain why > we're subtracting off an hour? And why is it based on the value of > tm_isdst of the *current* time? Some additional documentation within > the file would be helpful too. > > Note my conversion returns the correct 0 on various other M3 > implementations. > > Note the system time on my system is in a timezone that has DST > (namely, BST) but I have changed the TZ environment variable to UTC or GMT > before calling this code. > > Mika -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Mon Apr 11 11:01:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 11 Apr 2011 02:01:37 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: References: <20110410224516.E72F41A207A@async.async.caltech.edu> Message-ID: <20110411090137.BEF721A207A@async.async.caltech.edu> I'm not sure about impossible. But maybe... Here's the situation. I'm using unix time in my code. Sometimes it needs to run on Windows. Some versions of Modula-3 on Windows use the Windows epoch for timekeeping. So I first request the time at midnight UTC Jan 1 1970 and set that to "UnixEpoch". On Unix systems I expect it to be zero, on some other systems it might be another number. Now I've found what the problem is, and it's not anything you have done. In fact PM3 exhibits the same problem if you set everything up precisely the same. The issue is that the server I am using is located in the United Kingdom and it has its time zone set to Europe/London. Nowadays the Brits use the following definitions for time: in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 in summer "British Summer Time" abbrev. "BST" = UTC + 3600 However, between October 26, 1968 and October 31, 1971, they used something else, to wit: all year "British Standard Time" abbrev. "BST" = UTC + 3600 Note the confusing abbreviations. British Standard Time (which no longer exists) and British Summer Time have the same abbreviation. You might think this is sensible, because British Summer Time has the same value as British Standard Time (when the former is defined). There is just one problem. British Standard Time is not a "daylight savings time" whereas British Summer Time *is*. Witness the output of zdump for London: Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar 19 03:00:00 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct 29 02:59:59 1967 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct 29 02:00:00 1967 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb 18 01:59:59 1968 GMT isdst=0 gmtoff=0 Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb 18 03:00:00 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct 31 02:00:00 1971 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar 19 01:59:59 1972 GMT isdst=0 gmtoff=0 Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar 19 03:00:00 1972 BST isdst=1 gmtoff=3600 Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct 29 02:59:59 1972 BST isdst=1 gmtoff=3600 The following lines are the special ones: Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct 26 23:59:59 1968 BST isdst=1 gmtoff=3600 Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct 27 00:00:00 1968 BST isdst=0 gmtoff=3600 Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct 31 02:59:59 1971 BST isdst=0 gmtoff=3600 Note how the gmtoff stays the same but isdst changes. And this of course straddles the definition of Unix time at Jan 1 1970 00:00:00 UTC. Clearly there is a bug in M3's date code here. I just don't understand what invariant the original author appealed to when he inserted the dst-changing code. It is hardly unusual for a given locale to have changed time zone rules... it is completely wrong for that to affect the conversion of times for a *different* locale (here I am just trying to do a time conversion for UTC, it shouldn't matter what country my computer is in when it does that!) Mika Jay K writes: >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention. >Which I might not have right now=2C but which I definitely tried to apply a= >t the time. > >Can you compare with some historical versions? >e.g. > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > >and notice: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3 >Revision 1.3: download - view: text=2C markup=2C annotated - select for dif= >fs >Note: This also fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cy= >gwin=2C which appear to have been historically broken. > > >I'm usually a bit nervous when I make accusations of long-term breakage. > > >And yeah there is a general problem of determining when daylight savings is= > in effect. >The rules very often change. The operating system needs a lot of data to kn= >ow this. > > >This stuff should be easy to debug without m3gdb. >Just print stuff to stdout liberally. > >Aha=2C see here: >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > (* adjust result to reflect "d.offset" *) > EVAL Utime.time(ADR(now))=3B > local_now :=3D Utime.localtime(ADR(now))=3B > IF local_now.tm_isdst > 0 THEN > (* decrement the local time zone by one hour if DST is in effect *) > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour) > END=3B > > - Jay > >> To: m3devel at elegosoft.com=3B jay.krell at cornell.edu >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 >> From: mika at async.caltech.edu >> Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX >>=20 >> Hello m3devel (especially Jay)=2C >>=20 >> I'm having some time conversion problems. I *believe* (a bit hard to >> verify because m3gdb isn't working!!) that Date.ToTime called on >> Jan 1 1970 in timezone "UTC" or "GMT" is returning -3600 for me. >>=20 >> I note the following (fairly recent) code in DatePosixC.c: >>=20 >> #ifdef _TIME64_T >> t =3D mktime64(&tm)=3B >> #else >> t =3D mktime(&tm)=3B >> #endif >> #ifdef DATE_BSD >> if (t =3D=3D -1) >> goto Exit=3B >>=20 >> /* adjust result to reflect "date->offset" */ >> #ifdef _TIME64_T >> time64(&now)=3B >> local_now =3D localtime64(&now)=3B >> #else >> time(&now)=3B >> local_now =3D localtime(&now)=3B >> #endif >> assert(local_now !=3D NULL)=3B >> if (local_now->tm_isdst > 0) >> /* decrement the local time zone by one hour if DST is in effect */ >> local_now->m3_tm_gmtoff -=3D SecsPerHour=3B >>=20 >> /* As above=2C we must negate "date->offset" to account for the >> opposite sense of that field compared to Unix. */ >> t -=3D ((-date->offset) - local_now->m3_tm_gmtoff)=3B >> Exit: >> #endif >> return t=3B >>=20 >>=20 >> What I don't understand is the subtracting off of SecsPerHour if >> "now" is DST. Yeah=2C now is DST (it's April)=2C but the time I'm >> converting was in the winter=2C and besides GMT and UTC don't even >> have DST.=20 >>=20 >> I don't understand the intent of the code.. can someone explain why >> we're subtracting off an hour? And why is it based on the value of=20 >> tm_isdst of the *current* time? Some additional documentation within >> the file would be helpful too. >>=20 >> Note my conversion returns the correct 0 on various other M3 >> implementations. >>=20 >> Note the system time on my system is in a timezone that has DST=20 >> (namely=2C BST) but I have changed the TZ environment variable to UTC or = >GMT >> before calling this code. >>=20 >> Mika > = > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >This stuff is tricky. Maybe impossible. And requires a fair amount of time = >and attention.
Which I might not have right now=2C but which I definitel= >y tried to apply at the time.

Can you compare with some historical v= >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= >lain


and notice:

http://modula3.elegosoft.com/cgi-bin/cvs= >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision 1.3b>: 7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fplain" class=3D"download-link">download - view: tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" class=3D"display= >-link">text=2C gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= >=3Dtext%2Fx-cvsweb-markup" class=3D"display-link">markup=2C http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= >/POSIX/TimePosix.m3?annotate=3D1.3">annotated - a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: This a= >lso fixes the UTC timezone on Solaris=2C Interix=2C HP-UX=2C Cygwin=2C whic= >h appear to have been historically broken.


I'm usually a bit ner= >vous when I make accusations of long-term breakage.


And yeah the= >re is a general problem of determining when daylight savings is in effect.<= >br>The rules very often change. The operating system needs a lot of data to= > know this.


This stuff should be easy to debug without m3gdb.>Just print stuff to stdout liberally.

Aha=2C see here:
http://mo= >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

  (*=
> adjust result to reflect "d.offset" *)
    EVAL Utime.time(ADR(now))=3B=
>
    local_now :=3D Utime.localtime(ADR(now))=3B
    IF local_now.tm_=
>isdst >=3B 0 THEN
      (* decrement the local time zone by one hour i=
>f DST is in effect *)
      DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
>br>    END=3B

 =3B- Jay

>=3B To: m3devel at elegosof= >t.com=3B jay.krell at cornell.edu
>=3B Date: Sun=2C 10 Apr 2011 15:45:16 = >-0700
>=3B From: mika at async.caltech.edu
>=3B Subject: [M3devel] t= >ime conversion problems : Date.ToTime on AMD64_LINUX
>=3B
>=3B H= >ello m3devel (especially Jay)=2C
>=3B
>=3B I'm having some time = >conversion problems. I *believe* (a bit hard to
>=3B verify because m= >3gdb isn't working!!) that Date.ToTime called on
>=3B Jan 1 1970 in ti= >mezone "UTC" or "GMT" is returning -3600 for me.
>=3B
>=3B I not= >e the following (fairly recent) code in DatePosixC.c:
>=3B
>=3B = >#ifdef _TIME64_T
>=3B t =3D mktime64(&=3Btm)=3B
>=3B #else= >
>=3B t =3D mktime(&=3Btm)=3B
>=3B #endif
>=3B #ifdef= > DATE_BSD
>=3B if (t =3D=3D -1)
>=3B goto Exit=3B
= >>=3B
>=3B /* adjust result to reflect "date->=3Boffset" */>>=3B #ifdef _TIME64_T
>=3B time64(&=3Bnow)=3B
>=3B = >local_now =3D localtime64(&=3Bnow)=3B
>=3B #else
>=3B time= >(&=3Bnow)=3B
>=3B local_now =3D localtime(&=3Bnow)=3B
>= >=3B #endif
>=3B assert(local_now !=3D NULL)=3B
>=3B if (l= >ocal_now->=3Btm_isdst >=3B 0)
>=3B /* decrement the local ti= >me zone by one hour if DST is in effect */
>=3B local_now->=3B= >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B
>=3B /* As above=2C we= > must negate "date->=3Boffset" to account for the
>=3B opposi= >te sense of that field compared to Unix. */
>=3B t -=3D ((-date-&g= >t=3Boffset) - local_now->=3Bm3_tm_gmtoff)=3B
>=3B Exit:
>=3B #e= >ndif
>=3B return t=3B
>=3B
>=3B
>=3B What I don't= > understand is the subtracting off of SecsPerHour if
>=3B "now" is DST= >. Yeah=2C now is DST (it's April)=2C but the time I'm
>=3B converting= > was in the winter=2C and besides GMT and UTC don't even
>=3B have DST= >.
>=3B
>=3B I don't understand the intent of the code.. can som= >eone explain why
>=3B we're subtracting off an hour? And why is it ba= >sed on the value of
>=3B tm_isdst of the *current* time? Some additi= >onal documentation within
>=3B the file would be helpful too.
>= >=3B
>=3B Note my conversion returns the correct 0 on various other M3= >
>=3B implementations.
>=3B
>=3B Note the system time on my= > system is in a timezone that has DST
>=3B (namely=2C BST) but I have= > changed the TZ environment variable to UTC or GMT
>=3B before calling= > this code.
>=3B
>=3B Mika
> >= > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- From dabenavidesd at yahoo.es Mon Apr 11 18:28:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 11 Apr 2011 17:28:45 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <20110411090137.BEF721A207A@async.async.caltech.edu> Message-ID: <926161.28913.qm@web29713.mail.ird.yahoo.com> Hi all: although my understanding of the problem is not that good, I think there are two routes for this abstract way one could try to prove the correctness of some implementation by setting directly instrumention code to try to see for your self dynamically if you may that or what you want to see: http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s This could would run with just 3 missing modules from this sources besides their examples, maybe another time the HP guys allow us to see their works on that: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ This package is missing from public repositories Elego too as I know, though they were in (see bottom links dead as of now, but, ... maybe you can look for yourselves): http://www.cs.arizona.edu/~collberg/Research/Modula-3/ I know some ESC/Modula-3 tries to make some inferencing of invariants, but as I said it dones't mean this is totally computable, im fact in general its isn't, but it would be good to try. Essentially where are the codes for showing that, or are you implementing this as private code? Thanks in advance --- El lun, 11/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 04:01 > I'm not sure about impossible. > But maybe... > > Here's the situation. > > I'm using unix time in my code. Sometimes it needs to > run on Windows. > Some versions of Modula-3 on Windows use the Windows epoch > for timekeeping. > So I first request the time at midnight UTC Jan 1 1970 and > set that to > "UnixEpoch". On Unix systems I expect it to be zero, > on some other systems > it might be another number. > > Now I've found what the problem is, and it's not anything > you have done. > In fact PM3 exhibits the same problem if you set everything > up precisely > the same. > > The issue is that the server I am using is located in the > United Kingdom > and it has its time zone set to Europe/London. > Nowadays the Brits use the > following definitions for time: > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + 0 > in summer "British Summer Time" abbrev. "BST" = UTC + 3600 > > However, between October 26, 1968 and October 31, 1971, > they used something > else, to wit: > > all year "British Standard Time" abbrev. "BST" = UTC + > 3600 > > Note the confusing abbreviations. British Standard > Time (which no longer > exists) and British Summer Time have the same > abbreviation. You might think > this is sensible, because British Summer Time has the same > value as British > Standard Time (when the former is defined). > > There is just one problem. British Standard Time is > not a "daylight > savings time" whereas British Summer Time *is*. > Witness the output of > zdump for London: > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun Mar > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun Oct > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun Oct > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun Feb > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun Feb > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun Oct > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun Mar > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun Mar > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun Oct > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > The following lines are the special ones: > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat Oct > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun Oct > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun Oct > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Note how the gmtoff stays the same but isdst changes. > And this of course > straddles the definition of Unix time at Jan 1 1970 > 00:00:00 UTC. > > Clearly there is a bug in M3's date code here. I just > don't understand what > invariant the original author appealed to when he inserted > the dst-changing code. > It is hardly unusual for a given locale to have changed > time zone rules... > it is completely wrong for that to affect the conversion of > times for a *different* > locale (here I am just trying to do a time conversion for > UTC, it shouldn't > matter what country my computer is in when it does that!) > > Mika > > Jay K writes: > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention. > >Which I might not have right now=2C but which I > definitely tried to apply a= > >t the time. > > > >Can you compare with some historical versions? > >e.g. > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > >and notice: > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3 > >Revision 1.3: download - view: text=2C markup=2C > annotated - select for dif= > >fs > >Note: This also fixes the UTC timezone on Solaris=2C > Interix=2C HP-UX=2C Cy= > >gwin=2C which appear to have been historically broken. > > > > > >I'm usually a bit nervous when I make accusations of > long-term breakage. > > > > > >And yeah there is a general problem of determining when > daylight savings is= > > in effect. > >The rules very often change. The operating system needs > a lot of data to kn= > >ow this. > > > > > >This stuff should be easy to debug without m3gdb. > >Just print stuff to stdout liberally. > > > >Aha=2C see here: > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > (* adjust result to reflect "d.offset" *) > > EVAL Utime.time(ADR(now))=3B > > local_now :=3D > Utime.localtime(ADR(now))=3B > > IF local_now.tm_isdst > 0 THEN > > (* decrement the local time zone > by one hour if DST is in effect *) > > DEC(local_now.tm_gmtoff=2C 1 * > SecsPerHour) > > END=3B > > > > - Jay > > > >> To: m3devel at elegosoft.com=3B > jay.krell at cornell.edu > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > >> From: mika at async.caltech.edu > >> Subject: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > >>=20 > >> Hello m3devel (especially Jay)=2C > >>=20 > >> I'm having some time conversion problems. I > *believe* (a bit hard to > >> verify because m3gdb isn't working!!) that > Date.ToTime called on > >> Jan 1 1970 in timezone "UTC" or "GMT" is returning > -3600 for me. > >>=20 > >> I note the following (fairly recent) code in > DatePosixC.c: > >>=20 > >> #ifdef _TIME64_T > >> t =3D > mktime64(&tm)=3B > >> #else > >> t =3D mktime(&tm)=3B > >> #endif > >> #ifdef DATE_BSD > >> if (t =3D=3D -1) > >> goto > Exit=3B > >>=20 > >> /* adjust result to > reflect "date->offset" */ > >> #ifdef _TIME64_T > >> time64(&now)=3B > >> local_now =3D > localtime64(&now)=3B > >> #else > >> time(&now)=3B > >> local_now =3D > localtime(&now)=3B > >> #endif > >> assert(local_now !=3D > NULL)=3B > >> if (local_now->tm_isdst > > 0) > >> /* decrement the > local time zone by one hour if DST is in effect */ > >> > local_now->m3_tm_gmtoff -=3D > SecsPerHour=3B > >>=20 > >> /* As above=2C we must > negate "date->offset" to account for the > >> opposite sense of that > field compared to Unix. */ > >> t -=3D ((-date->offset) > - local_now->m3_tm_gmtoff)=3B > >> Exit: > >> #endif > >> return t=3B > >>=20 > >>=20 > >> What I don't understand is the subtracting off of > SecsPerHour if > >> "now" is DST. Yeah=2C now is DST (it's > April)=2C but the time I'm > >> converting was in the winter=2C and besides GMT > and UTC don't even > >> have DST.=20 > >>=20 > >> I don't understand the intent of the code.. can > someone explain why > >> we're subtracting off an hour? And why is it > based on the value of=20 > >> tm_isdst of the *current* time? Some > additional documentation within > >> the file would be helpful too. > >>=20 > >> Note my conversion returns the correct 0 on > various other M3 > >> implementations. > >>=20 > >> Note the system time on my system is in a timezone > that has DST=20 > >> (namely=2C BST) but I have changed the TZ > environment variable to UTC or = > >GMT > >> before calling this code. > >>=20 > >> Mika > > > > > = > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And requires a > fair amount of time = > >and attention.
Which I might not have right > now=2C but which I definitel= > >y tried to apply at the time.

Can > you compare with some historical v= > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > >lain


and > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > 1.3 >b>: >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fplain" > class=3D"download-link">download - view: href=3D"ht= > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > class=3D"display= > >-link">text=2C >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > >=3Dtext%2Fx-cvsweb-markup" > class=3D"display-link">markup=2C href=3D"= > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > - >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > This a= > >lso fixes the UTC timezone on Solaris=2C Interix=2C > HP-UX=2C Cygwin=2C whic= > >h appear to have been historically > broken.


I'm usually a bit ner= > >vous when I make accusations of long-term > breakage.


And yeah the= > >re is a general problem of determining when daylight > savings is in effect.<= > >br>The rules very often change. The operating system > needs a lot of data to= > > know this.


This stuff > should be easy to debug without m3gdb. >>Just print stuff to stdout > liberally.

Aha=2C see here:
http://mo= > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

 
> (*=
> > adjust result to reflect "d.offset" *)
 
>   EVAL Utime.time(ADR(now))=3B=
> >
    local_now :=3D
> Utime.localtime(ADR(now))=3B
    IF
> local_now.tm_=
> >isdst >=3B 0 THEN
     
> (* decrement the local time zone by one hour i=
> >f DST is in effect *)
     
> DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> >br>   
> END=3B

 =3B- > Jay

>=3B To: m3devel at elegosof= > >t.com=3B jay.krell at cornell.edu
>=3B > Date: Sun=2C 10 Apr 2011 15:45:16 = > >-0700
>=3B From: mika at async.caltech.edu
>=3B > Subject: [M3devel] t= > >ime conversion problems : Date.ToTime on > AMD64_LINUX
>=3B
>=3B H= > >ello m3devel (especially Jay)=2C
>=3B >
>=3B I'm having some time = > >conversion problems. I *believe* (a bit hard > to
>=3B verify because m= > >3gdb isn't working!!) that Date.ToTime called > on
>=3B Jan 1 1970 in ti= > >mezone "UTC" or "GMT" is returning -3600 for > me.
>=3B
>=3B I not= > >e the following (fairly recent) code in > DatePosixC.c:
>=3B
>=3B = > >#ifdef _TIME64_T
>=3B > t =3D > mktime64(&=3Btm)=3B
>=3B #else= > >
>=3B t =3D > mktime(&=3Btm)=3B
>=3B > #endif
>=3B #ifdef= > > DATE_BSD
>=3B > if (t =3D=3D -1)
>=3B > goto Exit=3B
= > >>=3B
>=3B > /* adjust result to reflect > "date->=3Boffset" */ >>>=3B #ifdef > _TIME64_T
>=3B > time64(&=3Bnow)=3B
>=3B > = > >local_now =3D > localtime64(&=3Bnow)=3B
>=3B > #else
>=3B time= > >(&=3Bnow)=3B
>=3B > local_now =3D > localtime(&=3Bnow)=3B
>= > >=3B #endif
>=3B > assert(local_now !=3D > NULL)=3B
>=3B if (l= > >ocal_now->=3Btm_isdst >=3B > 0)
>=3B /* > decrement the local ti= > >me zone by one hour if DST is in effect > */
>=3B > local_now->=3B= > >m3_tm_gmtoff -=3D SecsPerHour=3B
>=3B >
>=3B /* As above=2C > we= > > must negate "date->=3Boffset" to account for > the
>=3B opposi= > >te sense of that field compared to Unix. > */
>=3B t -=3D > ((-date-&g= > >t=3Boffset) - > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > Exit:
>=3B #e= > >ndif
>=3B return > t=3B
>=3B
>=3B >
>=3B What I don't= > > understand is the subtracting off of SecsPerHour > if
>=3B "now" is DST= > >. Yeah=2C now is DST (it's April)=2C but the time > I'm
>=3B converting= > > was in the winter=2C and besides GMT and UTC don't > even
>=3B have DST= > >.
>=3B
>=3B I don't > understand the intent of the code.. can som= > >eone explain why
>=3B we're subtracting > off an hour? And why is it ba= > >sed on the value of
>=3B tm_isdst of > the *current* time? Some additi= > >onal documentation within
>=3B the file > would be helpful too.
>= > >=3B
>=3B Note my conversion returns the > correct 0 on various other M3= > >
>=3B > implementations.
>=3B
>=3B > Note the system time on my= > > system is in a timezone that has DST >
>=3B (namely=2C BST) but I have= > > changed the TZ environment variable to UTC or > GMT
>=3B before calling= > > this code.
>=3B >
>=3B Mika
> > > > > > >= > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > From dabenavidesd at yahoo.es Wed Apr 13 17:14:51 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 13 Apr 2011 16:14:51 +0100 (BST) Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <171917.14906.qm@web29716.mail.ird.yahoo.com> Hi all: it would nice to ask HP whether they have decommissioned algorithm animation examples in Zeus3D, to get their sources, since there hasn't been much advances in this area even without Modula-3 on the Scene (let's say of my knowledge there has not been too much attention to hold animation festivals regularly, etc), I just wonder what is about their example implementations: ftp://gatekeeper.research.compaq.com/pub/DEC/SRC/research-reports/SRC-110a.pdf The ones seen in first video report at least, the heap sort: http://replay.waybackmachine.org/20051217123443/http://www.research.compaq.com/SRC/zeus/heapsort3D.html http://replay.waybackmachine.org/20051101010009/http://research.compaq.com/SRC/zeus/sorting3D.html I think 234 trees are missing screenshots (binary three search) though redblack trees appears: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/redblack3D.html The same example for computational geometry kd-tree: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/2dtree3D.html And the last shortest path algorithm example: http://replay.waybackmachine.org/20051217131627/http://www.research.compaq.com/SRC/zeus/shortestpath3D.html The ones, decommissioned ones I read in initial paragraph here: http://www.internotesstrategy.com/AlgorithMain.htm The most interesting ones in Modula2+ if so are the parallel sorting algorithms, but who might know where are they? Thanks in advance --- El lun, 11/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX > Para: "Jay K" , "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 11 de abril, 2011 11:28 > Hi all: > although my understanding of the problem is not that good, > I think there are two routes for this abstract way one could > try to prove the correctness of some implementation by > setting directly instrumention code to try to see for your > self dynamically if you may that or what you want to see: > http://www.youtube.com/watch?v=zIgu9q0vVc0#t=2m06s > > This could would run with just 3 missing modules from this > sources besides their examples, maybe another time the HP > guys allow us to see their works on that: > http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/ > > This package is missing from public repositories Elego too > as I know, though they were in (see bottom links dead as of > now, but, ... maybe you can look for yourselves): > http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > > I know some ESC/Modula-3 tries to make some inferencing of > invariants, but as I said it dones't mean this is totally > computable, im fact in general its isn't, but it would be > good to try. > > Essentially where are the codes for showing that, or are > you implementing this as private code? > > Thanks in advance > > --- El lun, 11/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] time conversion problems : > Date.ToTime on AMD64_LINUX > > Para: "Jay K" > > CC: m3devel at elegosoft.com > > Fecha: lunes, 11 de abril, 2011 04:01 > > I'm not sure about impossible. > > But maybe... > > > > Here's the situation. > > > > I'm using unix time in my code. Sometimes it > needs to > > run on Windows. > > Some versions of Modula-3 on Windows use the Windows > epoch > > for timekeeping. > > So I first request the time at midnight UTC Jan 1 1970 > and > > set that to > > "UnixEpoch". On Unix systems I expect it to be > zero, > > on some other systems > > it might be another number. > > > > Now I've found what the problem is, and it's not > anything > > you have done. > > In fact PM3 exhibits the same problem if you set > everything > > up precisely > > the same. > > > > The issue is that the server I am using is located in > the > > United Kingdom > > and it has its time zone set to Europe/London. > > Nowadays the Brits use the > > following definitions for time: > > > > in winter "Greenwich Mean Time" abbrev. "GMT" = UTC + > 0 > > in summer "British Summer Time" abbrev. "BST" = UTC + > 3600 > > > > However, between October 26, 1968 and October 31, > 1971, > > they used something > > else, to wit: > > > > all year "British Standard Time" abbrev. "BST" = UTC > + > > 3600 > > > > Note the confusing abbreviations. British > Standard > > Time (which no longer > > exists) and British Summer Time have the same > > abbreviation. You might think > > this is sensible, because British Summer Time has the > same > > value as British > > Standard Time (when the former is defined). > > > > There is just one problem. British Standard Time > is > > not a "daylight > > savings time" whereas British Summer Time *is*. > > Witness the output of > > zdump for London: > > > > Europe/London Sun Mar 19 02:00:00 1967 UTC = Sun > Mar > > 19 03:00:00 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1967 UTC = Sun > Oct > > 29 02:59:59 1967 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 02:00:00 1967 UTC = Sun > Oct > > 29 02:00:00 1967 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 01:59:59 1968 UTC = Sun > Feb > > 18 01:59:59 1968 GMT isdst=0 gmtoff=0 > > Europe/London Sun Feb 18 02:00:00 1968 UTC = Sun > Feb > > 18 03:00:00 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 02:00:00 1971 UTC = Sun > Oct > > 31 02:00:00 1971 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 01:59:59 1972 UTC = Sun > Mar > > 19 01:59:59 1972 GMT isdst=0 gmtoff=0 > > Europe/London Sun Mar 19 02:00:00 1972 UTC = Sun > Mar > > 19 03:00:00 1972 BST isdst=1 gmtoff=3600 > > Europe/London Sun Oct 29 01:59:59 1972 UTC = Sun > Oct > > 29 02:59:59 1972 BST isdst=1 gmtoff=3600 > > > > The following lines are the special ones: > > > > Europe/London Sat Oct 26 22:59:59 1968 UTC = Sat > Oct > > 26 23:59:59 1968 BST isdst=1 gmtoff=3600 > > Europe/London Sat Oct 26 23:00:00 1968 UTC = Sun > Oct > > 27 00:00:00 1968 BST isdst=0 gmtoff=3600 > > Europe/London Sun Oct 31 01:59:59 1971 UTC = Sun > Oct > > 31 02:59:59 1971 BST isdst=0 gmtoff=3600 > > > > Note how the gmtoff stays the same but isdst changes. > > > And this of course > > straddles the definition of Unix time at Jan 1 1970 > > 00:00:00 UTC. > > > > Clearly there is a bug in M3's date code here. I > just > > don't understand what > > invariant the original author appealed to when he > inserted > > the dst-changing code. > > It is hardly unusual for a given locale to have > changed > > time zone rules... > > it is completely wrong for that to affect the > conversion of > > times for a *different* > > locale (here I am just trying to do a time conversion > for > > UTC, it shouldn't > > matter what country my computer is in when it does > that!) > > > > Mika > > > > Jay K writes: > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/plain; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention. > > >Which I might not have right now=2C but which I > > definitely tried to apply a= > > >t the time. > > > > > >Can you compare with some historical versions? > > >e.g. > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fplain > > > > > > > > >and notice: > > > > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > >/POSIX/TimePosix.m3 > > >Revision 1.3: download - view: text=2C markup=2C > > annotated - select for dif= > > >fs > > >Note: This also fixes the UTC timezone on > Solaris=2C > > Interix=2C HP-UX=2C Cy= > > >gwin=2C which appear to have been historically > broken. > > > > > > > > >I'm usually a bit nervous when I make accusations > of > > long-term breakage. > > > > > > > > >And yeah there is a general problem of determining > when > > daylight savings is= > > > in effect. > > >The rules very often change. The operating system > needs > > a lot of data to kn= > > >ow this. > > > > > > > > >This stuff should be easy to debug without m3gdb. > > >Just print stuff to stdout liberally. > > > > > >Aha=2C see here: > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/Attic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain > > > > > > (* adjust result to reflect "d.offset" *) > > > EVAL Utime.time(ADR(now))=3B > > > local_now :=3D > > Utime.localtime(ADR(now))=3B > > > IF local_now.tm_isdst > 0 THEN > > > (* decrement the local time > zone > > by one hour if DST is in effect *) > > > DEC(local_now.tm_gmtoff=2C 1 > * > > SecsPerHour) > > > END=3B > > > > > > - Jay > > > > > >> To: m3devel at elegosoft.com=3B > > jay.krell at cornell.edu > > >> Date: Sun=2C 10 Apr 2011 15:45:16 -0700 > > >> From: mika at async.caltech.edu > > >> Subject: [M3devel] time conversion problems > : > > Date.ToTime on AMD64_LINUX > > >>=20 > > >> Hello m3devel (especially Jay)=2C > > >>=20 > > >> I'm having some time conversion > problems. I > > *believe* (a bit hard to > > >> verify because m3gdb isn't working!!) that > > Date.ToTime called on > > >> Jan 1 1970 in timezone "UTC" or "GMT" is > returning > > -3600 for me. > > >>=20 > > >> I note the following (fairly recent) code in > > DatePosixC.c: > > >>=20 > > >> #ifdef _TIME64_T > > >> t =3D > > mktime64(&tm)=3B > > >> #else > > >> t =3D > mktime(&tm)=3B > > >> #endif > > >> #ifdef DATE_BSD > > >> if (t =3D=3D -1) > > >> goto > > Exit=3B > > >>=20 > > >> /* adjust result to > > reflect "date->offset" */ > > >> #ifdef _TIME64_T > > >> time64(&now)=3B > > >> local_now =3D > > localtime64(&now)=3B > > >> #else > > >> time(&now)=3B > > >> local_now =3D > > localtime(&now)=3B > > >> #endif > > >> assert(local_now > !=3D > > NULL)=3B > > >> if > (local_now->tm_isdst > > > 0) > > >> /* decrement > the > > local time zone by one hour if DST is in effect */ > > >> > > local_now->m3_tm_gmtoff -=3D > > SecsPerHour=3B > > >>=20 > > >> /* As above=2C we > must > > negate "date->offset" to account for the > > >> opposite sense of > that > > field compared to Unix. */ > > >> t -=3D > ((-date->offset) > > - local_now->m3_tm_gmtoff)=3B > > >> Exit: > > >> #endif > > >> return t=3B > > >>=20 > > >>=20 > > >> What I don't understand is the subtracting > off of > > SecsPerHour if > > >> "now" is DST. Yeah=2C now is DST (it's > > April)=2C but the time I'm > > >> converting was in the winter=2C and besides > GMT > > and UTC don't even > > >> have DST.=20 > > >>=20 > > >> I don't understand the intent of the code.. > can > > someone explain why > > >> we're subtracting off an hour? And why > is it > > based on the value of=20 > > >> tm_isdst of the *current* time? Some > > additional documentation within > > >> the file would be helpful too. > > >>=20 > > >> Note my conversion returns the correct 0 on > > various other M3 > > >> implementations. > > >>=20 > > >> Note the system time on my system is in a > timezone > > that has DST=20 > > >> (namely=2C BST) but I have changed the TZ > > environment variable to UTC or = > > >GMT > > >> before calling this code. > > >>=20 > > >> Mika > > > > > > > > > = > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_ > > >Content-Type: text/html; charset="iso-8859-1" > > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > > > > > > > > >This stuff is tricky. Maybe impossible. And > requires a > > fair amount of time = > > >and attention.
Which I might not have > right > > now=2C but which I definitel= > > >y tried to apply at the > time.

Can > > you compare with some historical v= > > >ersions?
e.g.

http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3= > > > >/m3-libs/m3core/src/time/POSIX/DatePosix.m3?rev=3D1.1.1.2=3Bcontent-type=3D= > > >text%2Fplain
http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs= > > > >/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.1.1.1=3Bcontent-type=3Dtext%2Fp= > > >lain


and > > notice:

http://modula3.elegosoft.com/cgi-bin/cvs= > > > >web.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3
Revision > > 1.3 > >b>: > > >7E/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fplain" > > class=3D"download-link">download - view: > > href=3D"ht= > > > >tp://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/P= > > > >OSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type=3Dtext%2Fplain" > > class=3D"display= > > >-link">text=2C > > >gi/cm3/m3-libs/m3core/src/time/POSIX/TimePosix.m3?rev=3D1.3=3Bcontent-type= > > >=3Dtext%2Fx-cvsweb-markup" > > class=3D"display-link">markup=2C > href=3D"= > > >http://modula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time= > > > >/POSIX/TimePosix.m3?annotate=3D1.3">annotated > > - > > >a3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/TimeP= > > > >osix.m3?r1=3D1.3#rev1.3">select =3Bfor =3Bdiffs
Note: > > This a= > > >lso fixes the UTC timezone on Solaris=2C > Interix=2C > > HP-UX=2C Cygwin=2C whic= > > >h appear to have been historically > > broken.


I'm usually a bit > ner= > > >vous when I make accusations of long-term > > breakage.


And yeah the= > > >re is a general problem of determining when > daylight > > savings is in effect.<= > > >br>The rules very often change. The operating > system > > needs a lot of data to= > > > know this.


This > stuff > > should be easy to debug without m3gdb. > >>Just print stuff to stdout > > liberally.

Aha=2C see > here:
http://mo= > > > >dula3.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/time/POSIX/At= > > > >tic/DateBsd.m3?rev=3D1.1.1.2=3Bcontent-type=3Dtext%2Fplain

> 
> > (*=
> > > adjust result to reflect "d.offset" *)

> 
> >   EVAL Utime.time(ADR(now))=3B=
> > >
    local_now :=3D
> > Utime.localtime(ADR(now))=3B
   
> IF
> > local_now.tm_=
> > >isdst >=3B 0 THEN
 
>    
> > (* decrement the local time zone by one hour i=
> > >f DST is in effect *)
 
>    
> > DEC(local_now.tm_gmtoff=2C 1 * SecsPerHour)<=
> > >br>   
> > END=3B

 =3B- > > Jay

>=3B To: > m3devel at elegosof= > > >t.com=3B jay.krell at cornell.edu
>=3B > > Date: Sun=2C 10 Apr 2011 15:45:16 = > > >-0700
>=3B From: mika at async.caltech.edu
>=3B > > Subject: [M3devel] t= > > >ime conversion problems : Date.ToTime on > > AMD64_LINUX
>=3B
>=3B > H= > > >ello m3devel (especially > Jay)=2C
>=3B > >
>=3B I'm having some time = > > >conversion problems. I *believe* (a bit > hard > > to
>=3B verify because m= > > >3gdb isn't working!!) that Date.ToTime called > > on
>=3B Jan 1 1970 in ti= > > >mezone "UTC" or "GMT" is returning -3600 for > > me.
>=3B
>=3B I not= > > >e the following (fairly recent) code in > > DatePosixC.c:
>=3B
>=3B > = > > >#ifdef _TIME64_T
>=3B > > t =3D > > mktime64(&=3Btm)=3B
>=3B #else= > > >
>=3B t > =3D > > mktime(&=3Btm)=3B
>=3B > > #endif
>=3B #ifdef= > > > DATE_BSD
>=3B > > if (t =3D=3D -1)
>=3B > > goto Exit=3B
= > > >>=3B
>=3B > > /* adjust result to reflect > > "date->=3Boffset" */ > >>>=3B #ifdef > > _TIME64_T
>=3B > > > time64(&=3Bnow)=3B
>=3B > > = > > >local_now =3D > > localtime64(&=3Bnow)=3B
>=3B > > #else
>=3B > time= > > >(&=3Bnow)=3B
>=3B > > local_now =3D > > localtime(&=3Bnow)=3B
>= > > >=3B #endif
>=3B > > assert(local_now !=3D > > NULL)=3B
>=3B > if (l= > > >ocal_now->=3Btm_isdst >=3B > > 0)
>=3B > /* > > decrement the local ti= > > >me zone by one hour if DST is in effect > > */
>=3B > > local_now->=3B= > > >m3_tm_gmtoff -=3D > SecsPerHour=3B
>=3B > >
>=3B /* As > above=2C > > we= > > > must negate "date->=3Boffset" to account > for > > the
>=3B > opposi= > > >te sense of that field compared to Unix. > > */
>=3B t -=3D > > ((-date-&g= > > >t=3Boffset) - > > > local_now->=3Bm3_tm_gmtoff)=3B
>=3B > > Exit:
>=3B #e= > > >ndif
>=3B > return > > t=3B
>=3B
>=3B > >
>=3B What I don't= > > > understand is the subtracting off of SecsPerHour > > if
>=3B "now" is DST= > > >. Yeah=2C now is DST (it's April)=2C but the > time > > I'm
>=3B converting= > > > was in the winter=2C and besides GMT and UTC > don't > > even
>=3B have DST= > > >.
>=3B
>=3B I > don't > > understand the intent of the code.. can som= > > >eone explain why
>=3B we're > subtracting > > off an hour? And why is it ba= > > >sed on the value of
>=3B tm_isdst > of > > the *current* time? Some additi= > > >onal documentation within
>=3B the > file > > would be helpful too.
>= > > >=3B
>=3B Note my conversion > returns the > > correct 0 on various other M3= > > >
>=3B > > implementations.
>=3B >
>=3B > > Note the system time on my= > > > system is in a timezone that has DST > >
>=3B (namely=2C BST) but I have= > > > changed the TZ environment variable to UTC or > > GMT
>=3B before calling= > > > this code.
>=3B > >
>=3B > Mika
> > > > > > > > > > > >= > > > > > >--_2ede827e-b296-47fe-88ba-2c08d419ca54_-- > > > From rodney_bates at lcwb.coop Thu Apr 14 17:45:06 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Thu, 14 Apr 2011 10:45:06 -0500 Subject: [M3devel] Thread local storage? Message-ID: <4DA71682.9040709@lcwb.coop> I have never understood what "thread local storage" is all about, in particular, what need it addresses. All local variables and parameters are already private to a thread, as consequence of semantics going back as far as Algol-60. They are allocated space when and only when a call is executed. That can't happen until there is a thread to execute the call and it has to have its own private stack where the space is allocated, or it wouldn't work as a thread, (OK leaving out languages like very old Fortrans that don't support recursion either.) That leaves global variables. At least some of these have to be shared by all the threads of a process, or the threads could have no shared memory interaction, and thus no function that would not be served by just separate processes running separate main programs. If you need variables that are private to a thread but in some sense global, just put them in the Thread.Closure. They would hardly make sense as declared lexically global variables anyway, since, except for the initial thread provided by the OS/RTS, their lifetime does not cover the entire execution of the main program. So what are they really, and what are they for? And do we even have or use them in Modula-3? It is looking to me like some sort of accommodation for them in cm3cg may be what has undermined m3gdb in the CVS head. Rodney Bates From hosking at cs.purdue.edu Thu Apr 14 23:05:27 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 17:05:27 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: <4DA71682.9040709@lcwb.coop> References: <4DA71682.9040709@lcwb.coop> Message-ID: I think that these were introduced into the C libraries by Jay. On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > I have never understood what "thread local storage" is all about, in particular, > what need it addresses. > > All local variables and parameters are already private to a thread, as consequence > of semantics going back as far as Algol-60. They are allocated space when and > only when a call is executed. That can't happen until there is a thread to > execute the call and it has to have its own private stack where the space is > allocated, or it wouldn't work as a thread, (OK leaving out languages like > very old Fortrans that don't support recursion either.) > > That leaves global variables. At least some of these have to be shared by all > the threads of a process, or the threads could have no shared memory interaction, > and thus no function that would not be served by just separate processes running > separate main programs. > > If you need variables that are private to a thread but in some sense global, > just put them in the Thread.Closure. They would hardly make sense as declared > lexically global variables anyway, since, except for the initial thread provided > by the OS/RTS, their lifetime does not cover the entire execution of the main > program. > > So what are they really, and what are they for? And do we even have or use them in > Modula-3? > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > undermined m3gdb in the CVS head. > > Rodney Bates From jay.krell at cornell.edu Thu Apr 14 23:27:10 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:27:10 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, Message-ID: [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 14 23:32:22 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 14 Apr 2011 21:32:22 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Fri Apr 15 02:28:12 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 14 Apr 2011 20:28:12 -0400 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , Message-ID: There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend. I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > > We've always had thread locals > > ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? > > - Jay > > From: jay.krell at cornell.edu > To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop > Date: Thu, 14 Apr 2011 21:27:10 +0000 > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > [I wrote a long answer here but it seemed to have gotten lost.] > > We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. > On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. > There is nothing in cm3cg about this. > > http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c > Revision 1.155: download - view: text, markup, annotated - select for diffs > Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell > Branches: MAIN > Diff to: previous 1.154: preferred, unified > Changes since revision 1.154: +2 -1 lines > Don't use __thread on Solaris. > It failed to link. > See: > http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console > Revision 1.151: download - view: text, markup, annotated - select for diffs > Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell > Branches: MAIN > Diff to: previous 1.150: preferred, unified > Changes since revision 1.150: +30 -0 lines > Use __thread on Linux and Solaris. > Doesn't compile on Apple. > Segfaults on OpenBSD and NetBSD. > Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. > In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker > or C++ exceptions (C++ backend). > > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Thu, 14 Apr 2011 17:05:27 -0400 > > To: rodney_bates at lcwb.coop > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] Thread local storage? > > > > I think that these were introduced into the C libraries by Jay. > > > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > > > I have never understood what "thread local storage" is all about, in particular, > > > what need it addresses. > > > > > > All local variables and parameters are already private to a thread, as consequence > > > of semantics going back as far as Algol-60. They are allocated space when and > > > only when a call is executed. That can't happen until there is a thread to > > > execute the call and it has to have its own private stack where the space is > > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > > very old Fortrans that don't support recursion either.) > > > > > > That leaves global variables. At least some of these have to be shared by all > > > the threads of a process, or the threads could have no shared memory interaction, > > > and thus no function that would not be served by just separate processes running > > > separate main programs. > > > > > > If you need variables that are private to a thread but in some sense global, > > > just put them in the Thread.Closure. They would hardly make sense as declared > > > lexically global variables anyway, since, except for the initial thread provided > > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > > program. > > > > > > So what are they really, and what are they for? And do we even have or use them in > > > Modula-3? > > > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > > undermined m3gdb in the CVS head. > > > > > > Rodney Bates > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 15 04:46:46 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 15 Apr 2011 02:46:46 +0000 Subject: [M3devel] Thread local storage? In-Reply-To: References: <4DA71682.9040709@lcwb.coop>, , , , , , , Message-ID: __thread is only used in C code. The difference is in the gcc backend within the gcc C/C++ compiler. Not cm3cg. I put in using __thread for Linux. There is some potential for using it elsewhere, but that's all I did. e.g. Solaris sometimes, AIX. All other non-Windows targets use pthread_getspecific/setspecific. It is hidden from any Modula-3 code. It is all in C. m3gdb might might might need to know about it. It could also be that m3gdb gets confused by any C code linked in. Or that m3gdb is old (it is) and is confused by __thread (but again, m3gdb might or might not have a builtin need to often look at these, or maybe just the human debugger wants to see them). Currently no targets support stack walking. I couldn't get SOLgnu/SOLsun's stack walker to work with the gcc 4.5 backend. I really really tried. I tried not optimizing at all. And/or putting volatile everywhere. Tru64 also didn't work despite a bit of an attempt. I probably also tried not optimizing there. I probably didn't try as much as for SOLgnu/SOLsun due to the machine being slower and the platform being in less use. - Jay From: hosking at cs.purdue.edu Date: Thu, 14 Apr 2011 20:28:12 -0400 To: jay.krell at cornell.edu CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? There is a difference between an API call for thread locals (pthread_getspecific/pthread_setspecific) and use of __thread in the gcc backend.I think the API call should work fine. But, as I understand it we are now using __thread for thread exception stacks on targets that don't support stack unwinding (only SOLgnu/SOLsun AFAIK). Antony Hosking | Associate Professor | Computer Science | Purdue University305 N. University Street | West Lafayette | IN 47907 | USAOffice +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 14, 2011, at 5:32 PM, Jay K wrote: > We've always had thread locals ps: user threads do this differently and it could be that m3gdb was never updated for pthreads? - Jay From: jay.krell at cornell.edu To: hosking at cs.purdue.edu; rodney_bates at lcwb.coop Date: Thu, 14 Apr 2011 21:27:10 +0000 CC: m3devel at elegosoft.com Subject: Re: [M3devel] Thread local storage? [I wrote a long answer here but it seemed to have gotten lost.] We've always had thread locals via TlsAlloc/TlsSetValue/TlsGetValue / pthread_key_create, pthread_setspecific, pthread_getspecific. On Linux we now use __thread instead. It might be a little more efficient. But a much better more difficult change will be to eliminate them in favor of underlying stack walkers. There is nothing in cm3cg about this. http://dcvs.elegosoft.com/cgi-bin/cvsweb.cgi/cm3/m3-libs/m3core/src/thread/PTHREAD/ThreadPThreadC.c Revision 1.155: download - view: text, markup, annotated - select for diffs Wed Jan 5 18:32:00 2011 MET (3 months, 1 week ago) by jkrell Branches: MAIN Diff to: previous 1.154: preferred, unified Changes since revision 1.154: +2 -1 lines Don't use __thread on Solaris. It failed to link. See: http://hudson.modula3.com:8080/job/cm3-current-build-SOLsun-opencsw-current9s/166/console Revision 1.151: download - view: text, markup, annotated - select for diffs Tue Dec 28 12:53:16 2010 MET (3 months, 2 weeks ago) by jkrell Branches: MAIN Diff to: previous 1.150: preferred, unified Changes since revision 1.150: +30 -0 lines Use __thread on Linux and Solaris. Doesn't compile on Apple. Segfaults on OpenBSD and NetBSD. Not tested on FreeBSD, AIX, Irix, HP-UX, etc., stick with pthread_get/setspecific. In future hope to eliminate this code anyway, in favor of gcc/libgcc stack walker or C++ exceptions (C++ backend). - Jay > From: hosking at cs.purdue.edu > Date: Thu, 14 Apr 2011 17:05:27 -0400 > To: rodney_bates at lcwb.coop > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] Thread local storage? > > I think that these were introduced into the C libraries by Jay. > > On Apr 14, 2011, at 11:45 AM, Rodney M. Bates wrote: > > > I have never understood what "thread local storage" is all about, in particular, > > what need it addresses. > > > > All local variables and parameters are already private to a thread, as consequence > > of semantics going back as far as Algol-60. They are allocated space when and > > only when a call is executed. That can't happen until there is a thread to > > execute the call and it has to have its own private stack where the space is > > allocated, or it wouldn't work as a thread, (OK leaving out languages like > > very old Fortrans that don't support recursion either.) > > > > That leaves global variables. At least some of these have to be shared by all > > the threads of a process, or the threads could have no shared memory interaction, > > and thus no function that would not be served by just separate processes running > > separate main programs. > > > > If you need variables that are private to a thread but in some sense global, > > just put them in the Thread.Closure. They would hardly make sense as declared > > lexically global variables anyway, since, except for the initial thread provided > > by the OS/RTS, their lifetime does not cover the entire execution of the main > > program. > > > > So what are they really, and what are they for? And do we even have or use them in > > Modula-3? > > > > It is looking to me like some sort of accommodation for them in cm3cg may be what has > > undermined m3gdb in the CVS head. > > > > Rodney Bates > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 15 12:37:37 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:37:37 -0700 Subject: [M3devel] malloc/free from Modula-3? Message-ID: <20110415103737.E73B31A2078@async.async.caltech.edu> Hi m3devel, I'm debugging a problem with some code I have that calls out to C code. It's calling pqlib (PostgreSQL library) and you might expect things to go wrong there, but as it turns out it's dying in CM3's code. The system is AMD64_LINUX (user threading because of the problems with pthreads) and the program is just freezing up hard. No segfault, bus error or anything. It locks up a couple of times a day. What are the rules on calling malloc/free from Modula-3? Stack trace: (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) Here is the code that causes the problem: VAR s := CopyTtoS(query); BEGIN TRY t.res := t.myExec(s,busyWait) FINALLY FreeCopiedS(s) (* stuck here *) END END; The query is a SQL query---t.myExec eventually calls the libpq SQL exec routine (PQexec I think it's called). Since this is already an unsafe module I suppose I could just make an array of CHARs on the stack and pass a pointer to that... Regards, Mika From mika at async.caltech.edu Fri Apr 15 12:46:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 03:46:47 -0700 Subject: [M3devel] malloc/free from Modula-3 (more details) Message-ID: <20110415104647.6AFF81A2078@async.async.caltech.edu> Here is some more info. Note that all my access to the pq library is inside a single giant lock (I've run this code for years and years but not under CM3 and not under AMD64_LINUX or for that matter any kind of Linux). The locking is not likely to be obviously wrong, but of course there could be something subtly wrong. In gdb the program is completely frozen: (gdb) stepi ^C Program received signal SIGINT, Interrupt. 0x00002ac519a9202e in ?? () from /lib/libc.so.6 (gdb) info registers rax 0xfffffffffffffe00 -512 rbx 0x2ac519d059e0 47026030008800 rcx 0xffffffffffffffff -1 rdx 0x2 2 rsi 0x80 128 rdi 0x2ac519d059e0 47026030008800 rbp 0x2ac63b0ef0e0 0x2ac63b0ef0e0 rsp 0x2ac63b0ef028 0x2ac63b0ef028 r8 0x2fe2620 50210336 r9 0x2ac6400008ef 47030965635311 r10 0x0 0 r11 0x302 770 r12 0x411858 4266072 r13 0x1 1 r14 0x2f5b6d0 49657552 r15 0x0 0 rip 0x2ac519a9202e 0x2ac519a9202e eflags 0x202 [ IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 fctrl 0x37f 895 fstat 0x0 0 ftag 0xffff 65535 fiseg 0x0 0 fioff 0x0 0 foseg 0x0 0 fooff 0x0 0 fop 0x0 0 mxcsr 0x1fa0 [ PE IM DM ZM OM UM PM ] (gdb) (gdb) where #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 #2 0x00002ac519a2caab in free () from /lib/libc.so.6 #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 ) at ../src/C/Common/CstdlibC.c:26 #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 ) at ../src/C/Common/M3toC.m3:46 #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 ) at ../src/UnsafeDatabase.m3:230 #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 ) at ../src/UnsafeDatabase.m3:228 #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 ) at ../src/UnsafeDatabase.m3:258 #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 ) at ../src/DesynchronizedDB.m3:126 #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 ) at ../src/DesynchronizedDB.m3:293 #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 ) at ../src/GCOMSOrderManager.m3:596 #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 ) at ../src/GCOMSOrderManager.m3:310 #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 ) at ../src/EuroDriver.m3:1108 #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 ) at ../src/EuroDriver.m3:1545 #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 #17 0x0000000000000000 in ?? () (gdb) disassemble 0x00002ac519a9202e No function contains specified address. (gdb) disassemble 0x00002ac519a2caab Dump of assembler code for function free: 0x00002ac519a2ca40 : mov 0x2d7519(%rip),%rax # 0x2ac519d03f60 0x00002ac519a2ca47 : push %rbx 0x00002ac519a2ca48 : mov %rdi,%r8 0x00002ac519a2ca4b : mov (%rax),%rax 0x00002ac519a2ca4e : test %rax,%rax 0x00002ac519a2ca51 : je 0x2ac519a2ca60 0x00002ac519a2ca53 : mov 0x8(%rsp),%rsi 0x00002ac519a2ca58 : mov %rax,%r11 0x00002ac519a2ca5b : pop %rbx 0x00002ac519a2ca5c : jmpq *%r11 0x00002ac519a2ca5f : nop 0x00002ac519a2ca60 : test %rdi,%rdi 0x00002ac519a2ca63 : je 0x2ac519a2cad2 0x00002ac519a2ca65 : mov -0x8(%rdi),%rax 0x00002ac519a2ca69 : lea -0x10(%rdi),%rdx 0x00002ac519a2ca6d : test $0x2,%al 0x00002ac519a2ca6f : jne 0x2ac519a2cad8 0x00002ac519a2ca71 : test $0x4,%al 0x00002ac519a2ca73 : lea 0x2d8f66(%rip),%rbx # 0x2ac519d059e0 0x00002ac519a2ca7a : je 0x2ac519a2ca86 0x00002ac519a2ca7c : and $0xfffffffffc000000,%rdx 0x00002ac519a2ca83 : mov (%rdx),%rbx 0x00002ac519a2ca86 : mov $0x1,%esi 0x00002ac519a2ca8b : xor %eax,%eax 0x00002ac519a2ca8d : cmpl $0x0,0x2dc594(%rip) # 0x2ac519d09028 0x00002ac519a2ca94 : je 0x2ac519a2caa2 0x00002ac519a2ca96 : lock cmpxchg %esi,(%rbx) 0x00002ac519a2ca9a : jne 0x2ac519a2fb34 0x00002ac519a2caa0 : jmp 0x2ac519a2caab 0x00002ac519a2caa2 : cmpxchg %esi,(%rbx) 0x00002ac519a2caa5 : jne 0x2ac519a2fb34 0x00002ac519a2caab : mov %r8,%rsi 0x00002ac519a2caae : mov %rbx,%rdi 0x00002ac519a2cab1 : callq 0x2ac519a2c2b0 0x00002ac519a2cab6 : cmpl $0x0,0x2dc56b(%rip) # 0x2ac519d09028 0x00002ac519a2cabd : je 0x2ac519a2caca 0x00002ac519a2cabf : lock decl (%rbx) 0x00002ac519a2cac2 : jne 0x2ac519a2fb4f 0x00002ac519a2cac8 : jmp 0x2ac519a2cad2 0x00002ac519a2caca : decl (%rbx) 0x00002ac519a2cacc : jne 0x2ac519a2fb4f 0x00002ac519a2cad2 : pop %rbx 0x00002ac519a2cad3 : retq 0x00002ac519a2cad4 : nopl 0x0(%rax) 0x00002ac519a2cad8 : mov 0x2d97c6(%rip),%ecx # 0x2ac519d062a4 0x00002ac519a2cade : test %ecx,%ecx 0x00002ac519a2cae0 : jne 0x2ac519a2cb10 0x00002ac519a2cae2 : cmp 0x2d97a7(%rip),%rax # 0x2ac519d06290 0x00002ac519a2cae9 : jbe 0x2ac519a2cb10 0x00002ac519a2caeb : cmp $0x2000000,%rax 0x00002ac519a2caf1 : ja 0x2ac519a2cb10 0x00002ac519a2caf3 : and $0xfffffffffffffff8,%rax 0x00002ac519a2caf7 : mov %rax,0x2d9792(%rip) # 0x2ac519d06290 0x00002ac519a2cafe : add %rax,%rax 0x00002ac519a2cb01 : mov %rax,0x2d9778(%rip) # 0x2ac519d06280 0x00002ac519a2cb08 : nopl 0x0(%rax,%rax,1) ---Type to continue, or q to quit--- 0x00002ac519a2cb10 : pop %rbx 0x00002ac519a2cb11 : mov %rdx,%rdi 0x00002ac519a2cb14 : jmpq 0x2ac519a2bc40 End of assembler dump. Mika From hosking at cs.purdue.edu Fri Apr 15 17:26:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Fri, 15 Apr 2011 11:26:15 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110415103737.E73B31A2078@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > Hi m3devel, > > I'm debugging a problem with some code I have that calls out to C > code. It's calling pqlib (PostgreSQL library) and you might expect > things to go wrong there, but as it turns out it's dying in CM3's code. > > The system is AMD64_LINUX (user threading because of the problems with > pthreads) and the program is just freezing up hard. No segfault, > bus error or anything. It locks up a couple of times a day. > > What are the rules on calling malloc/free from Modula-3? > > Stack trace: > > (gdb) where > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > ) at ../src/C/Common/CstdlibC.c:26 > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > ) at ../src/C/Common/M3toC.m3:46 > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > ) at ../src/UnsafeDatabase.m3:230 > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > ) at ../src/UnsafeDatabase.m3:228 > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > ) at ../src/UnsafeDatabase.m3:258 > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > ) at ../src/DesynchronizedDB.m3:126 > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > ) at ../src/DesynchronizedDB.m3:293 > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > ) at ../src/GCOMSOrderManager.m3:596 > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > ) at ../src/GCOMSOrderManager.m3:310 > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > ) at ../src/EuroDriver.m3:1108 > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > ) at ../src/EuroDriver.m3:1545 > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > #17 0x0000000000000000 in ?? () > (gdb) > > Here is the code that causes the problem: > > VAR > s := CopyTtoS(query); > BEGIN > TRY > t.res := t.myExec(s,busyWait) > FINALLY > FreeCopiedS(s) (* stuck here *) > END > END; > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > routine (PQexec I think it's called). > > Since this is already an unsafe module I suppose I could just make an > array of CHARs on the stack and pass a pointer to that... > > > Regards, > Mika From mika at async.caltech.edu Fri Apr 15 21:03:49 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 15 Apr 2011 12:03:49 -0700 Subject: [M3devel] time conversion problems : Date.ToTime on AMD64_LINUX In-Reply-To: <926161.28913.qm@web29713.mail.ird.yahoo.com> References: <926161.28913.qm@web29713.mail.ird.yahoo.com> Message-ID: <20110415190349.E4F531A2078@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >although my understanding of the problem is not that good, I think there ar= >e two routes for this abstract way one could try to prove the correctness o= >f some implementation by setting directly instrumention code to try to see = >for your self dynamically if you may that or what you want to see: >http://www.youtube.com/watch?v=3DzIgu9q0vVc0#t=3D2m06s > >This could would run with just 3 missing modules from this sources besides = >their examples, maybe another time the HP guys allow us to see their works = >on that: >http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D= >/ > >This package is missing from public repositories Elego too as I know, thoug= >h they were in (see bottom links dead as of now, but, ... maybe you can loo= >k for yourselves): >http://www.cs.arizona.edu/~collberg/Research/Modula-3/ > >I know some ESC/Modula-3 tries to make some inferencing of invariants, but = >as I said it dones't mean this is totally computable, im fact in general it= >s isn't, but it would be good to try. > >Essentially where are the codes for showing that, or are you implementing t= >his as private code? The problem is in DatePosix.m3 . To reproduce, set your TZ environment variable to Europe/London and try to convert a Date of 1970-01-01 @ 00:00:00 in the time zone UTC to a Time.T. You should get 0; you will get -3600. Mika > >Thanks in advance > >--- El lun, 11/4/11, Mika Nystrom escribi=F3: > From dragisha at m3w.org Sun Apr 17 16:05:58 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 16:05:58 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? Message-ID: I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? TIA, dd From hosking at cs.purdue.edu Sun Apr 17 17:25:04 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 11:25:04 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: Message-ID: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? > > TIA, > dd > From dabenavidesd at yahoo.es Sun Apr 17 17:31:54 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 16:31:54 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: Message-ID: <274332.77724.qm@web29720.mail.ird.yahoo.com> Hi all: it's possible to get yet another implementation of it, perhaps when we have the ESC we will be able to check it for Lock level annotations, earlier there was an implementation: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 Is important to keep in mind this await construct annotation may be verified as the one in Locking Level, it would be interesting to see whether this means one can execute commands without locking, is in some way specifying locks underneath. If you want to ask, if their statements is sound it would be good to ask if they might have a copy of the implementation If this is better to check for checks under Pthreads module would be a good idea to use directly their semantics. Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" > Fecha: domingo, 17 de abril, 2011 09:05 > I am still puzzled with this... What > exactly do we have now? Do we have a version where things > work, or something? Or we don't have threads for cm3 > anymore? > > TIA, > dd > > From dragisha at m3w.org Sun Apr 17 18:10:33 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 18:10:33 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. > > On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: > >> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >> >> TIA, >> dd >> > From dabenavidesd at yahoo.es Sun Apr 17 18:04:08 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 17:04:08 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <274332.77724.qm@web29720.mail.ird.yahoo.com> Message-ID: <617521.89071.qm@web29714.mail.ird.yahoo.com> Hi all: And yet to more on this, the first on the implementation of mutex abstraction theory based on await construct: http://www.cs.rug.nl/~wim/pub/whh233.ps.gz And better than that I guess, this with Useful Guarded Commands of Dijkstra's language, this is really a good idea to use it inside the ESC checker since it's its internal AST rep: http://www.cs.rug.nl/~wim/onderwijs/concurrency/dikt.pdf.gz Thanks in advance: --- El dom, 17/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "m3devel" , "Dragi?a Duri?" > Fecha: domingo, 17 de abril, 2011 10:31 > Hi all: > it's possible to get yet another implementation of it, > perhaps when we have the ESC we will be able to check it for > Lock level annotations, earlier there was an > implementation: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1092 > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.166 > > Is important to keep in mind this await construct > annotation may be verified as the one in Locking Level, it > would be interesting to see whether this means one can > execute commands without locking, is in some way specifying > locks underneath. > > If you want to ask, if their statements is sound it would > be good to ask if they might have a copy of the > implementation > > If this is better to check for checks under Pthreads module > would be a good idea to use directly their semantics. > > Thanks in advance > > --- El dom, 17/4/11, Dragi?a Duri? > escribi?: > > > De: Dragi?a Duri? > > Asunto: [M3devel] State of threads? pthreads not safe > to use on some platforms? > > Para: "m3devel" > > Fecha: domingo, 17 de abril, 2011 09:05 > > I am still puzzled with this... What > > exactly do we have now? Do we have a version where > things > > work, or something? Or we don't have threads for cm3 > > anymore? > > > > TIA, > > dd > > > > > From hosking at cs.purdue.edu Sun Apr 17 18:43:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 12:43:16 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> Message-ID: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> So the question is what has changed. I know threads were functional and stable at one point. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 17, 2011, at 12:10 PM, Dragi?a Duri? wrote: > Program I had problem with is fork-less... Simple and pure Modula-3 and it used to work - it's what puzzless me most... It is a platform I only use for single application running and another building (for 32bit version)... Nothing cm3 related changed in ages.... > > On Apr 17, 2011, at 5:25 PM, Tony Hosking wrote: > >> Things should work, modulo some of the corner cases for programs using fork. Mika has a test case for this, that I encourage exploration with. I am snowed and won't be able to do so until the summer comes around. >> >> On Apr 17, 2011, at 10:05 AM, Dragi?a Duri? wrote: >> >>> I am still puzzled with this... What exactly do we have now? Do we have a version where things work, or something? Or we don't have threads for cm3 anymore? >>> >>> TIA, >>> dd >>> >> > From hendrik at topoi.pooq.com Sun Apr 17 19:34:55 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Sun, 17 Apr 2011 13:34:55 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> Message-ID: <20110417173455.GB11082@topoi.pooq.com> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: > So the question is what has changed. > I know threads were functional and stable at one point. Was that back when they were green threads? Has our use of posix threads ever been stable? Are green threads still available? -- hendrik From hosking at cs.purdue.edu Sun Apr 17 20:13:01 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sun, 17 Apr 2011 14:13:01 -0400 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <20110417173455.GB11082@topoi.pooq.com> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >> So the question is what has changed. >> I know threads were functional and stable at one point. > > Was that back when they were green threads? User threads have historically been pretty stable, and I believe still are. Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > Has our use of posix threads ever been stable? > Are green threads still available? > > -- hendrik From dragisha at m3w.org Sun Apr 17 20:18:04 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Sun, 17 Apr 2011 20:18:04 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> But, what recent change when my now problematic platform is on 5.8.6....? On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From dabenavidesd at yahoo.es Sun Apr 17 23:21:47 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 17 Apr 2011 22:21:47 +0100 (BST) Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <946881.5570.qm@web29717.mail.ird.yahoo.com> Hi all: it's good to check if at all the platform per target and per threading too, SMP/HT, x86 or AMD64. I'm suspecting more than anything else this really affects specially SMP/HT, but specially not missing that information, perhaps even historically we can't assure that, at least that's how I see it without it, what do you think? Thanks in advance --- El dom, 17/4/11, Dragi?a Duri? escribi?: > De: Dragi?a Duri? > Asunto: Re: [M3devel] State of threads? pthreads not safe to use on some platforms? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: domingo, 17 de abril, 2011 13:18 > But, what recent change when my now > problematic platform is on 5.8.6....? > > On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > > > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > > > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony > Hosking wrote: > >>> So the question is what has changed. > >>> I know threads were functional and stable at > one point. > >> > >> Was that back when they were green threads? > > > > User threads have historically been pretty stable, and > I believe still are. > > Similarly, our use of pthreads was stable at one point > but there have been recent changes that appear to have > degraded them. > > > >> Has our use of posix threads ever been stable? > >> Are green threads still available? > >> > >> -- hendrik > > > > From dragisha at m3w.org Mon Apr 18 14:07:40 2011 From: dragisha at m3w.org (=?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?=) Date: Mon, 18 Apr 2011 14:07:40 +0200 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> Message-ID: <7D82E5E5-EC36-4254-A4A7-782A230320FD@m3w.org> User threads were stable, but they had problems of their own... I met some of them when I implemented first pthread support - user threads switching is in order, and pthread switching is out of order. m3pp was failing because of that. On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > > On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: > >> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>> So the question is what has changed. >>> I know threads were functional and stable at one point. >> >> Was that back when they were green threads? > > User threads have historically been pretty stable, and I believe still are. > Similarly, our use of pthreads was stable at one point but there have been recent changes that appear to have degraded them. > >> Has our use of posix threads ever been stable? >> Are green threads still available? >> >> -- hendrik > From mika at async.caltech.edu Mon Apr 18 18:10:28 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:10:28 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu> Message-ID: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Tony Hosking writes: >I suspect the problem is re-entrance to the library. Should you be = >disabling user thread switching somewhere? I suspect the same. That's why I asked what the "rules" are for calling malloc and free. I notice that the m3core code has Scheduler.DisableSwitching around all calls to malloc but not around calls to free. Why would one be unsafe and the other safe? It is always free that my program hangs up on by the way. Surrounding all "suspicious" calls in my code with Scheduler.DisableSwitching doesn't solve the problem, as far as I can tell. I am further suspecting that my problem is related to my having linked the program with -pthread. I am also guessing that this is the same reason you can't link FreeBSD user threads programs with -pthread: namely that if you go reentrantly into the pthread malloc library some code assumes the two invocations are from separate threads and locks up if they are from the same thread. That's my guess anyhow. But that doesn't explain why I've never seen any malloc/free problems on FreeBSD without -pthread. (I don't think I have, anyhow.) Mika From hosking at cs.purdue.edu Mon Apr 18 18:30:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 12:30:45 -0400 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu> <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Yes, I think you should avoid -pthread. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > Tony Hosking writes: >> I suspect the problem is re-entrance to the library. Should you be = >> disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the "rules" are > for calling malloc and free. I notice that the m3core code has > Scheduler.DisableSwitching around all calls to malloc but not around calls > to free. Why would one be unsafe and the other safe? It is always free > that my program hangs up on by the way. Surrounding all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my having linked the > program with -pthread. I am also guessing that this is the same reason > you can't link FreeBSD user threads programs with -pthread: namely that if > you go reentrantly into the pthread malloc library some code assumes > the two invocations are from separate threads and locks up if they are from > the same thread. That's my guess anyhow. But that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without -pthread. > (I don't think I have, anyhow.) > > Mika From mika at async.caltech.edu Mon Apr 18 18:42:39 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 09:42:39 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> Message-ID: <20110418164239.84CEB1A207A@async.async.caltech.edu> Tony Hosking writes: >Well, that is reassuring news. >If the problems are isolated to threading then they can be resolved easily. > >On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: > >> >> Remember that for mission-critical applications there's always user >> threading.. I find that the system (up to and including the CVS head >> and on both amd64 and i386 flavors of Linux as well as every other OS I >> have tried) is very very stable with user threading. I'm eager to use >> pthreads again of course, for all sorts of reasons... >> >> Mika Actually I'm afraid I have to take this back. After digging a bit here, I *think* user threads are OK with the proviso that one not link with -pthread. But I'm not sure. Unfortunately, user threads require -pthread on most systems because of pthread_atfork. I asked Jay what pthread_atfork is used for *under user threads* but he never answered my question. I had understood that pthread_atfork is used in the pthreads implementation to simulate the old behavior of user threads and that it wasn't needed under user threads... I'm also not sure how to change RTProcessC.c so that it picks up pthread_atfork exactly when user threading has been specified. (Programs won't link with pthread_atfork in them without -pthread.) In any case I am pretty sure that linking with -pthread picks up versions of malloc and free that really don't like being called reentrantly---and by "don't like" I mean the worst kind of crash: the application locks up hard in some sort of spin lock. A segfault would be preferable by far. And m3core has a lot of unprotected calls to free in it. There may be some really nasty bugs here. They are nasty because they kill reliability, kill liveness, and can't really be programmed around because they are in such a vital subsystem (namely threading). I would go back to PM3 if I could but I think I'm stuck with CM3 now. Mika P.S. as far as the pthreads bugs go I think it's still a possibility the bugs are in the garbage collector? The user threads m3core disables the garbage collector at different points from where the pthreads m3core does, no? From hosking at cs.purdue.edu Mon Apr 18 19:05:46 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 13:05:46 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418164239.84CEB1A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> Message-ID: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> On Apr 18, 2011, at 12:42 PM, Mika Nystrom wrote: > Tony Hosking writes: >> Well, that is reassuring news. >> If the problems are isolated to threading then they can be resolved easily. >> >> On Apr 7, 2011, at 3:02 PM, Mika Nystrom wrote: >> >>> >>> Remember that for mission-critical applications there's always user >>> threading.. I find that the system (up to and including the CVS head >>> and on both amd64 and i386 flavors of Linux as well as every other OS I >>> have tried) is very very stable with user threading. I'm eager to use >>> pthreads again of course, for all sorts of reasons... >>> >>> Mika > > > Actually I'm afraid I have to take this back. After digging a bit here, > I *think* user threads are OK with the proviso that one not link with > -pthread. But I'm not sure. > > Unfortunately, user threads require -pthread on most systems because > of pthread_atfork. I asked Jay what pthread_atfork is used for *under > user threads* but he never answered my question. I had understood > that pthread_atfork is used in the pthreads implementation to simulate > the old behavior of user threads and that it wasn't needed under user > threads... I'm also not sure how to change RTProcessC.c so that it > picks up pthread_atfork exactly when user threading has been specified. > (Programs won't link with pthread_atfork in them without -pthread.) pthread_atfork should not be needed under user threads. > > In any case I am pretty sure that linking with -pthread picks up versions > of malloc and free that really don't like being called reentrantly---and > by "don't like" I mean the worst kind of crash: the application locks up > hard in some sort of spin lock. A segfault would be preferable by far. > And m3core has a lot of unprotected calls to free in it. > > There may be some really nasty bugs here. They are nasty because they > kill reliability, kill liveness, and can't really be programmed around > because they are in such a vital subsystem (namely threading). I would > go back to PM3 if I could but I think I'm stuck with CM3 now. > > Mika > > P.S. as far as the pthreads bugs go I think it's still a possibility the > bugs are in the garbage collector? Not so much bugs in GC as bugs in thread interactions with GC perhaps. The GC itself has been stable for a long time. > The user threads m3core disables > the garbage collector at different points from where the pthreads m3core > does, no? From dabenavidesd at yahoo.es Mon Apr 18 19:17:35 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 18:17:35 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418161028.8DA9C1A207A@async.async.caltech.edu> Message-ID: <886452.94715.qm@web29705.mail.ird.yahoo.com> Hi all: the only thing you need to keep control as of CM3 is concerned is the modula-3 free calls equals the number of news, that is, if not is runtime error, but it isn't vice versa, if you broke this then you are obviously unsafely corrupted, but as far as the process is concerned it will be growing in time the heap until termination. If this is exacerbated then you will run out of memory either in Modula-3 or C which is dangerous, so this tells you why the scheduler must be aware of this calls, at least that's my rationale. You are right I think there is a unixdb client program that crashes that can't be build in cm3 as it asks to provide their external implementation function calls so you can't declare external calls on your own, you need like compile it too, which is very strange, though my memories about this are very diffuse. Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Tony Hosking" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 11:10 > > Tony Hosking writes: > >I suspect the problem is re-entrance to the library. > Should you be = > >disabling user thread switching somewhere? > > I suspect the same. That's why I asked what the > "rules" are > for calling malloc and free. I notice that the m3core > code has > Scheduler.DisableSwitching around all calls to malloc but > not around calls > to free. Why would one be unsafe and the other > safe? It is always free > that my program hangs up on by the way. Surrounding > all "suspicious" calls > in my code with Scheduler.DisableSwitching doesn't solve > the problem, as > far as I can tell. > > I am further suspecting that my problem is related to my > having linked the > program with -pthread. I am also guessing that this > is the same reason > you can't link FreeBSD user threads programs with -pthread: > namely that if > you go reentrantly into the pthread malloc library some > code assumes > the two invocations are from separate threads and locks up > if they are from > the same thread. That's my guess anyhow. But > that doesn't explain why > I've never seen any malloc/free problems on FreeBSD without > -pthread. > (I don't think I have, anyhow.) > > Mika > > From mika at async.caltech.edu Mon Apr 18 20:37:20 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:37:20 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> Message-ID: <20110418183720.E4C311A207A@async.async.caltech.edu> I don't think the pthreads implementation has ever been 100% correct. If it has, please let me know which CVS date to check out!!! (I use a PM3 that is almost ten years old simply because this stuff works and am happy to work with ANYTHING newer that works.) As far as user ("green") threads goes, I think they *may* be right if you are careful not to link with -pthread. This means you have to change RTThreadC.c (remove pthread_atfork) or your program won't link. (Please don't call pthreads "Posix" threads on this list: I know it is correct but it is confusing because Modula-3's user threads implementation for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 usage predates IEEE/ISO standardization of threads so there was no confusion at the time the name was introduced.) Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >But, what recent change when my now problematic platform is on = >5.8.6....? > >On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: > >>=20 >> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>=20 >>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>> So the question is what has changed. >>>> I know threads were functional and stable at one point. >>>=20 >>> Was that back when they were green threads? >>=20 >> User threads have historically been pretty stable, and I believe still = >are. >> Similarly, our use of pthreads was stable at one point but there have = >been recent changes that appear to have degraded them. >>=20 >>> Has our use of posix threads ever been stable? >>> Are green threads still available? >>>=20 >>> -- hendrik >>=20 From mika at async.caltech.edu Mon Apr 18 20:45:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:45:40 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> Message-ID: <20110418184540.C2D801A2078@async.async.caltech.edu> Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From mika at async.caltech.edu Mon Apr 18 20:47:40 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 11:47:40 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <886452.94715.qm@web29705.mail.ird.yahoo.com> References: <886452.94715.qm@web29705.mail.ird.yahoo.com> Message-ID: <20110418184740.D20261A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >the only thing you need to keep control as of CM3 is concerned is the modul= >a-3 free calls equals the number of news, that is, if not is runtime error,= Not exactly. You can't necessarily call malloc/free from multiple threads. This is a big problem for Modula-3 programs that call out to nontrivial C code. (Why don't we wrap malloc/free ourselves in a special library? Linker problems I suppose... and I also suppose the effort should rather be on getting pthreads working again than messing around with problems with user threads...) Mika From mika at async.caltech.edu Mon Apr 18 21:01:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:01:44 -0700 Subject: [M3devel] State of threads? pthreads not safe to use on some platforms? In-Reply-To: <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> References: <310D9CC9-2165-4414-A2FF-0EAD2483BB4E@cs.purdue.edu> <8A8EBFDC-F2D4-41A8-9CB9-E20FF35D3C6A@cs.purdue.edu> <20110417173455.GB11082@topoi.pooq.com> <3EED6C6B-597E-44CF-89F4-8C27E72C1150@m3w.org> <20110418183720.E4C311A207A@async.async.caltech.edu> <30BDA25F-5778-4F7A-96F1-82C37C061BB5@m3w.org> Message-ID: <20110418190144.406E41A207A@async.async.caltech.edu> If you have a copy, can you try the thread tester program on it? Mika =?utf-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes: >First implementation of pthreads for m3 was made for pm3 in December of = >2004... To the best of my knowledge, it was correct. As long as you do = >not define correct like "repetitive behaviour of green threads". I used = >it to run DC++ hub I wrote as a pet project for stress testign it. = >Worked well. > >On Apr 18, 2011, at 8:37 PM, Mika Nystrom wrote: > >> I don't think the pthreads implementation has ever been 100% correct. >>=20 >> If it has, please let me know which CVS date to check out!!! (I use a >> PM3 that is almost ten years old simply because this stuff works and = >am >> happy to work with ANYTHING newer that works.) >>=20 >> As far as user ("green") threads goes, I think they *may* be right if = >you >> are careful not to link with -pthread. This means you have to change = >RTThreadC.c >> (remove pthread_atfork) or your program won't link. >>=20 >> (Please don't call pthreads "Posix" threads on this list: I know it is >> correct but it is confusing because Modula-3's user threads = >implementation >> for Unix is called "POSIX"---i.e., as opposed to "WIN32". Modula-3 = >usage=20 >> predates IEEE/ISO standardization of threads so there was no confusion = >at=20 >> the time the name was introduced.) >>=20 >> Mika >>=20 >> =3D?utf-8?Q?Dragi=3DC5=3DA1a_Duri=3DC4=3D87?=3D writes: >>> But, what recent change when my now problematic platform is on =3D >>> 5.8.6....? >>>=20 >>> On Apr 17, 2011, at 8:13 PM, Tony Hosking wrote: >>>=20 >>>> =3D20 >>>> On Apr 17, 2011, at 1:34 PM, Hendrik Boom wrote: >>>> =3D20 >>>>> On Sun, Apr 17, 2011 at 12:43:16PM -0400, Tony Hosking wrote: >>>>>> So the question is what has changed. >>>>>> I know threads were functional and stable at one point. >>>>> =3D20 >>>>> Was that back when they were green threads? >>>> =3D20 >>>> User threads have historically been pretty stable, and I believe = >still =3D >>> are. >>>> Similarly, our use of pthreads was stable at one point but there = >have =3D >>> been recent changes that appear to have degraded them. >>>> =3D20 >>>>> Has our use of posix threads ever been stable? >>>>> Are green threads still available? >>>>> =3D20 >>>>> -- hendrik >>>> =3D20 From dabenavidesd at yahoo.es Mon Apr 18 20:57:01 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 19:57:01 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418184740.D20261A207A@async.async.caltech.edu> Message-ID: <564805.90054.qm@web29714.mail.ird.yahoo.com> Hi all: ok, but are you telling this about in user threads or working ill or all threads and pthreads? Thanks in advance --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 13:47 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >the only thing you need to keep control as of CM3 is > concerned is the modul= > >a-3 free calls equals the number of news, that is, if > not is runtime error,= > > Not exactly. You can't necessarily call malloc/free > from multiple threads. > This is a big problem for Modula-3 programs that call out > to nontrivial C code. > > (Why don't we wrap malloc/free ourselves in a special > library? Linker problems > I suppose... and I also suppose the effort should rather be > on getting pthreads > working again than messing around with problems with user > threads...) > > Mika > From mika at async.caltech.edu Mon Apr 18 21:05:55 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 12:05:55 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <564805.90054.qm@web29714.mail.ird.yahoo.com> References: <564805.90054.qm@web29714.mail.ird.yahoo.com> Message-ID: <20110418190555.526411A207A@async.async.caltech.edu> "Daniel Alejandro Benavides D." writes: >Hi all: >ok, but are you telling this about in user threads or working ill or all th= >reads and pthreads? >Thanks in advance > I don't know exactly. There are certainly bad combinations of threading implementations and memory allocators. And I'm sure it may be system dependent as well. In any case, linking user threads with the -pthreads version of libc is probably bad. Calling malloc willy-nilly from multiple user threads is probably bad. Calling free, I don't know. Pthreads currently don't work so maybe malloc works there but that doesn't help very much if the system is anyhow unstable. Mika >--- El lun, 18/4/11, Mika Nystrom escribi=F3: > >> De: Mika Nystrom >> Asunto: Re: [M3devel] malloc/free from Modula-3? >> Para: "Daniel Alejandro Benavides D." >> CC: m3devel at elegosoft.com >> Fecha: lunes, 18 de abril, 2011 13:47 >> "Daniel Alejandro Benavides D." >> writes: >> >Hi all: >> >the only thing you need to keep control as of CM3 is >> concerned is the modul=3D >> >a-3 free calls equals the number of news, that is, if >> not is runtime error,=3D >>=20 >> Not exactly. You can't necessarily call malloc/free >> from multiple threads. >> This is a big problem for Modula-3 programs that call out >> to nontrivial C code. >>=20 >> (Why don't we wrap malloc/free ourselves in a special >> library? Linker problems >> I suppose... and I also suppose the effort should rather be >> on getting pthreads >> working again than messing around with problems with user >> threads...) >>=20 >> Mika >> From dabenavidesd at yahoo.es Mon Apr 18 22:18:49 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 21:18:49 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110418190555.526411A207A@async.async.caltech.edu> Message-ID: <967344.66270.qm@web29718.mail.ird.yahoo.com> Hi all: yes pthread allocation must keep their pthread under system constraints, but per thread user space the outer space is limited, then that's why perhaps RT is breaking due system behaviour not by its own consideration, if I may say so, so perhaps this is just the unsafe pattern break codes showed, since the implementation is responsible for avoiding this flaws but assuming they satisfies their restrictions, which is modularly sound, but if not then somebody either doesn't guarantee our assumptions, or can't meet their specifications and ours too so don't assume they satisfy theirs own ones too. Then in fact would explain why there are many crash patterns around which is very system dependent not us (that's why I favour one implementation per target source, mixing both is like meeting their specifications and so like if it would be our goal, correct if I'm wrong here please). Anything else is just falling apart of breakages that somehow rather than literally must be addressed by their own, if its so breaking the code they should, or us adapt to it literally. --- El lun, 18/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Daniel Alejandro Benavides D." > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 14:05 > "Daniel Alejandro Benavides D." > writes: > >Hi all: > >ok, but are you telling this about in user threads or > working ill or all th= > >reads and pthreads? > >Thanks in advance > > > > I don't know exactly. There are certainly bad > combinations of threading > implementations and memory allocators. And I'm sure > it may be system > dependent as well. In any case, linking user threads > with the -pthreads > version of libc is probably bad. Calling malloc > willy-nilly from multiple > user threads is probably bad. Calling free, I don't > know. Pthreads currently > don't work so maybe malloc works there but that doesn't > help very much if > the system is anyhow unstable. > > Mika > > >--- El lun, 18/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] malloc/free from Modula-3? > >> Para: "Daniel Alejandro Benavides D." > >> CC: m3devel at elegosoft.com > >> Fecha: lunes, 18 de abril, 2011 13:47 > >> "Daniel Alejandro Benavides D." > >> writes: > >> >Hi all: > >> >the only thing you need to keep control as of > CM3 is > >> concerned is the modul=3D > >> >a-3 free calls equals the number of news, that > is, if > >> not is runtime error,=3D > >>=20 > >> Not exactly. You can't necessarily call > malloc/free > >> from multiple threads. > >> This is a big problem for Modula-3 programs that > call out > >> to nontrivial C code. > >>=20 > >> (Why don't we wrap malloc/free ourselves in a > special > >> library? Linker problems > >> I suppose... and I also suppose the effort should > rather be > >> on getting pthreads > >> working again than messing around with problems > with user > >> threads...) > >>=20 > >> Mika > >> > From dabenavidesd at yahoo.es Mon Apr 18 23:01:10 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 22:01:10 +0100 (BST) Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <967344.66270.qm@web29718.mail.ird.yahoo.com> Message-ID: <411737.17355.qm@web29709.mail.ird.yahoo.com> Hi all: this unsound trend is what it makes software development uncontrolled, so many unstable codes, it has an effect as Dr Nelson said software quality and perhaps more dangerous ever more, at least with Modula-3 you can keep it under certain control you know are restricted but this is the result of that, their codes are broken then why one need to fix them if it's worse than before. What a mess if so. Thanks in advance --- El lun, 18/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: Re: [M3devel] malloc/free from Modula-3? > Para: "Mika Nystrom" > CC: m3devel at elegosoft.com > Fecha: lunes, 18 de abril, 2011 15:18 > Hi all: > yes pthread allocation must keep their pthread under system > constraints, but per thread user space the outer space is > limited, then that's why perhaps RT is breaking due system > behaviour not by its own consideration, if I may say so, so > perhaps this is just the unsafe pattern break codes showed, > since the implementation is responsible for avoiding this > flaws but assuming they satisfies their restrictions, which > is modularly sound, but if not then somebody either doesn't > guarantee our assumptions, or can't meet their > specifications and ours too so don't assume they satisfy > theirs own ones too. > Then in fact would explain why there are many crash > patterns around which is very system dependent not us > (that's why I favour one implementation per target source, > mixing both is like meeting their specifications and so like > if it would be our goal, correct if I'm wrong here please). > Anything else is just falling apart of breakages that > somehow rather than literally must be addressed by their > own, if its so breaking the code they should, or us adapt to > it literally. > > --- El lun, 18/4/11, Mika Nystrom > escribi?: > > > De: Mika Nystrom > > Asunto: Re: [M3devel] malloc/free from Modula-3? > > Para: "Daniel Alejandro Benavides D." > > CC: m3devel at elegosoft.com > > Fecha: lunes, 18 de abril, 2011 14:05 > > "Daniel Alejandro Benavides D." > > writes: > > >Hi all: > > >ok, but are you telling this about in user threads > or > > working ill or all th= > > >reads and pthreads? > > >Thanks in advance > > > > > > > I don't know exactly. There are certainly bad > > combinations of threading > > implementations and memory allocators. And I'm > sure > > it may be system > > dependent as well. In any case, linking user > threads > > with the -pthreads > > version of libc is probably bad. Calling malloc > > willy-nilly from multiple > > user threads is probably bad. Calling free, I > don't > > know. Pthreads currently > > don't work so maybe malloc works there but that > doesn't > > help very much if > > the system is anyhow unstable. > > > > Mika > > > > >--- El lun, 18/4/11, Mika Nystrom > > escribi=F3: > > > > > >> De: Mika Nystrom > > >> Asunto: Re: [M3devel] malloc/free from > Modula-3? > > >> Para: "Daniel Alejandro Benavides D." > > >> CC: m3devel at elegosoft.com > > >> Fecha: lunes, 18 de abril, 2011 13:47 > > >> "Daniel Alejandro Benavides D." > > >> writes: > > >> >Hi all: > > >> >the only thing you need to keep control > as of > > CM3 is > > >> concerned is the modul=3D > > >> >a-3 free calls equals the number of news, > that > > is, if > > >> not is runtime error,=3D > > >>=20 > > >> Not exactly. You can't necessarily > call > > malloc/free > > >> from multiple threads. > > >> This is a big problem for Modula-3 programs > that > > call out > > >> to nontrivial C code. > > >>=20 > > >> (Why don't we wrap malloc/free ourselves in > a > > special > > >> library? Linker problems > > >> I suppose... and I also suppose the effort > should > > rather be > > >> on getting pthreads > > >> working again than messing around with > problems > > with user > > >> threads...) > > >>=20 > > >> Mika > > >> > > > From rcolebur at SCIRES.COM Mon Apr 18 23:35:40 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Mon, 18 Apr 2011 17:35:40 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: -pthreads -user threads -Win32 native threads -green threads -etc. I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Monday, April 18, 2011 2:46 PM To: Tony Hosking Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Tony Hosking writes: ... >pthread_atfork should not be needed under user threads. ... The following code from RTProcessC.c ensures that it is neeeded on every Unix except FreeBSD before 6. The comment is from the checked-in source file. /* NOTE: Even userthreads now depends * on availability of pthreads. * This can be fixed if need be. */ INTEGER __cdecl RTProcess__RegisterForkHandlers( ForkHandler prepare, ForkHandler parent, ForkHandler child) { /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. * VMS lacks pthread_atfork? Would be good to use autoconf. * Win32 lacks pthread_atfork and fork. OK. * * As well, for all Posix systems, we could implement * atfork ourselves, as long as we provide a fork() * wrapper that code uses. */ #if defined(_WIN32) \ || defined(__vms) \ || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) return 0; #else while (1) { int i = pthread_atfork(prepare, parent, child); if (i != EAGAIN) return i; sleep(0); } #endif } From hosking at cs.purdue.edu Tue Apr 19 00:11:26 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Mon, 18 Apr 2011 18:11:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110418184540.C2D801A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org> <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu> <20110407190219.C00E31A2078@async.async.caltech.edu> <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu> <20110418164239.84CEB1A207A@async.async.caltech.edu> <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu> <20110418184540.C2D801A2078@async.async.caltech.edu> Message-ID: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > Tony Hosking writes: > ... >> pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } From dabenavidesd at yahoo.es Tue Apr 19 00:46:52 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Mon, 18 Apr 2011 23:46:52 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <937894.40594.qm@web29704.mail.ird.yahoo.com> Hi all: I think if each thread handle is RT specific form then you need to be aware of that before wanting to change system and platform, rather than of implementation, so each specific module procedure must be not UNSAFE but platform ready for its available platforms, now if something is specified implementation dependent must have a CM3 suffix for that purpose so everybody keeps and eye on its implementation aspects, so each RTThreadCM3.i3 is clearer to group as they are intended to serve as are RTCollectorSRC.i3 to RTCollector.i3 to mean is not specific to CM3 so to have implementations e.g SOLsun user threads, LINUXLIBC6 x86 user threads (implemented for Multimedia Modula-3, called MM3, etc) but anything else like for unique for Windows, SPIN, is like normal like RTException.i3 and if something is supported just for DEC Unix for instance, then it is better to say RTExceptionOSF.i3 etc in that way if so and not CM3 or anything else. Thanks in advance --- El lun, 18/4/11, Coleburn, Randy escribi?: > De: Coleburn, Randy > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "m3devel at elegosoft.com" > Fecha: lunes, 18 de abril, 2011 16:35 > I think it would be good at this > point to clarify for folks on the m3devel list what is meant > by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" > threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still > using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not > behaving well. This situation concerns me because > thread safety is something I've always relied on Modula-3 to > provide. > > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel > 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user > threads. > ... > > The following code from RTProcessC.c ensures that it is > neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in > source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to > use autoconf. > * VMS lacks pthread_atfork? Would be good to use > autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, > parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > > From jay.krell at cornell.edu Tue Apr 19 01:20:57 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:20:57 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: And "yes" you need to disable switching around free, the same as malloc. And calloc. And realloc. The disable switching *might* be at a lower level than you realize, so *might* not be missing. That is, it might be in our C wrappers. I can check later. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 12:30:45 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Yes, I think you should avoid -pthread. > > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 18, 2011, at 12:10 PM, Mika Nystrom wrote: > > > > > Tony Hosking writes: > >> I suspect the problem is re-entrance to the library. Should you be = > >> disabling user thread switching somewhere? > > > > I suspect the same. That's why I asked what the "rules" are > > for calling malloc and free. I notice that the m3core code has > > Scheduler.DisableSwitching around all calls to malloc but not around calls > > to free. Why would one be unsafe and the other safe? It is always free > > that my program hangs up on by the way. Surrounding all "suspicious" calls > > in my code with Scheduler.DisableSwitching doesn't solve the problem, as > > far as I can tell. > > > > I am further suspecting that my problem is related to my having linked the > > program with -pthread. I am also guessing that this is the same reason > > you can't link FreeBSD user threads programs with -pthread: namely that if > > you go reentrantly into the pthread malloc library some code assumes > > the two invocations are from separate threads and locks up if they are from > > the same thread. That's my guess anyhow. But that doesn't explain why > > I've never seen any malloc/free problems on FreeBSD without -pthread. > > (I don't think I have, anyhow.) > > > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:26:26 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:26:26 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ Ok, "we" should probably go ahead and fix that. I'll try to, but no promises, sorry. - Jay > From: hosking at cs.purdue.edu > Date: Mon, 18 Apr 2011 18:11:26 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Probably unnecessary, given that I think there is another entry point to forking a process (I forget where) in the thread-specific portion of m3core. In which case the necessary work might be done there? > > On Apr 18, 2011, at 2:45 PM, Mika Nystrom wrote: > > > Tony Hosking writes: > > ... > >> pthread_atfork should not be needed under user threads. > > ... > > > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > > FreeBSD before 6. The comment is from the checked-in source file. > > > > /* NOTE: Even userthreads now depends > > * on availability of pthreads. > > * This can be fixed if need be. > > */ > > > > INTEGER > > __cdecl > > RTProcess__RegisterForkHandlers( > > ForkHandler prepare, > > ForkHandler parent, > > ForkHandler child) > > { > > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > > * VMS lacks pthread_atfork? Would be good to use autoconf. > > * Win32 lacks pthread_atfork and fork. OK. > > * > > * As well, for all Posix systems, we could implement > > * atfork ourselves, as long as we provide a fork() > > * wrapper that code uses. > > */ > > #if defined(_WIN32) \ > > || defined(__vms) \ > > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > > return 0; > > #else > > while (1) > > { > > int i = pthread_atfork(prepare, parent, child); > > if (i != EAGAIN) > > return i; > > sleep(0); > > } > > #endif > > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 01:28:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Mon, 18 Apr 2011 23:28:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, Message-ID: "Green" threads are probably another name for user threads. Java used to use something with this name. > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? Certainly. Has anything ever given you a different impression? Cygwin maybe uses pthreads, but that'd still use Win32 threads under the covers. It *might* be nice to have a user thread option that used fibers or Win7's UMS (64-bit only!) but don't hold your breath. Native Win32 threads have been around forever and will be a good option forever and will be in heavy use forever. - Jay > From: rcolebur at SCIRES.COM > To: m3devel at elegosoft.com > Date: Mon, 18 Apr 2011 17:35:40 -0400 > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > I think it would be good at this point to clarify for folks on the m3devel list what is meant by the various terms: > -pthreads > -user threads > -Win32 native threads > -green threads > -etc. > > I think I know what all these mean, except for "green" threads, but I'm not quite sure anymore. > > Also, in the current CM3 HEAD implementation, are we still using Windows native threads on WIN32 platforms? > > I too am observing problems with threading on Windows not behaving well. This situation concerns me because thread safety is something I've always relied on Modula-3 to provide. > > I think we need to post how the various threading models map to the various implementation platforms, and what options one needs to use to select between the models, and on which platforms a choice of threading model is available. > > Regards, > Randy Coleburn > > -----Original Message----- > From: Mika Nystrom [mailto:mika at async.caltech.edu] > Sent: Monday, April 18, 2011 2:46 PM > To: Tony Hosking > Cc: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Tony Hosking writes: > ... > >pthread_atfork should not be needed under user threads. > ... > > The following code from RTProcessC.c ensures that it is neeeded on every Unix except > FreeBSD before 6. The comment is from the checked-in source file. > > /* NOTE: Even userthreads now depends > * on availability of pthreads. > * This can be fixed if need be. > */ > > INTEGER > __cdecl > RTProcess__RegisterForkHandlers( > ForkHandler prepare, > ForkHandler parent, > ForkHandler child) > { > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > * VMS lacks pthread_atfork? Would be good to use autoconf. > * Win32 lacks pthread_atfork and fork. OK. > * > * As well, for all Posix systems, we could implement > * atfork ourselves, as long as we provide a fork() > * wrapper that code uses. > */ > #if defined(_WIN32) \ > || defined(__vms) \ > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > return 0; > #else > while (1) > { > int i = pthread_atfork(prepare, parent, child); > if (i != EAGAIN) > return i; > sleep(0); > } > #endif > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 04:27:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:27:04 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu> Message-ID: <20110419022704.6EFB71A207A@async.async.caltech.edu> Jay K writes: >--_f28739e3-2465-450f-b041-5795980d5bfe_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >And "yes" you need to disable switching around free=2C the same as malloc. = >And calloc. And realloc. >The disable switching *might* be at a lower level than you realize=2C so *m= >ight* not be missing. >That is=2C it might be in our C wrappers. I can check later. >=20 > - Jay Well in that case someone should remove the SchedulerPosix.DisableSwitching (etc) from around the mallocs. Or are you telling me that perhaps free() is wrapped but malloc() isn't? That would be... confusing. Mika From mika at async.caltech.edu Tue Apr 19 04:30:58 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Mon, 18 Apr 2011 19:30:58 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, <20110407190219.C00E31A2078@async.async.caltech.edu>, <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, <20110418164239.84CEB1A207A@async.async.caltech.edu>, <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, <20110418184540.C2D801A2078@async.async.caltech.edu>, <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu> Message-ID: <20110419023058.4FBB11A207A@async.async.caltech.edu> If you or Tony could describe roughly what you think needs to be done I'd be happy to look into it. The basic problem is that a decision that's made/described in m3core/thread/m3makefile needs to someone find its way to controlling what C code gets compiled elsewhere in the system. Or maybe there should be an indirection into the thread library to pick up pthread_atfork (or not). But then again you still haven't explained why you made the user threads use pthread_atfork. I just remove it from my installations, but then again I'm not trying to run CVSup so I don't know if it breaks that program to do so. Mika Jay K writes: >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> The following code from RTProcessC.c ensures that it is neeeded on every = >Unix except >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ > >=20 >Ok=2C "we" should probably go ahead and fix that. >I'll try to=2C but no promises=2C sorry. >=20 > - Jay >=20 >> From: hosking at cs.purdue.edu >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >> Probably unnecessary=2C given that I think there is another entry point t= >o forking a process (I forget where) in the thread-specific portion of m3co= >re. In which case the necessary work might be done there? >>=20 >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: >>=20 >> > Tony Hosking writes: >> > ... >> >> pthread_atfork should not be needed under user threads. >> > ... >> >=20 >> > The following code from RTProcessC.c ensures that it is neeeded on ever= >y Unix except >> > FreeBSD before 6. The comment is from the checked-in source file. >> >=20 >> > /* NOTE: Even userthreads now depends >> > * on availability of pthreads. >> > * This can be fixed if need be. >> > */ >> >=20 >> > INTEGER >> > __cdecl >> > RTProcess__RegisterForkHandlers( >> > ForkHandler prepare=2C >> > ForkHandler parent=2C >> > ForkHandler child) >> > { >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> > * Win32 lacks pthread_atfork and fork. OK. >> > * >> > * As well=2C for all Posix systems=2C we could implement >> > * atfork ourselves=2C as long as we provide a fork() >> > * wrapper that code uses. >> > */ >> > #if defined(_WIN32) \ >> > || defined(__vms) \ >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> > return 0=3B >> > #else >> > while (1) >> > { >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B >> > if (i !=3D EAGAIN) >> > return i=3B >> > sleep(0)=3B >> > } >> > #endif >> > } >>=20 > = > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > fixed if need be.
>=3B >=3B */

> =3B
>Ok=2C "we" should probably go ahead and fix that.
>I'll try to=2C but no =3Bpromises=2C sorry.
> =3B
> =3B- Jay
 =3B
>>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = >that I think there is another entry point to forking a process (I forget wh= >ere) in the thread-specific portion of m3core. In which case the necessary = >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child)>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = >as long as we provide a fork()
>=3B >=3B * wrapper that code uses.R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > >= > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- From jay.krell at cornell.edu Tue Apr 19 08:07:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:07:01 +0000 Subject: [M3devel] pthread_atfork Message-ID: I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: no-op for Win32: just return 0 It is tempting to have this assert(0) but that is wrong. The function will be called and should just do nothing. as-is for pthread: call pthread_atfork manually for userthreads: FreeBSD < 6: does it have pthreads? Can we declare such systems use user threads? take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 VMS: Punt for now. Maybe newer versions than I was using have it. And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible RTProcessPthread__Fork: return fork(); RTProcessUserThreads__Fork: wrapper around fork() that handles the callbacks from RTProcess__RegisterForkHandlers. RTProcessVms__Fork: assert(false) I think, unless newer VMS has fork(). One thing not stated here, is that I believe something "like" pthread_atfork is somewhere between reasonable and actually needed, on all Posix/Unix systems. Because, what isn't well known here, but I think is true, is that we depend on pthread_atfork (or a reimplementation thereof) as described here, so that fork + do more work with user threads behaves like pthreads, and doesn't have all the threads survive. Or..and I assume the answer is no: Do we want fork + do more work to keep all threads for user threads and keep only one thread for pthreads? How it was for a long time, but we didn't discover until fairly recently. I'm 99% sure this is all exactly correct. Possibly this is enough explanation for others to do it. Otherwise, maybe I'll get to it "soon". Maybe. This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:15:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:15:49 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: <20110419022704.6EFB71A207A@async.async.caltech.edu> References: <20110415103737.E73B31A2078@async.async.caltech.edu>, , , , <20110418161028.8DA9C1A207A@async.async.caltech.edu>, , <224F7E63-CD25-4A68-A44A-987BDF28B142@cs.purdue.edu>, , <20110419022704.6EFB71A207A@async.async.caltech.edu> Message-ID: Here is where malloc/calloc/free are handled, in a new, somewhat redundant way, that reduces/eliminates the burden at higher levels: jbook2:src jay$ pwd /dev2/cm3/m3-libs/m3core/src jbook2:src jay$ edit C/Common/CstdlibC.c M3WRAP_NO_SWITCHING(void*, Cstdlib__malloc, malloc, (WORD_T a), (a)) M3WRAP_NO_SWITCHING(void*, Cstdlib__calloc, calloc, (WORD_T a, WORD_T b), (a, b)) M3WRAP_RETURN_VOID_NO_SWITCHING(Cstdlib__free, free, (void* a), (a)) Now, if you have other C code that is calling malloc/free, you still need to disable switching around that. Perhaps all calls to <*extern*> functions should disable/enableswitching. Perhaps. The real answer is though, make sure pthreads and Win32 threads work, and then don't do much to make user threads faster or easier to maintain. see m3core/src/m3core.h for definitions of those macros. They should be pretty clear. If not, run the C preprocessor over those files and look at the results. -- The traditional correct thing is to wrap Cstdlib.malloc/Cstdlib.free whereever they are called. But I kept finding this missing, got fed up with it, fixed it at a lower level. Is that bad? Maybe. Maybe not. -- I wouldn't argue against wrapping all <*extern*> calls with disable/enableswitching. Given that you have to recompile everything to switch between user threads and kernel threads anyway, the compiler could insert these calls only for user threads, perhaps. - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:27:04 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > Jay K writes: > >--_f28739e3-2465-450f-b041-5795980d5bfe_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >And "yes" you need to disable switching around free=2C the same as malloc. = > >And calloc. And realloc. > >The disable switching *might* be at a lower level than you realize=2C so *m= > >ight* not be missing. > >That is=2C it might be in our C wrappers. I can check later. > >=20 > > - Jay > > Well in that case someone should remove the > SchedulerPosix.DisableSwitching (etc) from around the mallocs. > > Or are you telling me that perhaps free() is wrapped but malloc() isn't? > That would be... confusing. > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:23:55 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <937894.40594.qm@web29704.mail.ird.yahoo.com> References: , <937894.40594.qm@web29704.mail.ird.yahoo.com> Message-ID: > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 08:33:31 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 06:33:31 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: , , <937894.40594.qm@web29704.mail.ird.yahoo.com>, Message-ID: Caveat: OpenBSD now only offers pthreads. Modula-3 user threads require either get/set/make/swapcontext, or sigaltstack. OpenBSD offers neither. There were implementations using setjmp, but they were most unportable, only remained for OpenBSD, and I removed them. - Jay From: jay.krell at cornell.edu To: dabenavidesd at yahoo.es; m3devel at elegosoft.com; rcolebur at scires.com Date: Tue, 19 Apr 2011 06:23:55 +0000 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I think we need to post how the various threading models > map to the various implementation platforms, and what > options one needs to use to select between the models, and > on which platforms a choice of threading model is > available. Win32 only offers Win32 threads. That is the entire story for Win32. User threads have never been provided for Modula-3 on Win32. And probably never will. Though the idea isn't completely without merit. You might experiment with Fibers or UMS, but nobody has done anything here I believe. All other platforms offer user threads and pthreads. Pthreads are the default on all other platforms. Except if we detect we are hosting on and targeting FreeBSD 4.x. (Not FreeBSD 1.x, 2.x, 3,x, but only 4.x.) Then we default to user threads. To select user threads, you must edit m3-libs/m3core/src/thread/m3makefile and recompile everything. Or no edit, rebuild m3core with cm3 -DNOPTHREAD, and then still recompile everything. I believe you *cannot* just swap in one libm3core.so for another. It'd be nice to be able to do that, but I think it'd require a change that I suspect Tony would reject. I also would like an ability to select at link or execution time, but Tony definitely rejected that. You'd have to be sure to have indirection through function pointers, and the compiler could never inline lock usage. Not that it ever does today, but it might in the future. (Consider Win32 though: no inlining. And pthreads: generally abstract/opaque enough to not allow inlining?) Really -- user threads hardly have reason to exist. We need pthreads to work. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 13:51:06 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 04:51:06 -0700 Subject: [M3devel] LONGINT -> more bugs in m3tk Message-ID: <20110419115106.7E6A31A207A@async.async.caltech.edu> Hello m3devel, If anyone knows how to fix the following trivially, please go ahead. "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal (errors reported by m3tk). The offending line is: INTERFACE BasicCtypes; IMPORT Word, Long; TYPE (* the four signed integer types *) signed_char = [-16_7f-1 .. 16_7f]; short_int = [-16_7fff-1 .. 16_7fff]; int = [-16_7fffffff-1 .. 16_7fffffff]; long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) (* the four unsigned integer types *) unsigned_char = [16_0 .. 16_ff]; unsigned_short_int = [16_0 .. 16_ffff]; unsigned_int = [16_0 .. 16_ffffffff]; unsigned_long_int = Word.T; unsigned_long_long = Long.T; (* the three floating types *) float = REAL; double = LONGREAL; long_double = EXTENDED; (* char *) char = signed_char; END BasicCtypes. I'll look at it myself, but not immediately (no time). Workaround for now. Mika From jay.krell at cornell.edu Tue Apr 19 17:06:38 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 15:06:38 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419023058.4FBB11A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Tue Apr 19 17:13:55 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 16:13:55 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <768127.64883.qm@web29706.mail.ird.yahoo.com> Hi all: but I think there should be a way with the first implementation of MM3 if so, what is the risk of not having it? Can you see it, please: https://mail.elegosoft.com/pipermail/m3devel/2007-July/000272.html Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 10:06 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: ? user thread/pthread inconsistency ? cvsupd incompatibility with pthreads ? user threads using pthread_atfork/-pthread ? fork() calls having to go through a wrapper (ok -- you could miss this and ?? not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) ?- Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Tue Apr 19 17:29:24 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 08:29:24 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu> Message-ID: <20110419152924.22C411A207A@async.async.caltech.edu> Never explained this clearly before. Thanks, Jay. I suppose the reason CVSup forks and does work is to save copying overhead because the two forked processes can share read-only pages without any copying? I remember that the last time I looked at this my reaction was "fork() is not part of Modula-3 : people who use it are on their own". In a sense this is still true as FreeBSD simply will not work with -pthread. I'm afraid that, however, that removing -pthread didn't fix my Linux lockup problems. Turns out the library I was having trouble with is a PostgreSQL library written by Critical Mass. To fix it, I wrote a program using m3tk and some Scheme code that will take any M3 interface and wrap every routine in that interface in the following code structure: PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = BEGIN SchedulerIndirection.DisableSwitching(); TRY RETURN PQ.PQconnectPoll(conn) FINALLY SchedulerIndirection.EnableSwitching() END END PQconnectPoll; which then gave rise to my previous bug report about BasicCTypes, which m3tk cannot process a.t.m., owing to LONGINT breakage. Also I have to call "SchedulerIndirection" since the routines to turn off switching are named differently on CM3 (Scheduler.DisableSwitching) and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous to me but oh well I must have a dozen of these CM3/PM3 differences to deal with, what's another one? Mika Jay K writes: >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- From dabenavidesd at yahoo.es Tue Apr 19 18:08:43 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 17:08:43 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: <86414.26723.qm@web29716.mail.ird.yahoo.com> Hi all: I think the worst case scenario is to adapt changes to modification of such bugs, or UNSAFE parts, if it just gets to pick of a portion of the system well stabilized then all we can that depended on that, I hope this stabilizes soon, then we can all share the rest of the things. Thanks in advance --- El mar, 19/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > Para: "Jay K" > CC: m3devel at elegosoft.com > Fecha: martes, 19 de abril, 2011 10:29 > Never explained this clearly > before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save > copying > overhead because the two forked processes can share > read-only pages > without any copying? > > I remember that the last time I looked at this my reaction > was "fork() > is not part of Modula-3 : people who use it are on their > own". In a sense > this is still true as FreeBSD simply will not work with > -pthread. > > I'm afraid that, however, that removing -pthread didn't fix > my Linux > lockup problems. Turns out the library I was having > trouble with is > a PostgreSQL library written by Critical Mass. To fix > it, I wrote a > program using m3tk and some Scheme code that will take any > M3 interface > and wrap every routine in that interface in the following > code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : > PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about > BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the > routines to turn > off switching are named differently on CM3 > (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just > seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 > differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() > survived fork() > >Now: user threads and pthreads: only the thread calling > fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user > threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads > and user threads. > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost > immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to > fork(). > >Each module with a need to do something before/after > fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is > needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must > implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own > fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this > approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now > become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok > -- you could miss this an= > >d=20 > > not likely notice -- only fork() > calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but > people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same > thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you > think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's > made/described in > >> m3core/thread/m3makefile needs to someone find its > way to controlling > >> what C code gets compiled elsewhere in the > system. Or maybe there should > >> be an indirection into the thread library to pick > up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you > made the user threads = > >use > >> pthread_atfork. I just remove it from my > installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it > breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c > ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix > that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 > breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I > think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the > thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be > done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C > Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be > needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c > ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is > from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now > depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks > pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be > good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and > fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix > systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as > we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* > && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > > > > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B > charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of > pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B > >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix > that.
> >> >I'll try to=3D2C but > no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- > Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B > Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B > CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C > glibc-2.6-4
>=3D3B
>=3D3B > Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to > forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. > In which case the necessa= > >ry =3D > >> >work might be done > there?
>=3D3B
>=3D3B On Apr > 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom > wrote:
>=3D3B
>=3D3B > >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B > ...
>=3D3B >=3D3B>=3D3B > pthread_atfork sh= > >ould not be n=3D > >> >eeded under user > threads.
>=3D3B >=3D3B > ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from > RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B > >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B > >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B > >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if > need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B > >=3D3B INTEGER
>=3D3B >=3D3B > __cdecl
&= > >gt=3D3B >=3D3B =3D > >> > >RTProcess__RegisterForkHandlers(
>=3D3B > >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler > parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B > {
>=3D3B >=3D3B /* FreeBSD <=3D3B > 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use > autoconf.
>=3D3B >=3D3B * VMS lacks > pthread= > >_atfork=3D > >> >? Would be good to use > autoconf.
>=3D3B >=3D3B * Win32 lacks > pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B > >=3D3B *
>=3D3B >=3D3B * As > well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could > implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a > fork()
>=3D3B >=3D3B * wrapper that > code = > >uses. >> >R>>=3D3B >=3D3B > */
>=3D3B >=3D3B #if defined(_WIN32) > \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B > >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ > )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B > while (1)
>=3D3B >=3D3B > {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C > parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B > >=3D3B return i=3D3B
>=3D3B > >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B > }
>=3D3B >=3D3B > #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > > > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > > > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived > fork()
Previously: pthrea= > >ds: only the thread calling fork() survived > fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives > fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and > pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads > and user threads.
= > >It only really matters to the rare "fork + do more > work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or > exec almost immediately a= > >fter fork.


pthread_atfork > offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and > change all the calls to fork()= > >.
Each module with a need to do something > before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and > pthread_atfork cooperate to do w= > >hat is needed.


No function > pointer is needed.
Instead move th= > >e code to > m3core/thread.


If you really > must not use -pthread=2C = > >then you must implement pthread_atfork > functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that > cooperates
with = > >your pthread_atfork replacement.
There is no > free lunch -- there is a do= > >wnside to this approach=2C as plain fork() > calls
are ok and correct if p= > >thread_atfork is used=2C but now become > incorrect.
Pick your poison:
= > > =3B user thread/pthread > inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads > using pthread_atfork/-pthread >r> =3B fork() calls having to go through a > wrapper (ok -- you could mis= > >s this and
 =3B =3B not > likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper). >


(Most of this has = > >been explained multiple times=2C but people only pay > attention
when they= > > think it affects them. I'm guilty of the same > thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B > Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B > CC= > >: m3devel at elegosoft.com
>=3B > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C > glibc-2.6-4
>=3B
>=3B >
>=3B If = > >you or Tony could describe roughly what you think needs > to be done
>= > >=3B I'd be happy to look into it.
>=3B >
>=3B The basic problem is= > > that a decision that's made/described > in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to > controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there > should
>=3B be an ind= > >irection into the thread library to pick up > pthread_atfork (or not).
>= > >=3B
>=3B But then again you still > haven't explained why you made the = > >user threads use
>=3B > pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not > trying to run CVSup so I don't = > >know if it breaks that program to do > so.
>=3B
>=3B > Mika
= > >>=3B
>=3B Jay K > writes:
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B > >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B= > > >=3B
>=3B > >=3B
>=3B >=3B>=3B The > following code from RTP= > >rocessC.c ensures that it is neeeded on every > =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B > /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability > of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need > be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B > >=3B=3D20
>=3B >=3BOk=3D2C "we" > shoul= > >d probably go ahead and fix that.
>=3B > >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B > >=3B=3D20
>=3B >=3B - > Jay
>=3B = > >>=3B=3D20
>=3B > >=3B>=3B From: hosking at cs.purdue.edu
>=3B > &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 > -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B > >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: > [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B > >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B > Probably unnecessary=3D2C given that I thi= > >nk there is another entry point > t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of > m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done > there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On > Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B > >=3B>=3B=3D20
>=3B > >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B > >=3B>=3B >=3B ...
>=3B > >=3B>= > >=3B >=3B>=3B pthread_atfork should not be > needed under user threads. >>>=3B >=3B>=3B >=3B > ...
>=3B >=3B>=3B > >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code > from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By > Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from > the checked-in source file= > >.
>=3B >=3B>=3B > >=3B=3D20
>=3B >=3B>=3B > >=3B /* NOTE:= > > Even userthreads now depends
>=3B > >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B > >=3B>=3B >=3B * This can be fixed if need > b= > >e.
>=3B >=3B>=3B >=3B > */
>=3B >=3B>=3B > >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B > INTEGER
>=3B >=3B>=3B >=3B > __cdecl
= > >>=3B >=3B>=3B >=3B > RTProcess__RegisterForkHandlers(
>=3B > >= > >=3B>=3B >=3B ForkHandler > prepare=3D2C
>=3B >=3B>=3B > >=3B For= > >kHandler parent=3D2C
>=3B > >=3B>=3B >=3B ForkHandler > child)
&g= > >t=3B >=3B>=3B >=3B > {
>=3B >=3B>=3B >=3B /* > FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use > autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be > good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks > pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B > *
>=3B >=3B>=3B >=3B * As > well=3D2C for = > >all Posix systems=3D2C we could > implement
>=3B >=3B>=3B > >=3B * a= > >tfork ourselves=3D2C as long as we provide a > fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code > uses.
>=3B >=3B>=3B >=3B > */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) > \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B > >=3B>=3B >=3B || (defined(__FreeBSD__) > /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ > )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B > >=3B #else
>=3B >=3B>=3B > >= > >=3B while (1)
>=3B >=3B>=3B > >=3B {
>=3B >=3B>=3B > >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C > child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D > EAGAIN)
>=3B >=3B>=3B >=3B > retu= > >rn i=3D3B
>=3B >=3B>=3B > >=3B sleep(0)=3D3B
>=3B > >=3B>= > >=3B >=3B }
>=3B > >=3B>=3B >=3B #endif
>=3B > >=3B>=3B &g= > >t=3B }
>=3B > >=3B>=3B=3D20
>=3B >=3B > > > > =3D
>= > >=3B >=3B
>=3B > >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B > charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: > quoted-printable
>=3B > >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B > >=3B<=3Bhead>=3B
>=3B > >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B > >=3B.hmmessage P
>=3B > >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B > >=3Bpadding:0px
>=3B >=3B} >r>>=3B > >=3Bbody.hmmessage
>=3B > >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B > >=3Bfont-family:Tahoma
>=3B > >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B > >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody > class=3D3D'hmmessage'>=3B
>=3B > >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it > is neeeded on e=3D >>>=3B >=3Bvery Unix > except<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /= > >* NOTE: Even userthreads now > depends<=3BBR>=3B=3D
>=3B > >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of > pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can > be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > */<=3BBR>=3B<=3BBR>= > >=3B
>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3BOk=3D2C = > >"we" should probably go ahead and fix > that.<=3BBR>=3B
>=3B > >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C > sorry.<=3BBR>=3B >>>=3B > >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3Bnbsp=3D= > >3B- > Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B > >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B > Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B > >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B > CC: m3devel at el= > >egos=3D
>=3B > >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B > Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C > kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B > >=3Bthat I think there is anot= > >her entry point to forking a process (I forget > wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case > the necessary =3D >>>=3B >=3Bwork might be done > there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C > at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom > wrote:<=3BBR>=3B&=3Bgt=3D3B > <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking > writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B > ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be > n=3D
>=3B >=3Beede= > >d under user > threads.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B > <=3BBR>=3B&=3Bgt=3D3B > =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code > from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B > >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The > comment is from the=3D
&= > >gt=3B >=3B checked-in source > file.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now > depends<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B= > > * on availability of > pthreads.<=3BBR>=3B&=3B=3D
>=3B > >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need > be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B > __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B > >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler > prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B > &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 > lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use > autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks > pthread_atfork=3D
>=3B >=3B? Would be > good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. > OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix > systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork > ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a > fork()<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3= > >B * wrapper that code > uses.<=3BB=3D
>=3B > >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B > */<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) > \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > || (defined(__F= > >reeBSD__) /* > &=3Bamp=3D3B&=3Bam=3D
>=3B > >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ > )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D
>=3B >=3B=3D3B > #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B > {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > in=3D
= > >>=3B >=3Bt i =3D3D > pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B > if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B > sleep(0)=3D3B<=3BBR=3D
>=3B > >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B > }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B > &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B > =3D
>=3B >=3B > > > <=3B/body>=3B<= > >br>>=3B > >=3B<=3B/html>=3B=3D
>=3B > >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> > > > > >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- > From jay.krell at cornell.edu Tue Apr 19 20:50:16 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 18:50:16 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110419152924.22C411A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , <20110419152924.22C411A207A@async.async.caltech.edu> Message-ID: It's easy enough for us to support either name. - Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Tue Apr 19 22:32:54 2011 From: jay.krell at cornell.edu (Jay K) Date: Tue, 19 Apr 2011 20:32:54 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: > This change depends on pthread_atfork, for pthreads and user threads. Clarification: There's no change described there for pthreads, and indeed, getting this behavior with pthreads doesn't depend on using pthread_atfork. That is just how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs. fork()). The Solaris default used to be fork => forkall(). The current Solaris default is fork => fork1(). There really do exist on Solaris to functions you can call, fork1() and forkall(). All other systems and the Posix specification have the "fork1" semantic for fork. Therefore "fork1" is what we should provide unconditionally. If you really want "forkall", then you write: #ifdef __sun forkall() #else #error #endif and/or you go and hack up m3core/src/thread/POSIX, branch off m3core/src/thread/POSIX_FORKALL or such. I don't expect anyone to ever do that. The need for pthread_atfork is sort of different for user threads vs. pthreads, but it is needed either way. For pthreads, there is this problem -- at the time a thread calls fork(), what are other threads doing? Answer: Anything. Follow up question: What locks might they hold? Answer: Any. Follow up: So, if arbitrary locks are held, and you fork, and don't exec, and the child process attempts to use some of the locks that were held in the parent, and only the forking thread survived, then the other threads will never leave their locks, and later attempts to acquire them in the children will deadlock. Therefore, what pthread_atfork provides for, what you are supposed to use it for, is roughly speaking, right before fork(), in the parent, acquire all locks, and then fork(), and then in the parent and children, release all locks. Therefore, you have to come up with a locking order. Things are a little different for user threads. In user threads, the existance of threads is established by virtue of a global array or list of data describing the threads. And a timer to occasionally preempt. When the timer interrupts, the scheduler picks another thread from the global array/list. Previously, the data went unchanged when fork() was called. Therefore all threads survived. What we do now is that in the child "handler" for pthread_atfork, is reinitialize the various data, so as to forget about all but the current thread. Something like that. I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork, then ForkBeforeExec might get away with just fork() and ignore all this. So far we have not drawn this line -- there is just the one fork and it is used prior to exec or do-more-work and does the same thing w/o knowing what is to follow. The Posix documentation I think describes this all fairly well. Bing for "posix opengroup pthread_atfork". Really. I think it is saying much of the same thing I am saying, but in language that has received much more thought. Be sure to read the "rationale" part. - Jay From: jay.krell at cornell.edu To: mika at async.caltech.edu CC: m3devel at elegosoft.com Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Date: Tue, 19 Apr 2011 15:06:38 +0000 Previously: user threads: all threads survived fork() Previously: pthreads: only the thread calling fork() survived fork() Now: user threads and pthreads: only the thread calling fork() survives fork() This must be how pthreads behave, and this makes user threads and pthreads consistent. This change depends on pthread_atfork, for pthreads and user threads. It only really matters to the rare "fork + do more work" program, such as cvsupd. Most programs either never fork, or exec almost immediately after fork. pthread_atfork offers a good model. A sort of "distributed" model. You don't have to go and change all the calls to fork(). Each module with a need to do something before/after fork, calls the central pthread_atfork, and fork and pthread_atfork cooperate to do what is needed. No function pointer is needed. Instead move the code to m3core/thread. If you really must not use -pthread, then you must implement pthread_atfork functionality yourself and have all fork() calls go through your own fork() wrapper that cooperates with your pthread_atfork replacement. There is no free lunch -- there is a downside to this approach, as plain fork() calls are ok and correct if pthread_atfork is used, but now become incorrect. Pick your poison: user thread/pthread inconsistency cvsupd incompatibility with pthreads user threads using pthread_atfork/-pthread fork() calls having to go through a wrapper (ok -- you could miss this and not likely notice -- only fork() calls in fork+do-more-work programs need the wrapper). (Most of this has been explained multiple times, but people only pay attention when they think it affects them. I'm guilty of the same thing.) - Jay > To: jay.krell at cornell.edu > Date: Mon, 18 Apr 2011 19:30:58 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > > If you or Tony could describe roughly what you think needs to be done > I'd be happy to look into it. > > The basic problem is that a decision that's made/described in > m3core/thread/m3makefile needs to someone find its way to controlling > what C code gets compiled elsewhere in the system. Or maybe there should > be an indirection into the thread library to pick up pthread_atfork (or not). > > But then again you still haven't explained why you made the user threads use > pthread_atfork. I just remove it from my installations, but then again I'm > not trying to run CVSup so I don't know if it breaks that program to do so. > > Mika > > Jay K writes: > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >> The following code from RTProcessC.c ensures that it is neeeded on every = > >Unix except > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > > > >=20 > >Ok=2C "we" should probably go ahead and fix that. > >I'll try to=2C but no promises=2C sorry. > >=20 > > - Jay > >=20 > >> From: hosking at cs.purdue.edu > >> Date: Mon=2C 18 Apr 2011 18:11:26 -0400 > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >> Probably unnecessary=2C given that I think there is another entry point t= > >o forking a process (I forget where) in the thread-specific portion of m3co= > >re. In which case the necessary work might be done there? > >>=20 > >> On Apr 18=2C 2011=2C at 2:45 PM=2C Mika Nystrom wrote: > >>=20 > >> > Tony Hosking writes: > >> > ... > >> >> pthread_atfork should not be needed under user threads. > >> > ... > >> >=20 > >> > The following code from RTProcessC.c ensures that it is neeeded on ever= > >y Unix except > >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >=20 > >> > /* NOTE: Even userthreads now depends > >> > * on availability of pthreads. > >> > * This can be fixed if need be. > >> > */ > >> >=20 > >> > INTEGER > >> > __cdecl > >> > RTProcess__RegisterForkHandlers( > >> > ForkHandler prepare=2C > >> > ForkHandler parent=2C > >> > ForkHandler child) > >> > { > >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> > * Win32 lacks pthread_atfork and fork. OK. > >> > * > >> > * As well=2C for all Posix systems=2C we could implement > >> > * atfork ourselves=2C as long as we provide a fork() > >> > * wrapper that code uses. > >> > */ > >> > #if defined(_WIN32) \ > >> > || defined(__vms) \ > >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> > return 0=3B > >> > #else > >> > while (1) > >> > { > >> > int i =3D pthread_atfork(prepare=2C parent=2C child)=3B > >> > if (i !=3D EAGAIN) > >> > return i=3B > >> > sleep(0)=3B > >> > } > >> > #endif > >> > } > >>=20 > > = > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B /* NOTE: Even userthreads now depends
= > >>=3B >=3B * on availability of pthreads.
>=3B >=3B * This can be= > > fixed if need be.
>=3B >=3B */

> > =3B
> >Ok=2C "we" should probably go ahead and fix that.
> >I'll try to=2C but no =3Bpromises=2C sorry.
> > =3B
> > =3B- Jay
 =3B
> >>=3B From: hosking at cs.purdue.edu
>=3B Date: Mon=2C 18 Apr 2011 18:11= > >:26 -0400
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegos= > >oft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kerne= > >l 2.6.23=2C glibc-2.6-4
>=3B
>=3B Probably unnecessary=2C given = > >that I think there is another entry point to forking a process (I forget wh= > >ere) in the thread-specific portion of m3core. In which case the necessary = > >work might be done there?
>=3B
>=3B On Apr 18=2C 2011=2C at 2:45= > > PM=2C Mika Nystrom wrote:
>=3B
>=3B >=3B Tony Hosking writes:= > >
>=3B >=3B ...
>=3B >=3B>=3B pthread_atfork should not be n= > >eeded under user threads.
>=3B >=3B ...
>=3B >=3B
>=3B = > >>=3B The following code from RTProcessC.c ensures that it is neeeded on e= > >very Unix except
>=3B >=3B FreeBSD before 6. The comment is from the= > > checked-in source file.
>=3B >=3B
>=3B >=3B /* NOTE: Even u= > >serthreads now depends
>=3B >=3B * on availability of pthreads.
&= > >gt=3B >=3B * This can be fixed if need be.
>=3B >=3B */
>=3B = > >>=3B
>=3B >=3B INTEGER
>=3B >=3B __cdecl
>=3B >=3B = > >RTProcess__RegisterForkHandlers(
>=3B >=3B ForkHandler prepare=2C >>>=3B >=3B ForkHandler parent=2C
>=3B >=3B ForkHandler child) >>>=3B >=3B {
>=3B >=3B /* FreeBSD <=3B 6 lacks pthread_atfork.= > > Would be good to use autoconf.
>=3B >=3B * VMS lacks pthread_atfork= > >? Would be good to use autoconf.
>=3B >=3B * Win32 lacks pthread_atf= > >ork and fork. OK.
>=3B >=3B *
>=3B >=3B * As well=2C for all = > >Posix systems=2C we could implement
>=3B >=3B * atfork ourselves=2C = > >as long as we provide a fork()
>=3B >=3B * wrapper that code uses. >R>>=3B >=3B */
>=3B >=3B #if defined(_WIN32) \
>=3B >=3B = > >|| defined(__vms) \
>=3B >=3B || (defined(__FreeBSD__) /* &=3B&am= > >p=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B return 0=3B
>=3B >= > >=3B #else
>=3B >=3B while (1)
>=3B >=3B {
>=3B >=3B in= > >t i =3D pthread_atfork(prepare=2C parent=2C child)=3B
>=3B >=3B if (= > >i !=3D EAGAIN)
>=3B >=3B return i=3B
>=3B >=3B sleep(0)=3B >>>=3B >=3B }
>=3B >=3B #endif
>=3B >=3B }
>=3B
= > > > >= > > > >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 20 00:10:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Tue, 19 Apr 2011 23:10:45 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <718150.90366.qm@web29720.mail.ird.yahoo.com> Hi all: since there was just that interface in *nix platforms, then it must be called like that , but another name will be just create SchedulerCM3, thus suggests SchedulerPosix would be called rather than nothing suffixing it SchedulerSRC and the other one Scheduler alone be the default like to say SchedulerCM3 AS Scheduler? so to switch to SchedulerSRC AS Scheduler provided that both are implementation ready so one can select SchedulerCM3 as SchedulerSRC or Scheduler so one name comes after each rep rather than changing every other place each time, even of Internal Nodes as in m3tk which specifies its members as? SRC_NODE <: AST_NODE then something like CM3_NODE likiwise subtyped if so should be created to hold the extensions (can somebody name them WIDECHAR, LONGINT, LONGCARD, what else am I missing if so?),? like we could come up with something cleaner than just put all there. I will try to se a proposal for that. Thanks in advance --- El mar, 19/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "Mika Nystrom" CC: "m3devel" Fecha: martes, 19 de abril, 2011 13:50 It's easy enough for us to support either name. ?- Jay > To: jay.krell at cornell.edu > Date: Tue, 19 Apr 2011 08:29:24 -0700 > From: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Never explained this clearly before. Thanks, Jay. > > I suppose the reason CVSup forks and does work is to save copying > overhead because the two forked processes can share read-only pages > without any copying? > > I remember that the last time I looked at this my reaction was "fork() > is not part of Modula-3 : people who use it are on their own". In a sense > this is still true as FreeBSD simply will not work with -pthread. > > I'm afraid that, however, that removing -pthread didn't fix my Linux > lockup problems. Turns out the library I was having trouble with is > a PostgreSQL library written by Critical Mass. To fix it, I wrote a > program using m3tk and some Scheme code that will take any M3 interface > and wrap every routine in that interface in the following code structure: > > PROCEDURE PQconnectPoll(VALUE conn : PQ.PGconn_star) : PQ.PostgresPollingStatusType = > BEGIN > SchedulerIndirection.DisableSwitching(); > TRY > RETURN PQ.PQconnectPoll(conn) > FINALLY > SchedulerIndirection.EnableSwitching() > END > END PQconnectPoll; > > which then gave rise to my previous bug report about BasicCTypes, which > m3tk cannot process a.t.m., owing to LONGINT breakage. > > Also I have to call "SchedulerIndirection" since the routines to turn > off switching are named differently on CM3 (Scheduler.DisableSwitching) > and PM3 (SchedulerPosix.DisableSwitching).... this just seems gratuitous > to me but oh well I must have a dozen of these CM3/PM3 differences to deal > with, what's another one? > > Mika > > Jay K writes: > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/plain; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > >Previously: user threads: all threads survived fork() > >Previously: pthreads: only the thread calling fork() survived fork() > >Now: user threads and pthreads: only the thread calling fork() survives for= > >k() > >This must be how pthreads behave=2C and this makes user threads and pthread= > >s consistent. > > > >This change depends on pthread_atfork=2C for pthreads and user threads. > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd. > >Most programs either never fork=2C or exec almost immediately after fork. > > > > > >pthread_atfork offers a good model. > >A sort of "distributed" model. > >You don't have to go and change all the calls to fork(). > >Each module with a need to do something before/after fork=2C calls the cent= > >ral pthread_atfork=2C > >and fork and pthread_atfork cooperate to do what is needed. > > > > > >No function pointer is needed. > >Instead move the code to m3core/thread. > > > > > >If you really must not use -pthread=2C then you must implement pthread_atfo= > >rk functionality > >yourself and have all fork() calls go through your own fork() wrapper that = > >cooperates > >with your pthread_atfork replacement. > >There is no free lunch -- there is a downside to this approach=2C as plain = > >fork() calls > >are ok and correct if pthread_atfork is used=2C but now become incorrect. > >Pick your poison: > > user thread/pthread inconsistency > > cvsupd incompatibility with pthreads=20 > > user threads using pthread_atfork/-pthread=20 > > fork() calls having to go through a wrapper (ok -- you could miss this an= > >d=20 > > not likely notice -- only fork() calls in fork+do-more-work programs nee= > >d the wrapper).=20 > > > > > >(Most of this has been explained multiple times=2C but people only pay atte= > >ntion > >when they think it affects them. I'm guilty of the same thing.) > > > > > > - Jay > > > > > >> To: jay.krell at cornell.edu > >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >> From: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >c-2.6-4 > >>=20 > >>=20 > >> If you or Tony could describe roughly what you think needs to be done > >> I'd be happy to look into it. > >>=20 > >> The basic problem is that a decision that's made/described in > >> m3core/thread/m3makefile needs to someone find its way to controlling > >> what C code gets compiled elsewhere in the system. Or maybe there should > >> be an indirection into the thread library to pick up pthread_atfork (or n= > >ot). > >>=20 > >> But then again you still haven't explained why you made the user threads = > >use > >> pthread_atfork. I just remove it from my installations=2C but then again= > > I'm > >> not trying to run CVSup so I don't know if it breaks that program to do s= > >o. > >>=20 > >> Mika > >>=20 > >> Jay K writes: > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= > >ry =3D > >> >Unix except > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> > > >> >=3D20 > >> >Ok=3D2C "we" should probably go ahead and fix that. > >> >I'll try to=3D2C but no promises=3D2C sorry. > >> >=3D20 > >> > - Jay > >> >=3D20 > >> >> From: hosking at cs.purdue.edu > >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >> >> To: mika at async.caltech.edu > >> >> CC: m3devel at elegosoft.com > >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >2C glib=3D > >> >c-2.6-4 > >> >>=3D20 > >> >> Probably unnecessary=3D2C given that I think there is another entry po= > >int t=3D > >> >o forking a process (I forget where) in the thread-specific portion of m= > >3co=3D > >> >re. In which case the necessary work might be done there? > >> >>=3D20 > >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >> >>=3D20 > >> >> > Tony Hosking writes: > >> >> > ... > >> >> >> pthread_atfork should not be needed under user threads. > >> >> > ... > >> >> >=3D20 > >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= > >ver=3D > >> >y Unix except > >> >> > FreeBSD before 6. The comment is from the checked-in source file. > >> >> >=3D20 > >> >> > /* NOTE: Even userthreads now depends > >> >> > * on availability of pthreads. > >> >> > * This can be fixed if need be. > >> >> > */ > >> >> >=3D20 > >> >> > INTEGER > >> >> > __cdecl > >> >> > RTProcess__RegisterForkHandlers( > >> >> > ForkHandler prepare=3D2C > >> >> > ForkHandler parent=3D2C > >> >> > ForkHandler child) > >> >> > { > >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. > >> >> > * Win32 lacks pthread_atfork and fork. OK. > >> >> > * > >> >> > * As well=3D2C for all Posix systems=3D2C we could implement > >> >> > * atfork ourselves=3D2C as long as we provide a fork() > >> >> > * wrapper that code uses. > >> >> > */ > >> >> > #if defined(_WIN32) \ > >> >> > || defined(__vms) \ > >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >> >> > return 0=3D3B > >> >> > #else > >> >> > while (1) > >> >> > { > >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >> >> > if (i !=3D3D EAGAIN) > >> >> > return i=3D3B > >> >> > sleep(0)=3D3B > >> >> > } > >> >> > #endif > >> >> > } > >> >>=3D20 > >> > =3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >> >Content-Type: text/html=3B charset=3D"iso-8859-1" > >> >Content-Transfer-Encoding: quoted-printable > >> > > >> > > >> > > >> > > >> > > >> > > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >nds
=3D > >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >This can be=3D > >> > fixed if need be.
>=3D3B >=3D3B */

> >> > =3D3B
> >> >Ok=3D2C "we" should probably go ahead and fix that.
> >> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >> > =3D3B
> >> > =3D3B- Jay
 =3D3B
> >> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >011 18:11=3D > >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >@elegos=3D > >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >C kerne=3D > >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >=3D2C given =3D > >> >that I think there is another entry point to forking a process (I forget= > > wh=3D > >> >ere) in the thread-specific portion of m3core. In which case the necessa= > >ry =3D > >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >=3D2C at 2:45=3D > >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >ing writes:=3D > >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >ould not be n=3D > >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >
>=3D3B =3D > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > > on e=3D > >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >rom the=3D > >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >OTE: Even u=3D > >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >s.
&=3D > >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >
>=3D3B =3D > >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >gt=3D3B >=3D3B =3D > >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >e=3D2C >> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >ler child) >> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >hread_atfork.=3D > >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >_atfork=3D > >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >ead_atf=3D > >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >=3D2C for all =3D > >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >elves=3D2C =3D > >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >uses. >> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >=3D3B >=3D3B =3D > >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >p=3D3B&am=3D > >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >R>>=3D3B >=3D > >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >=3D3B >=3D3B in=3D > >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >B >=3D3B if (=3D > >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > > sleep(0)=3D3B >> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >BR>>=3D3B
=3D > >> > > >> >=3D > >> > > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > > = > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_ > >Content-Type: text/html; charset="iso-8859-1" > >Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= > >ds: only the thread calling fork() survived fork()
Now: user threads and= > > pthreads: only the thread calling fork() survives fork()
This must be h= > >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >br> > >This change depends on pthread_atfork=2C for pthreads and user threads.
= > >It only really matters to the rare "fork + do more work" program=2C such as= > > cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >tributed" model.
You don't have to go and change all the calls to fork()= > >.
Each module with a need to do something before/after fork=2C calls the= > > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >hat is needed.


No function pointer is needed.
Instead move th= > >e code to m3core/thread.


If you really must not use -pthread=2C = > >then you must implement pthread_atfork functionality
yourself and have a= > >ll fork() calls go through your own fork() wrapper that cooperates
with = > >your pthread_atfork replacement.
There is no free lunch -- there is a do= > >wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >ty with pthreads
 =3B user threads using pthread_atfork/-pthread >r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >s this and
 =3B =3B not likely notice -- only fork() calls in f= > >ork+do-more-work programs need the wrapper).


(Most of this has = > >been explained multiple times=2C but people only pay attention
when they= > > think it affects them. I'm guilty of the same thing.)


 =3B-= > > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >you or Tony could describe roughly what you think needs to be done
>= > >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > > that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >e needs to someone find its way to controlling
>=3B what C code gets c= > >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >irection into the thread library to pick up pthread_atfork (or not).
>= > >=3B
>=3B But then again you still haven't explained why you made the = > >user threads use
>=3B pthread_atfork. I just remove it from my instal= > >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >know if it breaks that program to do so.
>=3B
>=3B Mika
= > >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >. In which case the necessary work might be done there?
>=3B >=3B>= > >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >=3B >=3B>=3B pthread_atfork should not be needed under user threads. >>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > > The following code from RTProcessC.c ensures that it is neeeded on e=3D >>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >n the thread-specific portion of m3core. In which case the necessary =3D >>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >= > > > >--_9d99325a-59cc-496a-9b87-3881dbf2b655_-- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Wed Apr 20 04:18:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 20 Apr 2011 02:18:25 +0000 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: Can you by chance see the other threads? I suspect not -- user threads I presume. Can you try sprinking a lot more Disable/EnableSwitching around? I guess you tried that..and..? - Jay > From: hosking at cs.purdue.edu > Date: Fri, 15 Apr 2011 11:26:15 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] malloc/free from Modula-3? > > I suspect the problem is re-entrance to the library. Should you be disabling user thread switching somewhere? > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 15, 2011, at 6:37 AM, Mika Nystrom wrote: > > > Hi m3devel, > > > > I'm debugging a problem with some code I have that calls out to C > > code. It's calling pqlib (PostgreSQL library) and you might expect > > things to go wrong there, but as it turns out it's dying in CM3's code. > > > > The system is AMD64_LINUX (user threading because of the problems with > > pthreads) and the program is just freezing up hard. No segfault, > > bus error or anything. It locks up a couple of times a day. > > > > What are the rules on calling malloc/free from Modula-3? > > > > Stack trace: > > > > (gdb) where > > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 > > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 > > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 > > #3 0x000000000091462f in Cstdlib__free (a=Cannot access memory at address 0x2ac73b0ef0d8 > > ) at ../src/C/Common/CstdlibC.c:26 > > #4 0x000000000091477d in M3toC__FreeCopiedS (s=Cannot access memory at address 0x2ac73b0ef0f8 > > ) at ../src/C/Common/M3toC.m3:46 > > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=Cannot access memory at address 0x2ac73b0ef118 > > ) at ../src/UnsafeDatabase.m3:230 > > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x2ac73b0ef148 > > ) at ../src/UnsafeDatabase.m3:228 > > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x2ac73b0ef268 > > ) at ../src/UnsafeDatabase.m3:258 > > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=Cannot access memory at address 0x2ac73b0ef328 > > ) at ../src/DesynchronizedDB.m3:126 > > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=Cannot access memory at address 0x2ac73b0ef3e8 > > ) at ../src/DesynchronizedDB.m3:293 > > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=Cannot access memory at address 0x2ac73b0ef5a8 > > ) at ../src/GCOMSOrderManager.m3:596 > > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=Cannot access memory at address 0x2ac73b0ef668 > > ) at ../src/GCOMSOrderManager.m3:310 > > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price={}, count=Cannot access memory at address 0x2ac73b0ef7b8 > > ) at ../src/EuroDriver.m3:1108 > > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../src/EuroDriver.m3:1401 > > #14 0x00000000004197a1 in EuroDriver__OApply (cl=Cannot access memory at address 0x2ac73b0efd38 > > ) at ../src/EuroDriver.m3:1545 > > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 > > #17 0x0000000000000000 in ?? () > > (gdb) > > > > Here is the code that causes the problem: > > > > VAR > > s := CopyTtoS(query); > > BEGIN > > TRY > > t.res := t.myExec(s,busyWait) > > FINALLY > > FreeCopiedS(s) (* stuck here *) > > END > > END; > > > > The query is a SQL query---t.myExec eventually calls the libpq SQL exec > > routine (PQexec I think it's called). > > > > Since this is already an unsafe module I suppose I could just make an > > array of CHARs on the stack and pass a pointer to that... > > > > > > Regards, > > Mika > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Wed Apr 20 07:55:57 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 22:55:57 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, Message-ID: <20110420055557.089101A207A@async.async.caltech.edu> The rationale you directed us to says: "It is suggested that programs that use fork() call an exec function very soon afterwards in the child process, thus resetting all states. In the meantime, only a short list of async-signal-safe library routines are promised to be available." and later "Application programs may not be aware that a multi-threaded library is in use, and they feel free to call any number of library routines between the fork() and exec calls, just as they always have. Indeed, they may be extant single-threaded programs and cannot, therefore, be expected to obey new restrictions imposed by the threads library." In C maybe that is true. In Modula-3, every application has *always* had to assume that there are other threads floating around. There is no such thing as a "single-threaded" Modula-3 program---or at least, the distinction doesn't matter. I really think fork-and-do-more-work is just a bug. Maybe it is a special pattern used by some special program---maybe. In any case support for it I do not think belongs in m3core. Mika Jay K writes: >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > This change depends on pthread_atfork=2C for pthreads and user threads. > > >Clarification: There's no change described there for pthreads=2C and indeed= >=2C getting >this behavior with pthreads doesn't depend on using pthread_atfork. That is= > just >how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > fork()). >The Solaris default used to be fork =3D> forkall(). >The current Solaris default is fork =3D> fork1(). >There really do exist on Solaris to functions you can call=2C fork1() and f= >orkall(). >All other systems and the Posix specification have the "fork1" semantic for= > fork. >Therefore "fork1" is what we should provide unconditionally. >If you really want "forkall"=2C then you write: >#ifdef __sun > forkall() >#else > #error >#endif > > > >and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >thread/POSIX_FORKALL or such. >I don't expect anyone to ever do that. > > > >The need for pthread_atfork is sort of different for user threads vs. pthre= >ads=2C but it is needed either way. >For pthreads=2C there is this problem -- at the time a thread calls fork()= >=2C what are other threads doing? >Answer: Anything. >Follow up question: What locks might they hold? >Answer: Any. >Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >xec=2C and the child process >attempts to use some of the locks that were held in the parent=2C and only = >the forking thread survived=2C >then the other threads will never leave their locks=2C and later attempts t= >o acquire them in the children >will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >e supposed to use it for=2C >is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >locks=2C and then fork()=2C and then in the >parent and children=2C release all locks. Therefore=2C you have to come up = >with a locking order. > > > >Things are a little different for user threads. >In user threads=2C the existance of threads is established by virtue of a g= >lobal array or list of data describing the threads. >And a timer to occasionally preempt. When the timer interrupts=2C the sched= >uler picks another thread from the global >array/list. Previously=2C the data went unchanged when fork() was called. T= >herefore all threads survived. >What we do now is that in the child "handler" for pthread_atfork=2C is rein= >itialize the various data=2C so as to forget about >all but the current thread. Something like that. > > > >I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >rk=2C then ForkBeforeExec >might get away with just fork() and ignore all this. >So far we have not drawn this line -- there is just the one fork and it is = >used prior to exec or do-more-work >and does the same thing w/o knowing what is to follow. > > >The Posix documentation I think describes this all fairly well. >Bing for "posix opengroup pthread_atfork". >Really. I think it is saying much of the same thing I am saying=2C but in l= >anguage that has received >much more thought. Be sure to read the "rationale" part. > > > - Jay > > >From: jay.krell at cornell.edu >To: mika at async.caltech.edu >CC: m3devel at elegosoft.com >Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >2.6-4 >Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > > > > > > > > >Previously: user threads: all threads survived fork() >Previously: pthreads: only the thread calling fork() survived fork() >Now: user threads and pthreads: only the thread calling fork() survives for= >k() >This must be how pthreads behave=2C and this makes user threads and pthread= >s consistent. > >This change depends on pthread_atfork=2C for pthreads and user threads. >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd. >Most programs either never fork=2C or exec almost immediately after fork. > > >pthread_atfork offers a good model. >A sort of "distributed" model. >You don't have to go and change all the calls to fork(). >Each module with a need to do something before/after fork=2C calls the cent= >ral pthread_atfork=2C >and fork and pthread_atfork cooperate to do what is needed. > > >No function pointer is needed. >Instead move the code to m3core/thread. > > >If you really must not use -pthread=2C then you must implement pthread_atfo= >rk functionality >yourself and have all fork() calls go through your own fork() wrapper that = >cooperates >with your pthread_atfork replacement. >There is no free lunch -- there is a downside to this approach=2C as plain = >fork() calls >are ok and correct if pthread_atfork is used=2C but now become incorrect. >Pick your poison: > user thread/pthread inconsistency > cvsupd incompatibility with pthreads=20 > user threads using pthread_atfork/-pthread=20 > fork() calls having to go through a wrapper (ok -- you could miss this an= >d=20 > not likely notice -- only fork() calls in fork+do-more-work programs nee= >d the wrapper).=20 > > >(Most of this has been explained multiple times=2C but people only pay atte= >ntion >when they think it affects them. I'm guilty of the same thing.) > > > - Jay > > >> To: jay.krell at cornell.edu >> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >> From: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >c-2.6-4 >>=20 >>=20 >> If you or Tony could describe roughly what you think needs to be done >> I'd be happy to look into it. >>=20 >> The basic problem is that a decision that's made/described in >> m3core/thread/m3makefile needs to someone find its way to controlling >> what C code gets compiled elsewhere in the system. Or maybe there should >> be an indirection into the thread library to pick up pthread_atfork (or n= >ot). >>=20 >> But then again you still haven't explained why you made the user threads = >use >> pthread_atfork. I just remove it from my installations=2C but then again= > I'm >> not trying to run CVSup so I don't know if it breaks that program to do s= >o. >>=20 >> Mika >>=20 >> Jay K writes: >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/plain=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> >> The following code from RTProcessC.c ensures that it is neeeded on eve= >ry =3D >> >Unix except >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> > >> >=3D20 >> >Ok=3D2C "we" should probably go ahead and fix that. >> >I'll try to=3D2C but no promises=3D2C sorry. >> >=3D20 >> > - Jay >> >=3D20 >> >> From: hosking at cs.purdue.edu >> >> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >> >> To: mika at async.caltech.edu >> >> CC: m3devel at elegosoft.com >> >> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >2C glib=3D >> >c-2.6-4 >> >>=3D20 >> >> Probably unnecessary=3D2C given that I think there is another entry po= >int t=3D >> >o forking a process (I forget where) in the thread-specific portion of m= >3co=3D >> >re. In which case the necessary work might be done there? >> >>=3D20 >> >> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >> >>=3D20 >> >> > Tony Hosking writes: >> >> > ... >> >> >> pthread_atfork should not be needed under user threads. >> >> > ... >> >> >=3D20 >> >> > The following code from RTProcessC.c ensures that it is neeeded on e= >ver=3D >> >y Unix except >> >> > FreeBSD before 6. The comment is from the checked-in source file. >> >> >=3D20 >> >> > /* NOTE: Even userthreads now depends >> >> > * on availability of pthreads. >> >> > * This can be fixed if need be. >> >> > */ >> >> >=3D20 >> >> > INTEGER >> >> > __cdecl >> >> > RTProcess__RegisterForkHandlers( >> >> > ForkHandler prepare=3D2C >> >> > ForkHandler parent=3D2C >> >> > ForkHandler child) >> >> > { >> >> > /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >> >> > * VMS lacks pthread_atfork? Would be good to use autoconf. >> >> > * Win32 lacks pthread_atfork and fork. OK. >> >> > * >> >> > * As well=3D2C for all Posix systems=3D2C we could implement >> >> > * atfork ourselves=3D2C as long as we provide a fork() >> >> > * wrapper that code uses. >> >> > */ >> >> > #if defined(_WIN32) \ >> >> > || defined(__vms) \ >> >> > || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >> >> > return 0=3D3B >> >> > #else >> >> > while (1) >> >> > { >> >> > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >> >> > if (i !=3D3D EAGAIN) >> >> > return i=3D3B >> >> > sleep(0)=3D3B >> >> > } >> >> > #endif >> >> > } >> >>=3D20 >> > =3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >> >Content-Type: text/html=3B charset=3D"iso-8859-1" >> >Content-Transfer-Encoding: quoted-printable >> > >> > >> > >> > >> > >> > >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >nds
=3D >> >>=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >This can be=3D >> > fixed if need be.
>=3D3B >=3D3B */

>> > =3D3B
>> >Ok=3D2C "we" should probably go ahead and fix that.
>> >I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>> > =3D3B
>> > =3D3B- Jay
 =3D3B
>> >>=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >011 18:11=3D >> >:26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >@elegos=3D >> >oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >C kerne=3D >> >l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >=3D2C given =3D >> >that I think there is another entry point to forking a process (I forget= > wh=3D >> >ere) in the thread-specific portion of m3core. In which case the necessa= >ry =3D >> >work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >=3D2C at 2:45=3D >> > PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >ing writes:=3D >> >
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >ould not be n=3D >> >eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >
>=3D3B =3D >> >>=3D3B The following code from RTProcessC.c ensures that it is neeeded= > on e=3D >> >very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >rom the=3D >> > checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >OTE: Even u=3D >> >serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >s.
&=3D >> >gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >
>=3D3B =3D >> >>=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >gt=3D3B >=3D3B =3D >> >RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >e=3D2C> >>>=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >ler child)> >>>=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >hread_atfork.=3D >> > Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >_atfork=3D >> >? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >ead_atf=3D >> >ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >=3D2C for all =3D >> >Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >elves=3D2C =3D >> >as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >uses.> >R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >=3D3B >=3D3B =3D >> >|| defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >p=3D3B&am=3D >> >p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3BR>>=3D3B >=3D >> >=3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >=3D3B >=3D3B in=3D >> >t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >B >=3D3B if (=3D >> >i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > sleep(0)=3D3B> >>>=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >BR>>=3D3B
=3D >> > >> >=3D >> > >> >--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > = > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > > =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >er threads.


Clarification: There's no change described there for= > pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >epend on using pthread_atfork. That is just
how pthreads are -- except o= >ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >ault used to be fork =3D>=3B forkall().
The current Solaris default is= > fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >t we should provide unconditionally.
If you really want "forkall"=2C the= >n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >ror
#endif



and/or you go and hack up m3core/src/thread/PO= >SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >t anyone to ever do that.



The need for pthread_atfork is sor= >t of different for user threads vs. pthreads=2C but it is needed either way= >.
For pthreads=2C there is this problem -- at the time a thread calls fo= >rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >process
attempts to use some of the locks that were held in the parent= >=2C and only the forking thread survived=2C
then the other threads will = >never leave their locks=2C and later attempts to acquire them in the childr= >en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >you are supposed to use it for=2C
is roughly speaking=2C right before fo= >rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >n the
parent and children=2C release all locks. Therefore=2C you have to= > come up with a locking order.



Things are a little different= > for user threads.
In user threads=2C the existance of threads is establ= >ished by virtue of a global array or list of data describing the threads.r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >eduler picks another thread from the global
array/list. Previously=2C th= >e data went unchanged when fork() was called. Therefore all threads survive= >d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >s reinitialize the various data=2C so as to forget about
all but the cur= >rent thread. Something like that.



I suspect if we had RTProc= >ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >br>might get away with just fork() and ignore all this.
So far we have n= >ot drawn this line -- there is just the one fork and it is used prior to ex= >ec or do-more-work
and does the same thing w/o knowing what is to follow= >.


The Posix documentation I think describes this all fairly well= >.
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >ying much of the same thing I am saying=2C but in language that has receive= >d
much more thought. Be sure to read the "rationale" part.


&n= >bsp=3B- Jay


From: jay.krell at cornell.edu<= >br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >[M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >: Tue=2C 19 Apr 2011 15:06:38 +0000

> >"> > > > > >Previously: user threads: all threads survived fork()
Previously: pthrea= >ds: only the thread calling fork() survived fork()
Now: user threads and= > pthreads: only the thread calling fork() survives fork()
This must be h= >ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >br> >This change depends on pthread_atfork=2C for pthreads and user threads.
= >It only really matters to the rare "fork + do more work" program=2C such as= > cvsupd.
Most programs either never fork=2C or exec almost immediately a= >fter fork.


pthread_atfork offers a good model.
A sort of "dis= >tributed" model.
You don't have to go and change all the calls to fork()= >.
Each module with a need to do something before/after fork=2C calls the= > central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >hat is needed.


No function pointer is needed.
Instead move th= >e code to m3core/thread.


If you really must not use -pthread=2C = >then you must implement pthread_atfork functionality
yourself and have a= >ll fork() calls go through your own fork() wrapper that cooperates
with = >your pthread_atfork replacement.
There is no free lunch -- there is a do= >wnside to this approach=2C as plain fork() calls
are ok and correct if p= >thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >ty with pthreads
 =3B user threads using pthread_atfork/-pthread r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >s this and
 =3B =3B not likely notice -- only fork() calls in f= >ork+do-more-work programs need the wrapper).


(Most of this has = >been explained multiple times=2C but people only pay attention
when they= > think it affects them. I'm guilty of the same thing.)


 =3B-= > Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >: m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >you or Tony could describe roughly what you think needs to be done
>= >=3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > that a decision that's made/described in
>=3B m3core/thread/m3makefil= >e needs to someone find its way to controlling
>=3B what C code gets c= >ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >irection into the thread library to pick up pthread_atfork (or not).
>= >=3B
>=3B But then again you still haven't explained why you made the = >user threads use
>=3B pthread_atfork. I just remove it from my instal= >lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >know if it breaks that program to do so.
>=3B
>=3B Mika
= >>=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >/
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >>=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >=3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >. In which case the necessary work might be done there?
>=3B >=3B>= >=3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >=3B >=3B>=3B pthread_atfork should not be needed under user threads.>>=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >=3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >=3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >.
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >>=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >=3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >=3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >=3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >>=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >=3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >>=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >=3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >=3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >=3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >=3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >=3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >>=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > The following code from RTProcessC.c ensures that it is neeeded on e=3D>>=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >* NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >=3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >"we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >'ll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >=3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >=3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >n the thread-specific portion of m3core. In which case the necessary =3D>>=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >=3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >=3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >=3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >=3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >=3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >=3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >=3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >=3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >_ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >=3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >>=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >=3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >=3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >d397d4-ac1d-4148-a9ff-059d27dd794a_--
>= > >--_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From mika at async.caltech.edu Wed Apr 20 08:12:17 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Tue, 19 Apr 2011 23:12:17 -0700 Subject: [M3devel] malloc/free from Modula-3? In-Reply-To: References: <20110415103737.E73B31A2078@async.async.caltech.edu>, Message-ID: <20110420061217.C539A1A207A@async.async.caltech.edu> Jay K writes: >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume. >Can you try sprinking a lot more Disable/EnableSwitching around? >I guess you tried that..and..? So far so good. I'll let you know tomorrow. (The application runs 8 1/2 hours a day and has been crashing about twice a day.) The annoying thing is that when it does lock up, it locks up in one of the wrapped free()s, not in the unwrapped one. Mika > > - Jay > > >> From: hosking at cs.purdue.edu >> Date: Fri=2C 15 Apr 2011 11:26:15 -0400 >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: Re: [M3devel] malloc/free from Modula-3? >>=20 >> I suspect the problem is re-entrance to the library. Should you be disabl= >ing user thread switching somewhere? >>=20 >> Antony Hosking | Associate Professor | Computer Science | Purdue Universi= >ty >> 305 N. University Street | West Lafayette | IN 47907 | USA >> Office +1 765 494 6001 | Mobile +1 765 427 5484 >>=20 >>=20 >>=20 >>=20 >> On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote: >>=20 >> > Hi m3devel=2C >> >=20 >> > I'm debugging a problem with some code I have that calls out to C >> > code. It's calling pqlib (PostgreSQL library) and you might expect >> > things to go wrong there=2C but as it turns out it's dying in CM3's cod= >e. >> >=20 >> > The system is AMD64_LINUX (user threading because of the problems with >> > pthreads) and the program is just freezing up hard. No segfault=2C >> > bus error or anything. It locks up a couple of times a day. >> >=20 >> > What are the rules on calling malloc/free from Modula-3? >> >=20 >> > Stack trace: >> >=20 >> > (gdb) where >> > #0 0x00002ac519a9202e in ?? () from /lib/libc.so.6 >> > #1 0x00002ac519a2fb43 in ?? () from /lib/libc.so.6 >> > #2 0x00002ac519a2caab in free () from /lib/libc.so.6 >> > #3 0x000000000091462f in Cstdlib__free (a=3DCannot access memory at ad= >dress 0x2ac73b0ef0d8 >> > ) at ../src/C/Common/CstdlibC.c:26 >> > #4 0x000000000091477d in M3toC__FreeCopiedS (s=3DCannot access memory = >at address 0x2ac73b0ef0f8 >> > ) at ../src/C/Common/M3toC.m3:46 >> > #5 0x00000000007884f0 in UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot= > access memory at address 0x2ac73b0ef118 >> > ) at ../src/UnsafeDatabase.m3:230 >> > #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access memo= >ry at address 0x2ac73b0ef148 >> > ) at ../src/UnsafeDatabase.m3:228 >> > #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef268 >> > ) at ../src/UnsafeDatabase.m3:258 >> > #8 0x000000000077ad5b in DesynchronizedDB__SExec (t=3DCannot access me= >mory at address 0x2ac73b0ef328 >> > ) at ../src/DesynchronizedDB.m3:126 >> > #9 0x000000000077c190 in DesynchronizedDB__SMExec (t=3DCannot access m= >emory at address 0x2ac73b0ef3e8 >> > ) at ../src/DesynchronizedDB.m3:293 >> > #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t=3DCannot a= >ccess memory at address 0x2ac73b0ef5a8 >> > ) at ../src/GCOMSOrderManager.m3:596 >> > #11 0x0000000000566b95 in GCOMSOrderManager__PlaceSingleOrder (t=3DCann= >ot access memory at address 0x2ac73b0ef668 >> > ) at ../src/GCOMSOrderManager.m3:310 >> > #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price=3D= >{}=2C count=3DCannot access memory at address 0x2ac73b0ef7b= >8 >> > ) at ../src/EuroDriver.m3:1108 >> > #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../sr= >c/EuroDriver.m3:1401 >> > #14 0x00000000004197a1 in EuroDriver__OApply (cl=3DCannot access memory= > at address 0x2ac73b0efd38 >> > ) at ../src/EuroDriver.m3:1545 >> > #15 0x00000000009112c1 in ThreadPosix__RunThread () at ../src/thread/PO= >SIX/ThreadPosix.m3:1174 >> > #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6 >> > #17 0x0000000000000000 in ?? () >> > (gdb)=20 >> >=20 >> > Here is the code that causes the problem: >> >=20 >> > VAR >> > s :=3D CopyTtoS(query)=3B >> > BEGIN >> > TRY >> > t.res :=3D t.myExec(s=2CbusyWait) >> > FINALLY >> > FreeCopiedS(s) (* stuck here *) >> > END >> > END=3B >> >=20 >> > The query is a SQL query---t.myExec eventually calls the libpq SQL exec >> > routine (PQexec I think it's called). >> >=20 >> > Since this is already an unsafe module I suppose I could just make an >> > array of CHARs on the stack and pass a pointer to that... >> >=20 >> >=20 >> > Regards=2C >> > Mika >>=20 > = > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_ >Content-Type: text/html; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > > > > > >Can you by chance see the other threads? I suspect not -- user threads I pr= >esume.
Can you try sprinking a lot more Disable/EnableSwitching around?<= >br>I guess you tried that..and..?

 =3B- Jay


>=3B Fr= >om: hosking at cs.purdue.edu
>=3B Date: Fri=2C 15 Apr 2011 11:26:15 -0400= >
>=3B To: mika at async.caltech.edu
>=3B CC: m3devel at elegosoft.comr>>=3B Subject: Re: [M3devel] malloc/free from Modula-3?
>=3B
&g= >t=3B I suspect the problem is re-entrance to the library. Should you be dis= >abling user thread switching somewhere?
>=3B
>=3B Antony Hosking= > | Associate Professor | Computer Science | Purdue University
>=3B 305= > N. University Street | West Lafayette | IN 47907 | USA
>=3B Office +1= > 765 494 6001 | Mobile +1 765 427 5484
>=3B
>=3B
>=3B
= >>=3B
>=3B On Apr 15=2C 2011=2C at 6:37 AM=2C Mika Nystrom wrote:>>=3B
>=3B >=3B Hi m3devel=2C
>=3B >=3B
>=3B >=3B = >I'm debugging a problem with some code I have that calls out to C
>=3B= > >=3B code. It's calling pqlib (PostgreSQL library) and you might expect= >
>=3B >=3B things to go wrong there=2C but as it turns out it's dyin= >g in CM3's code.
>=3B >=3B
>=3B >=3B The system is AMD64_LIN= >UX (user threading because of the problems with
>=3B >=3B pthreads) = >and the program is just freezing up hard. No segfault=2C
>=3B >=3B = >bus error or anything. It locks up a couple of times a day.
>=3B >= >=3B
>=3B >=3B What are the rules on calling malloc/free from Modula= >-3?
>=3B >=3B
>=3B >=3B Stack trace:
>=3B >=3B
&g= >t=3B >=3B (gdb) where
>=3B >=3B #0 0x00002ac519a9202e in ?? () fr= >om /lib/libc.so.6
>=3B >=3B #1 0x00002ac519a2fb43 in ?? () from /li= >b/libc.so.6
>=3B >=3B #2 0x00002ac519a2caab in free () from /lib/li= >bc.so.6
>=3B >=3B #3 0x000000000091462f in Cstdlib__free (a=3DCanno= >t access memory at address 0x2ac73b0ef0d8
>=3B >=3B ) at ../src/C/Co= >mmon/CstdlibC.c:26
>=3B >=3B #4 0x000000000091477d in M3toC__FreeCo= >piedS (s=3DCannot access memory at address 0x2ac73b0ef0f8
>=3B >=3B = >) at ../src/C/Common/M3toC.m3:46
>=3B >=3B #5 0x00000000007884f0 in= > UnsafeDatabase_M3_LINE_230.1181 (L_2=3DCannot access memory at address 0x2= >ac73b0ef118
>=3B >=3B ) at ../src/UnsafeDatabase.m3:230
>=3B &g= >t=3B #6 0x000000000078881f in UnsafeDatabase__ExecM (t=3DCannot access mem= >ory at address 0x2ac73b0ef148
>=3B >=3B ) at ../src/UnsafeDatabase.m= >3:228
>=3B >=3B #7 0x0000000000788aa6 in UnsafeDatabase__TExecM (t= >=3DCannot access memory at address 0x2ac73b0ef268
>=3B >=3B ) at ../= >src/UnsafeDatabase.m3:258
>=3B >=3B #8 0x000000000077ad5b in Desync= >hronizedDB__SExec (t=3DCannot access memory at address 0x2ac73b0ef328
&g= >t=3B >=3B ) at ../src/DesynchronizedDB.m3:126
>=3B >=3B #9 0x0000= >00000077c190 in DesynchronizedDB__SMExec (t=3DCannot access memory at addre= >ss 0x2ac73b0ef3e8
>=3B >=3B ) at ../src/DesynchronizedDB.m3:293
&= >gt=3B >=3B #10 0x0000000000569367 in GCOMSOrderManager__OrderEntered (t= >=3DCannot access memory at address 0x2ac73b0ef5a8
>=3B >=3B ) at ../= >src/GCOMSOrderManager.m3:596
>=3B >=3B #11 0x0000000000566b95 in GCO= >MSOrderManager__PlaceSingleOrder (t=3DCannot access memory at address 0x2ac= >73b0ef668
>=3B >=3B ) at ../src/GCOMSOrderManager.m3:310
>=3B &= >gt=3B #12 0x000000000041ba25 in EuroDriver__OApply__PlaceOrder.1253 (price= >=3D{<=3BNo data fields>=3B}=2C count=3DCannot access memory at address = >0x2ac73b0ef7b8
>=3B >=3B ) at ../src/EuroDriver.m3:1108
>=3B &g= >t=3B #13 0x000000000041d97c in EuroDriver__OApply__MainLoop.1289 () at ../s= >rc/EuroDriver.m3:1401
>=3B >=3B #14 0x00000000004197a1 in EuroDriver= >__OApply (cl=3DCannot access memory at address 0x2ac73b0efd38
>=3B >= >=3B ) at ../src/EuroDriver.m3:1545
>=3B >=3B #15 0x00000000009112c1 = >in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174
= >>=3B >=3B #16 0x00002ac5199fa7b0 in ?? () from /lib/libc.so.6
>=3B= > >=3B #17 0x0000000000000000 in ?? ()
>=3B >=3B (gdb)
>=3B &= >gt=3B
>=3B >=3B Here is the code that causes the problem:
>=3B= > >=3B
>=3B >=3B VAR
>=3B >=3B s :=3D CopyTtoS(quer= >y)=3B
>=3B >=3B BEGIN
>=3B >=3B TRY
>=3B >=3B = > t.res :=3D t.myExec(s=2CbusyWait)
>=3B >=3B FINALLY
&= >gt=3B >=3B FreeCopiedS(s) (* stuck here *)
>=3B >=3B E= >ND
>=3B >=3B END=3B
>=3B >=3B
>=3B >=3B The query = >is a SQL query---t.myExec eventually calls the libpq SQL exec
>=3B >= >=3B routine (PQexec I think it's called).
>=3B >=3B
>=3B >= >=3B Since this is already an unsafe module I suppose I could just make anr>>=3B >=3B array of CHARs on the stack and pass a pointer to that...r>>=3B >=3B
>=3B >=3B
>=3B >=3B Regards=2C
>=3B = >>=3B Mika
>=3B
>= > >--_e3b69b7a-6d95-4c2e-bef8-027b0afc16b6_-- From hosking at cs.purdue.edu Wed Apr 20 16:58:06 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 10:58:06 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110420055557.089101A207A@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , <20110407190219.C00E31A2078@async.async.caltech.edu>, , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , <20110418184540.C2D801A2078@async.async.caltech.edu>, , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , <20110419023058.4FBB11A207A@async.async.caltech.edu>, <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: Agreed. But CVSup used it and we were trying to be supportive. Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > The rationale you directed us to says: > > "It is suggested that programs that use fork() call an exec function very > soon afterwards in the child process, thus resetting all states. In the > meantime, only a short list of async-signal-safe library routines are > promised to be available." > > and later > > "Application programs may not be aware that a multi-threaded library > is in use, and they feel free to call any number of library routines > between the fork() and exec calls, just as they always have. Indeed, > they may be extant single-threaded programs and cannot, therefore, > be expected to obey new restrictions imposed by the threads library." > > In C maybe that is true. In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is > no such thing as a "single-threaded" Modula-3 program---or at least, > the distinction doesn't matter. > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > pattern used by some special program---maybe. In any case support for it > I do not think belongs in m3core. > > Mika > > > > Jay K writes: >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/plain; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >>> This change depends on pthread_atfork=2C for pthreads and user threads. >> >> >> Clarification: There's no change described there for pthreads=2C and indeed= >> =2C getting >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= >> just >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= >> fork()). >> The Solaris default used to be fork =3D> forkall(). >> The current Solaris default is fork =3D> fork1(). >> There really do exist on Solaris to functions you can call=2C fork1() and f= >> orkall(). >> All other systems and the Posix specification have the "fork1" semantic for= >> fork. >> Therefore "fork1" is what we should provide unconditionally. >> If you really want "forkall"=2C then you write: >> #ifdef __sun >> forkall() >> #else >> #error >> #endif >> >> >> >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= >> thread/POSIX_FORKALL or such. >> I don't expect anyone to ever do that. >> >> >> >> The need for pthread_atfork is sort of different for user threads vs. pthre= >> ads=2C but it is needed either way. >> For pthreads=2C there is this problem -- at the time a thread calls fork()= >> =2C what are other threads doing? >> Answer: Anything. >> Follow up question: What locks might they hold? >> Answer: Any. >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= >> xec=2C and the child process >> attempts to use some of the locks that were held in the parent=2C and only = >> the forking thread survived=2C >> then the other threads will never leave their locks=2C and later attempts t= >> o acquire them in the children >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= >> e supposed to use it for=2C >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = >> locks=2C and then fork()=2C and then in the >> parent and children=2C release all locks. Therefore=2C you have to come up = >> with a locking order. >> >> >> >> Things are a little different for user threads. >> In user threads=2C the existance of threads is established by virtue of a g= >> lobal array or list of data describing the threads. >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= >> uler picks another thread from the global >> array/list. Previously=2C the data went unchanged when fork() was called. T= >> herefore all threads survived. >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= >> itialize the various data=2C so as to forget about >> all but the current thread. Something like that. >> >> >> >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= >> rk=2C then ForkBeforeExec >> might get away with just fork() and ignore all this. >> So far we have not drawn this line -- there is just the one fork and it is = >> used prior to exec or do-more-work >> and does the same thing w/o knowing what is to follow. >> >> >> The Posix documentation I think describes this all fairly well. >> Bing for "posix opengroup pthread_atfork". >> Really. I think it is saying much of the same thing I am saying=2C but in l= >> anguage that has received >> much more thought. Be sure to read the "rationale" part. >> >> >> - Jay >> >> >> From: jay.krell at cornell.edu >> To: mika at async.caltech.edu >> CC: m3devel at elegosoft.com >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= >> 2.6-4 >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 >> >> >> >> >> >> >> >> >> Previously: user threads: all threads survived fork() >> Previously: pthreads: only the thread calling fork() survived fork() >> Now: user threads and pthreads: only the thread calling fork() survives for= >> k() >> This must be how pthreads behave=2C and this makes user threads and pthread= >> s consistent. >> >> This change depends on pthread_atfork=2C for pthreads and user threads. >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd. >> Most programs either never fork=2C or exec almost immediately after fork. >> >> >> pthread_atfork offers a good model. >> A sort of "distributed" model. >> You don't have to go and change all the calls to fork(). >> Each module with a need to do something before/after fork=2C calls the cent= >> ral pthread_atfork=2C >> and fork and pthread_atfork cooperate to do what is needed. >> >> >> No function pointer is needed. >> Instead move the code to m3core/thread. >> >> >> If you really must not use -pthread=2C then you must implement pthread_atfo= >> rk functionality >> yourself and have all fork() calls go through your own fork() wrapper that = >> cooperates >> with your pthread_atfork replacement. >> There is no free lunch -- there is a downside to this approach=2C as plain = >> fork() calls >> are ok and correct if pthread_atfork is used=2C but now become incorrect. >> Pick your poison: >> user thread/pthread inconsistency >> cvsupd incompatibility with pthreads=20 >> user threads using pthread_atfork/-pthread=20 >> fork() calls having to go through a wrapper (ok -- you could miss this an= >> d=20 >> not likely notice -- only fork() calls in fork+do-more-work programs nee= >> d the wrapper).=20 >> >> >> (Most of this has been explained multiple times=2C but people only pay atte= >> ntion >> when they think it affects them. I'm guilty of the same thing.) >> >> >> - Jay >> >> >>> To: jay.krell at cornell.edu >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 >>> From: mika at async.caltech.edu >>> CC: m3devel at elegosoft.com >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= >> c-2.6-4 >>> =20 >>> =20 >>> If you or Tony could describe roughly what you think needs to be done >>> I'd be happy to look into it. >>> =20 >>> The basic problem is that a decision that's made/described in >>> m3core/thread/m3makefile needs to someone find its way to controlling >>> what C code gets compiled elsewhere in the system. Or maybe there should >>> be an indirection into the thread library to pick up pthread_atfork (or n= >> ot). >>> =20 >>> But then again you still haven't explained why you made the user threads = >> use >>> pthread_atfork. I just remove it from my installations=2C but then again= >> I'm >>> not trying to run CVSup so I don't know if it breaks that program to do s= >> o. >>> =20 >>> Mika >>> =20 >>> Jay K writes: >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= >> ry =3D >>>> Unix except >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>> >>>> =3D20 >>>> Ok=3D2C "we" should probably go ahead and fix that. >>>> I'll try to=3D2C but no promises=3D2C sorry. >>>> =3D20 >>>> - Jay >>>> =3D20 >>>>> From: hosking at cs.purdue.edu >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 >>>>> To: mika at async.caltech.edu >>>>> CC: m3devel at elegosoft.com >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= >> 2C glib=3D >>>> c-2.6-4 >>>>> =3D20 >>>>> Probably unnecessary=3D2C given that I think there is another entry po= >> int t=3D >>>> o forking a process (I forget where) in the thread-specific portion of m= >> 3co=3D >>>> re. In which case the necessary work might be done there? >>>>> =3D20 >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: >>>>> =3D20 >>>>>> Tony Hosking writes: >>>>>> ... >>>>>>> pthread_atfork should not be needed under user threads. >>>>>> ... >>>>>> =3D20 >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= >> ver=3D >>>> y Unix except >>>>>> FreeBSD before 6. The comment is from the checked-in source file. >>>>>> =3D20 >>>>>> /* NOTE: Even userthreads now depends >>>>>> * on availability of pthreads. >>>>>> * This can be fixed if need be. >>>>>> */ >>>>>> =3D20 >>>>>> INTEGER >>>>>> __cdecl >>>>>> RTProcess__RegisterForkHandlers( >>>>>> ForkHandler prepare=3D2C >>>>>> ForkHandler parent=3D2C >>>>>> ForkHandler child) >>>>>> { >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. >>>>>> * Win32 lacks pthread_atfork and fork. OK. >>>>>> * >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement >>>>>> * atfork ourselves=3D2C as long as we provide a fork() >>>>>> * wrapper that code uses. >>>>>> */ >>>>>> #if defined(_WIN32) \ >>>>>> || defined(__vms) \ >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) >>>>>> return 0=3D3B >>>>>> #else >>>>>> while (1) >>>>>> { >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B >>>>>> if (i !=3D3D EAGAIN) >>>>>> return i=3D3B >>>>>> sleep(0)=3D3B >>>>>> } >>>>>> #endif >>>>>> } >>>>> =3D20 >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" >>>> Content-Transfer-Encoding: quoted-printable >>>> >>>> >>>> >>>> >>>> >>>> >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= >> nds
=3D >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = >> This can be=3D >>>> fixed if need be.
>=3D3B >=3D3B */

>>>>  =3D3B
>>>> Ok=3D2C "we" should probably go ahead and fix that.
>>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
>>>>  =3D3B
>>>>  =3D3B- Jay
 =3D3B
>>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= >> 011 18:11=3D >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= >> @elegos=3D >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= >> C kerne=3D >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= >> =3D2C given =3D >>>> that I think there is another entry point to forking a process (I forget= >> wh=3D >>>> ere) in the thread-specific portion of m3core. In which case the necessa= >> ry =3D >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= >> =3D2C at 2:45=3D >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= >> ing writes:=3D >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= >> ould not be n=3D >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = >>
>=3D3B =3D >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= >> on e=3D >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= >> rom the=3D >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= >> OTE: Even u=3D >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= >> s.
&=3D >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= >>
>=3D3B =3D >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= >> gt=3D3B >=3D3B =3D >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= >> e=3D2C>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= >> ler child)>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= >> hread_atfork.=3D >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= >> _atfork=3D >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= >> ead_atf=3D >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= >> =3D2C for all =3D >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= >> elves=3D2C =3D >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = >> uses.>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= >> =3D3B >=3D3B =3D >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= >> p=3D3B&am=3D >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B> R>>=3D3B >=3D >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= >> =3D3B >=3D3B in=3D >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= >> B >=3D3B if (=3D >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= >> sleep(0)=3D3B>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= >> BR>>=3D3B
=3D >>>> >>>> =3D >>>> >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- >> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ >> Content-Type: text/html; charset="iso-8859-1" >> Content-Transfer-Encoding: quoted-printable >> >> >> >> >> >> >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= >> er threads.


Clarification: There's no change described there for= >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= >> epend on using pthread_atfork. That is just
how pthreads are -- except o= >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= >> t we should provide unconditionally.
If you really want "forkall"=2C the= >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= >> t anyone to ever do that.



The need for pthread_atfork is sor= >> t of different for user threads vs. pthreads=2C but it is needed either way= >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = >> process
attempts to use some of the locks that were held in the parent= >> =2C and only the forking thread survived=2C
then the other threads will = >> never leave their locks=2C and later attempts to acquire them in the childr= >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= >> n the
parent and children=2C release all locks. Therefore=2C you have to= >> come up with a locking order.



Things are a little different= >> for user threads.
In user threads=2C the existance of threads is establ= >> ished by virtue of a global array or list of data describing the threads.> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= >> eduler picks another thread from the global
array/list. Previously=2C th= >> e data went unchanged when fork() was called. Therefore all threads survive= >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= >> s reinitialize the various data=2C so as to forget about
all but the cur= >> rent thread. Something like that.



I suspect if we had RTProc= >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= >> br>might get away with just fork() and ignore all this.
So far we have n= >> ot drawn this line -- there is just the one fork and it is used prior to ex= >> ec or do-more-work
and does the same thing w/o knowing what is to follow= >> .


The Posix documentation I think describes this all fairly well= >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= >> ying much of the same thing I am saying=2C but in language that has receive= >> d
much more thought. Be sure to read the "rationale" part.


&n= >> bsp=3B- Jay


From: jay.krell at cornell.edu<= >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= >> : Tue=2C 19 Apr 2011 15:06:38 +0000

>> >> > "> >> >> >> >> >> Previously: user threads: all threads survived fork()
Previously: pthrea= >> ds: only the thread calling fork() survived fork()
Now: user threads and= >> pthreads: only the thread calling fork() survives fork()
This must be h= >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= >> br> >> This change depends on pthread_atfork=2C for pthreads and user threads.
= >> It only really matters to the rare "fork + do more work" program=2C such as= >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= >> tributed" model.
You don't have to go and change all the calls to fork()= >> .
Each module with a need to do something before/after fork=2C calls the= >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= >> hat is needed.


No function pointer is needed.
Instead move th= >> e code to m3core/thread.


If you really must not use -pthread=2C = >> then you must implement pthread_atfork functionality
yourself and have a= >> ll fork() calls go through your own fork() wrapper that cooperates
with = >> your pthread_atfork replacement.
There is no free lunch -- there is a do= >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread > r> =3B fork() calls having to go through a wrapper (ok -- you could mis= >> s this and
 =3B =3B not likely notice -- only fork() calls in f= >> ork+do-more-work programs need the wrapper).


(Most of this has = >> been explained multiple times=2C but people only pay attention
when they= >> think it affects them. I'm guilty of the same thing.)


 =3B-= >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = >> you or Tony could describe roughly what you think needs to be done
>= >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= >> e needs to someone find its way to controlling
>=3B what C code gets c= >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= >> irection into the thread library to pick up pthread_atfork (or not).
>= >> =3B
>=3B But then again you still haven't explained why you made the = >> user threads use
>=3B pthread_atfork. I just remove it from my instal= >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = >> know if it breaks that program to do so.
>=3B
>=3B Mika
= >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= >> . In which case the necessary work might be done there?
>=3B >=3B>= >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= >> =3B >=3B>=3B pthread_atfork should not be needed under user threads.>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B}> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= >> The following code from RTProcessC.c ensures that it is neeeded on e=3D>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= >> n the thread-specific portion of m3core. In which case the necessary =3D>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
>> = >> >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- From jay.krell at cornell.edu Thu Apr 21 02:27:15 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 00:27:15 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." Any idea why? I don't know. Is this such a burden? We need a global lock order for it. Unreasonable? I understand pthread_atfork is causing a problem with user threads, but - It really ought not. The libc vs. libpthread design is broken. I believe many systems don't have this broken design, but I guess some do. The broken design is that libc and libpthread implement some functions with the same name, and you get to chose which one. And some of them might be thread-safe. > In Modula-3, every application has *always* > had to assume that there are other threads floating around. There is This is the case in C too. I can't assume any library I link to doesn't create some helper threads with pthreads. pthreads should be assumed always in use, and there should be no ramifications of this. I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread safe vs. ones that are. We really need pthreads to work and to worry less about user threads imho. I believe we have seen the same problems with Win32 threads as with pthreads. (Yes, I know, some people might protest that Win32 threads work fine, but pthreads work very well too. It is only the new stress testing that is turning up problems as I understand, and it turns up problems in pthreads and Win32 threads. Ok, there is now some new mention of problems with other programs?) I suspect the problem is in the allocator/garbage collector, but that user threads don't stress it enough. I suspect I suspect I suspect...whatever.. real debugging is needed, (my own) suspicion and speculation isn't going to help much. It would of course be good if someone anyone can find a point in the history where pthreads and Win32 threads are really solid, with these new one or two test programs, if such a point exists -- I'm not sure one does, or that recent changes are the problem. But, again, I don't know. Time, debugging, investigation is needed. - Jay > From: hosking at cs.purdue.edu > Date: Wed, 20 Apr 2011 10:58:06 -0400 > To: mika at async.caltech.edu > CC: m3devel at elegosoft.com; jay.krell at cornell.edu > Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > > Agreed. But CVSup used it and we were trying to be supportive. > > Antony Hosking | Associate Professor | Computer Science | Purdue University > 305 N. University Street | West Lafayette | IN 47907 | USA > Office +1 765 494 6001 | Mobile +1 765 427 5484 > > > > > On Apr 20, 2011, at 1:55 AM, Mika Nystrom wrote: > > > > > The rationale you directed us to says: > > > > "It is suggested that programs that use fork() call an exec function very > > soon afterwards in the child process, thus resetting all states. In the > > meantime, only a short list of async-signal-safe library routines are > > promised to be available." > > > > and later > > > > "Application programs may not be aware that a multi-threaded library > > is in use, and they feel free to call any number of library routines > > between the fork() and exec calls, just as they always have. Indeed, > > they may be extant single-threaded programs and cannot, therefore, > > be expected to obey new restrictions imposed by the threads library." > > > > In C maybe that is true. In Modula-3, every application has *always* > > had to assume that there are other threads floating around. There is > > no such thing as a "single-threaded" Modula-3 program---or at least, > > the distinction doesn't matter. > > > > I really think fork-and-do-more-work is just a bug. Maybe it is a special > > pattern used by some special program---maybe. In any case support for it > > I do not think belongs in m3core. > > > > Mika > > > > > > > > Jay K writes: > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/plain; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >>> This change depends on pthread_atfork=2C for pthreads and user threads. > >> > >> > >> Clarification: There's no change described there for pthreads=2C and indeed= > >> =2C getting > >> this behavior with pthreads doesn't depend on using pthread_atfork. That is= > >> just > >> how pthreads are -- except optionally on Solaris (fork1() vs. forkall() vs.= > >> fork()). > >> The Solaris default used to be fork =3D> forkall(). > >> The current Solaris default is fork =3D> fork1(). > >> There really do exist on Solaris to functions you can call=2C fork1() and f= > >> orkall(). > >> All other systems and the Posix specification have the "fork1" semantic for= > >> fork. > >> Therefore "fork1" is what we should provide unconditionally. > >> If you really want "forkall"=2C then you write: > >> #ifdef __sun > >> forkall() > >> #else > >> #error > >> #endif > >> > >> > >> > >> and/or you go and hack up m3core/src/thread/POSIX=2C branch off m3core/src/= > >> thread/POSIX_FORKALL or such. > >> I don't expect anyone to ever do that. > >> > >> > >> > >> The need for pthread_atfork is sort of different for user threads vs. pthre= > >> ads=2C but it is needed either way. > >> For pthreads=2C there is this problem -- at the time a thread calls fork()= > >> =2C what are other threads doing? > >> Answer: Anything. > >> Follow up question: What locks might they hold? > >> Answer: Any. > >> Follow up: So=2C if arbitrary locks are held=2C and you fork=2C and don't e= > >> xec=2C and the child process > >> attempts to use some of the locks that were held in the parent=2C and only = > >> the forking thread survived=2C > >> then the other threads will never leave their locks=2C and later attempts t= > >> o acquire them in the children > >> will deadlock. Therefore=2C what pthread_atfork provides for=2C what you ar= > >> e supposed to use it for=2C > >> is roughly speaking=2C right before fork()=2C in the parent=2C acquire all = > >> locks=2C and then fork()=2C and then in the > >> parent and children=2C release all locks. Therefore=2C you have to come up = > >> with a locking order. > >> > >> > >> > >> Things are a little different for user threads. > >> In user threads=2C the existance of threads is established by virtue of a g= > >> lobal array or list of data describing the threads. > >> And a timer to occasionally preempt. When the timer interrupts=2C the sched= > >> uler picks another thread from the global > >> array/list. Previously=2C the data went unchanged when fork() was called. T= > >> herefore all threads survived. > >> What we do now is that in the child "handler" for pthread_atfork=2C is rein= > >> itialize the various data=2C so as to forget about > >> all but the current thread. Something like that. > >> > >> > >> > >> I suspect if we had RTProcess.ForkBeforeExec and RTProcess.ForkBeforeMoreWo= > >> rk=2C then ForkBeforeExec > >> might get away with just fork() and ignore all this. > >> So far we have not drawn this line -- there is just the one fork and it is = > >> used prior to exec or do-more-work > >> and does the same thing w/o knowing what is to follow. > >> > >> > >> The Posix documentation I think describes this all fairly well. > >> Bing for "posix opengroup pthread_atfork". > >> Really. I think it is saying much of the same thing I am saying=2C but in l= > >> anguage that has received > >> much more thought. Be sure to read the "rationale" part. > >> > >> > >> - Jay > >> > >> > >> From: jay.krell at cornell.edu > >> To: mika at async.caltech.edu > >> CC: m3devel at elegosoft.com > >> Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-= > >> 2.6-4 > >> Date: Tue=2C 19 Apr 2011 15:06:38 +0000 > >> > >> > >> > >> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork() > >> Previously: pthreads: only the thread calling fork() survived fork() > >> Now: user threads and pthreads: only the thread calling fork() survives for= > >> k() > >> This must be how pthreads behave=2C and this makes user threads and pthread= > >> s consistent. > >> > >> This change depends on pthread_atfork=2C for pthreads and user threads. > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd. > >> Most programs either never fork=2C or exec almost immediately after fork. > >> > >> > >> pthread_atfork offers a good model. > >> A sort of "distributed" model. > >> You don't have to go and change all the calls to fork(). > >> Each module with a need to do something before/after fork=2C calls the cent= > >> ral pthread_atfork=2C > >> and fork and pthread_atfork cooperate to do what is needed. > >> > >> > >> No function pointer is needed. > >> Instead move the code to m3core/thread. > >> > >> > >> If you really must not use -pthread=2C then you must implement pthread_atfo= > >> rk functionality > >> yourself and have all fork() calls go through your own fork() wrapper that = > >> cooperates > >> with your pthread_atfork replacement. > >> There is no free lunch -- there is a downside to this approach=2C as plain = > >> fork() calls > >> are ok and correct if pthread_atfork is used=2C but now become incorrect. > >> Pick your poison: > >> user thread/pthread inconsistency > >> cvsupd incompatibility with pthreads=20 > >> user threads using pthread_atfork/-pthread=20 > >> fork() calls having to go through a wrapper (ok -- you could miss this an= > >> d=20 > >> not likely notice -- only fork() calls in fork+do-more-work programs nee= > >> d the wrapper).=20 > >> > >> > >> (Most of this has been explained multiple times=2C but people only pay atte= > >> ntion > >> when they think it affects them. I'm guilty of the same thing.) > >> > >> > >> - Jay > >> > >> > >>> To: jay.krell at cornell.edu > >>> Date: Mon=2C 18 Apr 2011 19:30:58 -0700 > >>> From: mika at async.caltech.edu > >>> CC: m3devel at elegosoft.com > >>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glib= > >> c-2.6-4 > >>> =20 > >>> =20 > >>> If you or Tony could describe roughly what you think needs to be done > >>> I'd be happy to look into it. > >>> =20 > >>> The basic problem is that a decision that's made/described in > >>> m3core/thread/m3makefile needs to someone find its way to controlling > >>> what C code gets compiled elsewhere in the system. Or maybe there should > >>> be an indirection into the thread library to pick up pthread_atfork (or n= > >> ot). > >>> =20 > >>> But then again you still haven't explained why you made the user threads = > >> use > >>> pthread_atfork. I just remove it from my installations=2C but then again= > >> I'm > >>> not trying to run CVSup so I don't know if it breaks that program to do s= > >> o. > >>> =20 > >>> Mika > >>> =20 > >>> Jay K writes: > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/plain=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>>> The following code from RTProcessC.c ensures that it is neeeded on eve= > >> ry =3D > >>>> Unix except > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>> > >>>> =3D20 > >>>> Ok=3D2C "we" should probably go ahead and fix that. > >>>> I'll try to=3D2C but no promises=3D2C sorry. > >>>> =3D20 > >>>> - Jay > >>>> =3D20 > >>>>> From: hosking at cs.purdue.edu > >>>>> Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400 > >>>>> To: mika at async.caltech.edu > >>>>> CC: m3devel at elegosoft.com > >>>>> Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kernel 2.6.23=3D= > >> 2C glib=3D > >>>> c-2.6-4 > >>>>> =3D20 > >>>>> Probably unnecessary=3D2C given that I think there is another entry po= > >> int t=3D > >>>> o forking a process (I forget where) in the thread-specific portion of m= > >> 3co=3D > >>>> re. In which case the necessary work might be done there? > >>>>> =3D20 > >>>>> On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mika Nystrom wrote: > >>>>> =3D20 > >>>>>> Tony Hosking writes: > >>>>>> ... > >>>>>>> pthread_atfork should not be needed under user threads. > >>>>>> ... > >>>>>> =3D20 > >>>>>> The following code from RTProcessC.c ensures that it is neeeded on e= > >> ver=3D > >>>> y Unix except > >>>>>> FreeBSD before 6. The comment is from the checked-in source file. > >>>>>> =3D20 > >>>>>> /* NOTE: Even userthreads now depends > >>>>>> * on availability of pthreads. > >>>>>> * This can be fixed if need be. > >>>>>> */ > >>>>>> =3D20 > >>>>>> INTEGER > >>>>>> __cdecl > >>>>>> RTProcess__RegisterForkHandlers( > >>>>>> ForkHandler prepare=3D2C > >>>>>> ForkHandler parent=3D2C > >>>>>> ForkHandler child) > >>>>>> { > >>>>>> /* FreeBSD < 6 lacks pthread_atfork. Would be good to use autoconf. > >>>>>> * VMS lacks pthread_atfork? Would be good to use autoconf. > >>>>>> * Win32 lacks pthread_atfork and fork. OK. > >>>>>> * > >>>>>> * As well=3D2C for all Posix systems=3D2C we could implement > >>>>>> * atfork ourselves=3D2C as long as we provide a fork() > >>>>>> * wrapper that code uses. > >>>>>> */ > >>>>>> #if defined(_WIN32) \ > >>>>>> || defined(__vms) \ > >>>>>> || (defined(__FreeBSD__) /* && (__FreeBSD__ < 6)*/ ) > >>>>>> return 0=3D3B > >>>>>> #else > >>>>>> while (1) > >>>>>> { > >>>>>> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B > >>>>>> if (i !=3D3D EAGAIN) > >>>>>> return i=3D3B > >>>>>> sleep(0)=3D3B > >>>>>> } > >>>>>> #endif > >>>>>> } > >>>>> =3D20 > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_ > >>>> Content-Type: text/html=3B charset=3D"iso-8859-1" > >>>> Content-Transfer-Encoding: quoted-printable > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B /* NOTE: Even userthreads now depe= > >> nds
=3D > >>>> >=3D3B >=3D3B * on availability of pthreads.
>=3D3B >=3D3B * = > >> This can be=3D > >>>> fixed if need be.
>=3D3B >=3D3B */

> >>>>  =3D3B
> >>>> Ok=3D2C "we" should probably go ahead and fix that.
> >>>> I'll try to=3D2C but no =3D3Bpromises=3D2C sorry.
> >>>>  =3D3B
> >>>>  =3D3B- Jay
 =3D3B
> >>>> >=3D3B From: hosking at cs.purdue.edu
>=3D3B Date: Mon=3D2C 18 Apr 2= > >> 011 18:11=3D > >>>> :26 -0400
>=3D3B To: mika at async.caltech.edu
>=3D3B CC: m3devel= > >> @elegos=3D > >>>> oft.com
>=3D3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage=3D2= > >> C kerne=3D > >>>> l 2.6.23=3D2C glibc-2.6-4
>=3D3B
>=3D3B Probably unnecessary= > >> =3D2C given =3D > >>>> that I think there is another entry point to forking a process (I forget= > >> wh=3D > >>>> ere) in the thread-specific portion of m3core. In which case the necessa= > >> ry =3D > >>>> work might be done there?
>=3D3B
>=3D3B On Apr 18=3D2C 2011= > >> =3D2C at 2:45=3D > >>>> PM=3D2C Mika Nystrom wrote:
>=3D3B
>=3D3B >=3D3B Tony Hosk= > >> ing writes:=3D > >>>>
>=3D3B >=3D3B ...
>=3D3B >=3D3B>=3D3B pthread_atfork sh= > >> ould not be n=3D > >>>> eeded under user threads.
>=3D3B >=3D3B ...
>=3D3B >=3D3B = > >>
>=3D3B =3D > >>>> >=3D3B The following code from RTProcessC.c ensures that it is neeeded= > >> on e=3D > >>>> very Unix except
>=3D3B >=3D3B FreeBSD before 6. The comment is f= > >> rom the=3D > >>>> checked-in source file.
>=3D3B >=3D3B
>=3D3B >=3D3B /* N= > >> OTE: Even u=3D > >>>> serthreads now depends
>=3D3B >=3D3B * on availability of pthread= > >> s.
&=3D > >>>> gt=3D3B >=3D3B * This can be fixed if need be.
>=3D3B >=3D3B */= > >>
>=3D3B =3D > >>>> >=3D3B
>=3D3B >=3D3B INTEGER
>=3D3B >=3D3B __cdecl
&= > >> gt=3D3B >=3D3B =3D > >>>> RTProcess__RegisterForkHandlers(
>=3D3B >=3D3B ForkHandler prepar= > >> e=3D2C >>>>> >=3D3B >=3D3B ForkHandler parent=3D2C
>=3D3B >=3D3B ForkHand= > >> ler child) >>>>> >=3D3B >=3D3B {
>=3D3B >=3D3B /* FreeBSD <=3D3B 6 lacks pt= > >> hread_atfork.=3D > >>>> Would be good to use autoconf.
>=3D3B >=3D3B * VMS lacks pthread= > >> _atfork=3D > >>>> ? Would be good to use autoconf.
>=3D3B >=3D3B * Win32 lacks pthr= > >> ead_atf=3D > >>>> ork and fork. OK.
>=3D3B >=3D3B *
>=3D3B >=3D3B * As well= > >> =3D2C for all =3D > >>>> Posix systems=3D2C we could implement
>=3D3B >=3D3B * atfork ours= > >> elves=3D2C =3D > >>>> as long as we provide a fork()
>=3D3B >=3D3B * wrapper that code = > >> uses. >>>> R>>=3D3B >=3D3B */
>=3D3B >=3D3B #if defined(_WIN32) \
>= > >> =3D3B >=3D3B =3D > >>>> || defined(__vms) \
>=3D3B >=3D3B || (defined(__FreeBSD__) /* &am= > >> p=3D3B&am=3D > >>>> p=3D3B (__FreeBSD__ <=3D3B 6)*/ )
>=3D3B >=3D3B return 0=3D3B >> R>>=3D3B >=3D > >>>> =3D3B #else
>=3D3B >=3D3B while (1)
>=3D3B >=3D3B {
>= > >> =3D3B >=3D3B in=3D > >>>> t i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3D3= > >> B >=3D3B if (=3D > >>>> i !=3D3D EAGAIN)
>=3D3B >=3D3B return i=3D3B
>=3D3B >=3D3B= > >> sleep(0)=3D3B >>>>> >=3D3B >=3D3B }
>=3D3B >=3D3B #endif
>=3D3B >=3D3B }<= > >> BR>>=3D3B
=3D > >>>> > >>>> =3D > >>>> > >>>> --_3dd397d4-ac1d-4148-a9ff-059d27dd794a_-- > >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_ > >> Content-Type: text/html; charset="iso-8859-1" > >> Content-Transfer-Encoding: quoted-printable > >> > >> > >> > >> > >> > >> > >>  =3B>=3B This change depends on pthread_atfork=2C for pthreads and us= > >> er threads.


Clarification: There's no change described there for= > >> pthreads=2C and indeed=2C getting
this behavior with pthreads doesn't d= > >> epend on using pthread_atfork. That is just
how pthreads are -- except o= > >> ptionally on Solaris (fork1() vs. forkall() vs. fork()).
The Solaris def= > >> ault used to be fork =3D>=3B forkall().
The current Solaris default is= > >> fork =3D>=3B fork1().
There really do exist on Solaris to functions y= > >> ou can call=2C fork1() and forkall().
All other systems and the Posix sp= > >> ecification have the "fork1" semantic for fork.
Therefore "fork1" is wha= > >> t we should provide unconditionally.
If you really want "forkall"=2C the= > >> n you write:
#ifdef __sun
 =3B forkall()
#else
 =3B #er= > >> ror
#endif



and/or you go and hack up m3core/src/thread/PO= > >> SIX=2C branch off m3core/src/thread/POSIX_FORKALL or such.
I don't expec= > >> t anyone to ever do that.



The need for pthread_atfork is sor= > >> t of different for user threads vs. pthreads=2C but it is needed either way= > >> .
For pthreads=2C there is this problem -- at the time a thread calls fo= > >> rk()=2C what are other threads doing?
Answer: Anything.
Follow up que= > >> stion: What locks might they hold?
Answer: Any.
Follow up: So=2C if a= > >> rbitrary locks are held=2C and you fork=2C and don't exec=2C and the child = > >> process
attempts to use some of the locks that were held in the parent= > >> =2C and only the forking thread survived=2C
then the other threads will = > >> never leave their locks=2C and later attempts to acquire them in the childr= > >> en
will deadlock. Therefore=2C what pthread_atfork provides for=2C what = > >> you are supposed to use it for=2C
is roughly speaking=2C right before fo= > >> rk()=2C in the parent=2C acquire all locks=2C and then fork()=2C and then i= > >> n the
parent and children=2C release all locks. Therefore=2C you have to= > >> come up with a locking order.



Things are a little different= > >> for user threads.
In user threads=2C the existance of threads is establ= > >> ished by virtue of a global array or list of data describing the threads. >> r>And a timer to occasionally preempt. When the timer interrupts=2C the sch= > >> eduler picks another thread from the global
array/list. Previously=2C th= > >> e data went unchanged when fork() was called. Therefore all threads survive= > >> d.
What we do now is that in the child "handler" for pthread_atfork=2C i= > >> s reinitialize the various data=2C so as to forget about
all but the cur= > >> rent thread. Something like that.



I suspect if we had RTProc= > >> ess.ForkBeforeExec and RTProcess.ForkBeforeMoreWork=2C then ForkBeforeExec<= > >> br>might get away with just fork() and ignore all this.
So far we have n= > >> ot drawn this line -- there is just the one fork and it is used prior to ex= > >> ec or do-more-work
and does the same thing w/o knowing what is to follow= > >> .


The Posix documentation I think describes this all fairly well= > >> .
Bing for "posix opengroup pthread_atfork".
Really. I think it is sa= > >> ying much of the same thing I am saying=2C but in language that has receive= > >> d
much more thought. Be sure to read the "rationale" part.


&n= > >> bsp=3B- Jay


From: jay.krell at cornell.edu<= > >> br>To: mika at async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: = > >> [M3devel] 5.8.6 LINUXLIBC6 breakage=2C kernel 2.6.23=2C glibc-2.6-4
Date= > >> : Tue=2C 19 Apr 2011 15:06:38 +0000

> >> > >> >> "> > >> > >> > >> > >> > >> Previously: user threads: all threads survived fork()
Previously: pthrea= > >> ds: only the thread calling fork() survived fork()
Now: user threads and= > >> pthreads: only the thread calling fork() survives fork()
This must be h= > >> ow pthreads behave=2C and this makes user threads and pthreads consistent.<= > >> br> > >> This change depends on pthread_atfork=2C for pthreads and user threads.
= > >> It only really matters to the rare "fork + do more work" program=2C such as= > >> cvsupd.
Most programs either never fork=2C or exec almost immediately a= > >> fter fork.


pthread_atfork offers a good model.
A sort of "dis= > >> tributed" model.
You don't have to go and change all the calls to fork()= > >> .
Each module with a need to do something before/after fork=2C calls the= > >> central pthread_atfork=2C
and fork and pthread_atfork cooperate to do w= > >> hat is needed.


No function pointer is needed.
Instead move th= > >> e code to m3core/thread.


If you really must not use -pthread=2C = > >> then you must implement pthread_atfork functionality
yourself and have a= > >> ll fork() calls go through your own fork() wrapper that cooperates
with = > >> your pthread_atfork replacement.
There is no free lunch -- there is a do= > >> wnside to this approach=2C as plain fork() calls
are ok and correct if p= > >> thread_atfork is used=2C but now become incorrect.
Pick your poison:
= > >>  =3B user thread/pthread inconsistency
 =3B cvsupd incompatibili= > >> ty with pthreads
 =3B user threads using pthread_atfork/-pthread >> r> =3B fork() calls having to go through a wrapper (ok -- you could mis= > >> s this and
 =3B =3B not likely notice -- only fork() calls in f= > >> ork+do-more-work programs need the wrapper).


(Most of this has = > >> been explained multiple times=2C but people only pay attention
when they= > >> think it affects them. I'm guilty of the same thing.)


 =3B-= > >> Jay


>=3B To: jay.krell at cornell.edu
>=3B Date: Mon=2C 18 = > >> Apr 2011 19:30:58 -0700
>=3B From: mika at async.caltech.edu
>=3B CC= > >> : m3devel at elegosoft.com
>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 b= > >> reakage=2C kernel 2.6.23=2C glibc-2.6-4
>=3B
>=3B
>=3B If = > >> you or Tony could describe roughly what you think needs to be done
>= > >> =3B I'd be happy to look into it.
>=3B
>=3B The basic problem is= > >> that a decision that's made/described in
>=3B m3core/thread/m3makefil= > >> e needs to someone find its way to controlling
>=3B what C code gets c= > >> ompiled elsewhere in the system. Or maybe there should
>=3B be an ind= > >> irection into the thread library to pick up pthread_atfork (or not).
>= > >> =3B
>=3B But then again you still haven't explained why you made the = > >> user threads use
>=3B pthread_atfork. I just remove it from my instal= > >> lations=2C but then again I'm
>=3B not trying to run CVSup so I don't = > >> know if it breaks that program to do so.
>=3B
>=3B Mika
= > >> >=3B
>=3B Jay K writes:
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-= > >> 059d27dd794a_
>=3B >=3BContent-Type: text/plain=3B charset=3D"iso-88= > >> 59-1"
>=3B >=3BContent-Transfer-Encoding: quoted-printable
>=3B= > >> >=3B
>=3B >=3B
>=3B >=3B>=3B The following code from RTP= > >> rocessC.c ensures that it is neeeded on every =3D
>=3B >=3BUnix exce= > >> pt
>=3B >=3B>=3B >=3B /* NOTE: Even userthreads now depends
&= > >> gt=3B >=3B>=3B >=3B * on availability of pthreads.
>=3B >=3B&g= > >> t=3B >=3B * This can be fixed if need be.
>=3B >=3B>=3B >=3B *= > >> /
>=3B >=3B
>=3B >=3B=3D20
>=3B >=3BOk=3D2C "we" shoul= > >> d probably go ahead and fix that.
>=3B >=3BI'll try to=3D2C but no p= > >> romises=3D2C sorry.
>=3B >=3B=3D20
>=3B >=3B - Jay
>=3B = > >> >=3B=3D20
>=3B >=3B>=3B From: hosking at cs.purdue.edu
>=3B &g= > >> t=3B>=3B Date: Mon=3D2C 18 Apr 2011 18:11:26 -0400
>=3B >=3B>=3B= > >> To: mika at async.caltech.edu
>=3B >=3B>=3B CC: m3devel at elegosoft.co= > >> m
>=3B >=3B>=3B Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage= > >> =3D2C kernel 2.6.23=3D2C glib=3D
>=3B >=3Bc-2.6-4
>=3B >=3B&g= > >> t=3B=3D20
>=3B >=3B>=3B Probably unnecessary=3D2C given that I thi= > >> nk there is another entry point t=3D
>=3B >=3Bo forking a process (I= > >> forget where) in the thread-specific portion of m3co=3D
>=3B >=3Bre= > >> . In which case the necessary work might be done there?
>=3B >=3B>= > >> =3B=3D20
>=3B >=3B>=3B On Apr 18=3D2C 2011=3D2C at 2:45 PM=3D2C Mi= > >> ka Nystrom wrote:
>=3B >=3B>=3B=3D20
>=3B >=3B>=3B >=3B= > >> Tony Hosking writes:
>=3B >=3B>=3B >=3B ...
>=3B >=3B>= > >> =3B >=3B>=3B pthread_atfork should not be needed under user threads. >>> >=3B >=3B>=3B >=3B ...
>=3B >=3B>=3B >=3B=3D20
>= > >> =3B >=3B>=3B >=3B The following code from RTProcessC.c ensures that i= > >> t is neeeded on ever=3D
>=3B >=3By Unix except
>=3B >=3B>= > >> =3B >=3B FreeBSD before 6. The comment is from the checked-in source file= > >> .
>=3B >=3B>=3B >=3B=3D20
>=3B >=3B>=3B >=3B /* NOTE:= > >> Even userthreads now depends
>=3B >=3B>=3B >=3B * on availabili= > >> ty of pthreads.
>=3B >=3B>=3B >=3B * This can be fixed if need b= > >> e.
>=3B >=3B>=3B >=3B */
>=3B >=3B>=3B >=3B=3D20
&= > >> gt=3B >=3B>=3B >=3B INTEGER
>=3B >=3B>=3B >=3B __cdecl
= > >> >=3B >=3B>=3B >=3B RTProcess__RegisterForkHandlers(
>=3B >= > >> =3B>=3B >=3B ForkHandler prepare=3D2C
>=3B >=3B>=3B >=3B For= > >> kHandler parent=3D2C
>=3B >=3B>=3B >=3B ForkHandler child)
&g= > >> t=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B /* FreeBSD <=3B = > >> 6 lacks pthread_atfork. Would be good to use autoconf.
>=3B >=3B>= > >> =3B >=3B * VMS lacks pthread_atfork? Would be good to use autoconf.
&g= > >> t=3B >=3B>=3B >=3B * Win32 lacks pthread_atfork and fork. OK.
>= > >> =3B >=3B>=3B >=3B *
>=3B >=3B>=3B >=3B * As well=3D2C for = > >> all Posix systems=3D2C we could implement
>=3B >=3B>=3B >=3B * a= > >> tfork ourselves=3D2C as long as we provide a fork()
>=3B >=3B>=3B = > >> >=3B * wrapper that code uses.
>=3B >=3B>=3B >=3B */
>=3B= > >> >=3B>=3B >=3B #if defined(_WIN32) \
>=3B >=3B>=3B >=3B ||= > >> defined(__vms) \
>=3B >=3B>=3B >=3B || (defined(__FreeBSD__) /*= > >> &=3B&=3B (__FreeBSD__ <=3B 6)*/ )
>=3B >=3B>=3B >=3B re= > >> turn 0=3D3B
>=3B >=3B>=3B >=3B #else
>=3B >=3B>=3B >= > >> =3B while (1)
>=3B >=3B>=3B >=3B {
>=3B >=3B>=3B >=3B= > >> int i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B
>=3B = > >> >=3B>=3B >=3B if (i !=3D3D EAGAIN)
>=3B >=3B>=3B >=3B retu= > >> rn i=3D3B
>=3B >=3B>=3B >=3B sleep(0)=3D3B
>=3B >=3B>= > >> =3B >=3B }
>=3B >=3B>=3B >=3B #endif
>=3B >=3B>=3B &g= > >> t=3B }
>=3B >=3B>=3B=3D20
>=3B >=3B =3D
>= > >> =3B >=3B
>=3B >=3B--_3dd397d4-ac1d-4148-a9ff-059d27dd794a_
>= > >> =3B >=3BContent-Type: text/html=3B charset=3D"iso-8859-1"
>=3B >= > >> =3BContent-Transfer-Encoding: quoted-printable
>=3B >=3B
>=3B &= > >> gt=3B<=3Bhtml>=3B
>=3B >=3B<=3Bhead>=3B
>=3B >=3B<= > >> =3Bstyle>=3B<=3B!--
>=3B >=3B.hmmessage P
>=3B >=3B{
&= > >> gt=3B >=3Bmargin:0px=3D3B
>=3B >=3Bpadding:0px
>=3B >=3B} >> r>>=3B >=3Bbody.hmmessage
>=3B >=3B{
>=3B >=3Bfont-size: = > >> 10pt=3D3B
>=3B >=3Bfont-family:Tahoma
>=3B >=3B}
>=3B &g= > >> t=3B-->=3B<=3B/style>=3B
>=3B >=3B<=3B/head>=3B
>=3B = > >> >=3B<=3Bbody class=3D3D'hmmessage'>=3B
>=3B >=3B&=3Bgt=3D3B= > >> The following code from RTProcessC.c ensures that it is neeeded on e=3D >>> >=3B >=3Bvery Unix except<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /= > >> * NOTE: Even userthreads now depends<=3BBR>=3B=3D
>=3B >=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * on availability of pthreads.<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B * This can be=3D
>=3B >=3B fixed if need b= > >> e.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B */<=3BBR>=3B<=3BBR>= > >> =3B
>=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3BOk=3D2C = > >> "we" should probably go ahead and fix that.<=3BBR>=3B
>=3B >=3BI= > >> 'll try to=3D2C but no&=3Bnbsp=3D3Bpromises=3D2C sorry.<=3BBR>=3B >>> >=3B >=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3Bnbsp=3D= > >> 3B- Jay<=3BBR>=3B&=3Bnbsp=3D3B<=3BBR>=3B
>=3B >=3B&=3B= > >> gt=3D3B From: hosking at cs.purdue.edu<=3BBR>=3B&=3Bgt=3D3B Date: Mon= > >> =3D2C 18 Apr 2011 18:11=3D
>=3B >=3B:26 -0400<=3BBR>=3B&=3Bgt= > >> =3D3B To: mika at async.caltech.edu<=3BBR>=3B&=3Bgt=3D3B CC: m3devel at el= > >> egos=3D
>=3B >=3Boft.com<=3BBR>=3B&=3Bgt=3D3B Subject: Re: [M= > >> 3devel] 5.8.6 LINUXLIBC6 breakage=3D2C kerne=3D
>=3B >=3Bl 2.6.23=3D= > >> 2C glibc-2.6-4<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B Pro= > >> bably unnecessary=3D2C given =3D
>=3B >=3Bthat I think there is anot= > >> her entry point to forking a process (I forget wh=3D
>=3B >=3Bere) i= > >> n the thread-specific portion of m3core. In which case the necessary =3D >>> >=3B >=3Bwork might be done there?<=3BBR>=3B&=3Bgt=3D3B <=3BB= > >> R>=3B&=3Bgt=3D3B On Apr 18=3D2C 2011=3D2C at 2:45=3D
>=3B >=3B = > >> PM=3D2C Mika Nystrom wrote:<=3BBR>=3B&=3Bgt=3D3B <=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B Tony Hosking writes:=3D
>=3B >=3B<=3BBR&= > >> gt=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BBR>=3B&=3Bgt=3D3B &=3Bg= > >> t=3D3B&=3Bgt=3D3B pthread_atfork should not be n=3D
>=3B >=3Beede= > >> d under user threads.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ...<=3BB= > >> R>=3B&=3Bgt=3D3B &=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B =3D
&g= > >> t=3B >=3B&=3Bgt=3D3B The following code from RTProcessC.c ensures that= > >> it is neeeded on e=3D
>=3B >=3Bvery Unix except<=3BBR>=3B&= > >> =3Bgt=3D3B &=3Bgt=3D3B FreeBSD before 6. The comment is from the=3D
&= > >> gt=3B >=3B checked-in source file.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B /* NOTE: Even u=3D
>= > >> =3B >=3Bserthreads now depends<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B= > >> * on availability of pthreads.<=3BBR>=3B&=3B=3D
>=3B >=3Bgt= > >> =3D3B &=3Bgt=3D3B * This can be fixed if need be.<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B =3D
>=3B >=3B&am= > >> p=3Bgt=3D3B <=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B INTEGER<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B __cdecl<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B =3D
>=3B >=3BRTProcess__RegisterForkHandlers(<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler prepare=3D2C<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler parent=3D2C<=3B= > >> BR>=3B&=3Bgt=3D3B &=3Bgt=3D3B ForkHandler child)<=3BBR=3D
>= > >> =3B >=3B>=3B&=3Bgt=3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B /* FreeBSD &=3Blt=3D3B 6 lacks pthread_atfork.=3D
>= > >> =3B >=3B Would be good to use autoconf.<=3BBR>=3B&=3Bgt=3D3B &= > >> =3Bgt=3D3B * VMS lacks pthread_atfork=3D
>=3B >=3B? Would be good to= > >> use autoconf.<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * Win32 lacks pth= > >> read_atf=3D
>=3B >=3Bork and fork. OK.<=3BBR>=3B&=3Bgt=3D3B &= > >> amp=3Bgt=3D3B *<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B * As well=3D2C f= > >> or all =3D
>=3B >=3BPosix systems=3D2C we could implement<=3BBR>= > >> =3B&=3Bgt=3D3B &=3Bgt=3D3B * atfork ourselves=3D2C =3D
>=3B >= > >> =3Bas long as we provide a fork()<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3= > >> B * wrapper that code uses.<=3BB=3D
>=3B >=3BR>=3B&=3Bgt=3D3B= > >> &=3Bgt=3D3B */<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B #if defined(_= > >> WIN32) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B =3D
>=3B >=3B|| = > >> defined(__vms) \<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B || (defined(__F= > >> reeBSD__) /* &=3Bamp=3D3B&=3Bam=3D
>=3B >=3Bp=3D3B (__FreeBSD_= > >> _ &=3Blt=3D3B 6)*/ )<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return 0= > >> =3D3B<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D
>=3B >=3B=3D3B #else&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B while (1)<=3BBR>=3B&=3Bgt= > >> =3D3B &=3Bgt=3D3B {<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B in=3D
= > >> >=3B >=3Bt i =3D3D pthread_atfork(prepare=3D2C parent=3D2C child)=3D3B&= > >> lt=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B if (=3D
>=3B >=3Bi !=3D3D= > >> EAGAIN)<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B return i=3D3B<=3BBR&g= > >> t=3B&=3Bgt=3D3B &=3Bgt=3D3B sleep(0)=3D3B<=3BBR=3D
>=3B >=3B= > >> >=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&=3Bgt=3D3B &=3Bgt= > >> =3D3B #endif<=3BBR>=3B&=3Bgt=3D3B &=3Bgt=3D3B }<=3BBR>=3B&= > >> =3Bgt=3D3B <=3BBR>=3B =3D
>=3B >=3B <=3B/body>=3B<= > >> br>>=3B >=3B<=3B/html>=3B=3D
>=3B >=3B
>=3B >=3B--_3d= > >> d397d4-ac1d-4148-a9ff-059d27dd794a_--
> >> = > >> > >> --_0203aac2-5c8e-49c7-9074-6bc448ecf342_-- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Thu Apr 21 04:02:47 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:02:47 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, Message-ID: <20110421020247.31A0A1A2078@async.async.caltech.edu> Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From mika at async.caltech.edu Thu Apr 21 04:53:04 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 19:53:04 -0700 Subject: [M3devel] wrapping postgresql library not enough Message-ID: <20110421025304.685501A2078@async.async.caltech.edu> Hi again, So, as I described before, I wrote a program that wraps all of Critical Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency at which my application locks up, but it has not eliminated the lockups. User threads, AMD64_LINUX. I'm not quite sure what to try next. How can I find out where malloc and free might be used reentrantly, assuming that is the problem? There may not be any reliable threading in CM3 anymore. Possibly it's reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. Mika (gdb) where #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 #3 0x000000000077b08d in PQsendQuery () #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 ) at ../AMD64_LINUX/PQSchedulerWrap.m3:210 #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 ) at ../src/UnsafeDatabase.m3:92 #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 ) at ../src/UnsafeDatabase.m3:239 #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 ) at ../src/UnsafeDatabase.m3:269 #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 ) at ../src/DesynchronizedDB.m3:74 #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 ) at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 #16 0x0000000000000000 in ?? () (gdb) From hosking at cs.purdue.edu Thu Apr 21 05:12:53 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:12:53 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421025304.685501A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> Message-ID: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Can you dump all the user threads at this point? ThreadPosix__DumpEverybody() should do it. We want to see where the other threads might be. On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > Hi again, > > So, as I described before, I wrote a program that wraps all of Critical > Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / > Scheduler.EnableSwitching. Yes, it seems to have reduced the frequency > at which my application locks up, but it has not eliminated the lockups. > > User threads, AMD64_LINUX. > > I'm not quite sure what to try next. How can I find out where malloc > and free might be used reentrantly, assuming that is the problem? > > There may not be any reliable threading in CM3 anymore. Possibly it's > reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. > > Mika > > (gdb) where > #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 > #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 > #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 > #3 0x000000000077b08d in PQsendQuery () > #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=Cannot access memory at address 0x7f7b848a4298 > ) > at ../AMD64_LINUX/PQSchedulerWrap.m3:210 > #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=Cannot access memory at address 0x7f7b848a4328 > ) at ../src/UnsafeDatabase.m3:92 > #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=Cannot access memory at address 0x7f7b848a43f8 > ) at ../src/UnsafeDatabase.m3:239 > #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=Cannot access memory at address 0x7f7b848a4528 > ) at ../src/UnsafeDatabase.m3:269 > #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=Cannot access memory at address 0x7f7b848a45e8 > ) at ../src/DesynchronizedDB.m3:74 > #9 0x00000000005aeac6 in DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 (finalAttempt=Cannot access memory at address 0x7f7b848a463f > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty (cl=Cannot access memory at address 0x7f7b848a4778 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > #11 0x00000000005b171e in DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=Cannot access memory at address 0x7f7b848a4958 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync (cl=Cannot access memory at address 0x7f7b848a4ba8 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC (cl=Cannot access memory at address 0x7f7b848a4d58 > ) > at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 => /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > #14 0x0000000000922eb5 in ThreadPosix__RunThread () at ../src/thread/POSIX/ThreadPosix.m3:1174 > #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 > #16 0x0000000000000000 in ?? () > (gdb) From mika at async.caltech.edu Thu Apr 21 05:20:44 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Wed, 20 Apr 2011 20:20:44 -0700 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> Message-ID: <20110421032044.9BDAA1A2078@async.async.caltech.edu> Aha, I will try that if it happens again. I wrapped more things that I think might be iffy in my program. But I have a feeling ThreadPosix__DumpEverybody won't work... the application is probably deadlocked by having a single thread locking against itself with some sort of pthread lock. I should look at the source code of free. Mika Tony Hosking writes: >Can you dump all the user threads at this point? = >ThreadPosix__DumpEverybody() should do it. We want to see where the = >other threads might be. > >On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: > >> Hi again, >>=20 >> So, as I described before, I wrote a program that wraps all of = >Critical >> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >frequency >> at which my application locks up, but it has not eliminated the = >lockups. >>=20 >> User threads, AMD64_LINUX. >>=20 >> I'm not quite sure what to try next. How can I find out where malloc >> and free might be used reentrantly, assuming that is the problem? >>=20 >> There may not be any reliable threading in CM3 anymore. Possibly it's >> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>=20 >> Mika >>=20 >> (gdb) where >> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >> #3 0x000000000077b08d in PQsendQuery () >> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >access memory at address 0x7f7b848a4298 >> ) >> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >memory at address 0x7f7b848a4328 >> ) at ../src/UnsafeDatabase.m3:92 >> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >memory at address 0x7f7b848a43f8 >> ) at ../src/UnsafeDatabase.m3:239 >> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >memory at address 0x7f7b848a4528 >> ) at ../src/UnsafeDatabase.m3:269 >> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >access memory at address 0x7f7b848a45e8 >> ) at ../src/DesynchronizedDB.m3:74 >> #9 0x00000000005aeac6 in = >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >(finalAttempt=3DCannot access memory at address 0x7f7b848a463f >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >(cl=3DCannot access memory at address 0x7f7b848a4778 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >> #11 0x00000000005b171e in = >DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >at address 0x7f7b848a4958 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >(cl=3DCannot access memory at address 0x7f7b848a4ba8 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >(cl=3DCannot access memory at address 0x7f7b848a4d58 >> ) >> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >../src/thread/POSIX/ThreadPosix.m3:1174 >> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >> #16 0x0000000000000000 in ?? () >> (gdb)=20 From hosking at cs.purdue.edu Thu Apr 21 05:46:37 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 20 Apr 2011 23:46:37 -0400 Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421032044.9BDAA1A2078@async.async.caltech.edu> References: <20110421025304.685501A2078@async.async.caltech.edu> <576D88CF-3713-4C46-BC70-FE3D1AEFBB5C@cs.purdue.edu> <20110421032044.9BDAA1A2078@async.async.caltech.edu> Message-ID: <2EDBC39D-053F-4C7A-AD64-7D202E080581@cs.purdue.edu> If you are able to get the debugger to stop there then you should still be able to dump the threads from the debugger: call ThreadPosix__DumpEverybody() Antony Hosking | Associate Professor | Computer Science | Purdue University 305 N. University Street | West Lafayette | IN 47907 | USA Office +1 765 494 6001 | Mobile +1 765 427 5484 On Apr 20, 2011, at 11:20 PM, Mika Nystrom wrote: > Aha, I will try that if it happens again. I wrapped more things that I think > might be iffy in my program. > > But I have a feeling ThreadPosix__DumpEverybody won't work... the > application is probably deadlocked by having a single thread locking > against itself with some sort of pthread lock. I should look at the > source code of free. > > Mika > > Tony Hosking writes: >> Can you dump all the user threads at this point? = >> ThreadPosix__DumpEverybody() should do it. We want to see where the = >> other threads might be. >> >> On Apr 20, 2011, at 10:53 PM, Mika Nystrom wrote: >> >>> Hi again, >>> =20 >>> So, as I described before, I wrote a program that wraps all of = >> Critical >>> Mass's "postgresql" library (PQ.i3) with Scheduler.DisableSwitching / >>> Scheduler.EnableSwitching. Yes, it seems to have reduced the = >> frequency >>> at which my application locks up, but it has not eliminated the = >> lockups. >>> =20 >>> User threads, AMD64_LINUX. >>> =20 >>> I'm not quite sure what to try next. How can I find out where malloc >>> and free might be used reentrantly, assuming that is the problem? >>> =20 >>> There may not be any reliable threading in CM3 anymore. Possibly it's >>> reliable on FreeBSD (without -pthread) but I wouldn't bet money on it. >>> =20 >>> Mika >>> =20 >>> (gdb) where >>> #0 0x00007f7a8da0102e in ?? () from /lib/libc.so.6 >>> #1 0x00007f7a8d99eb43 in ?? () from /lib/libc.so.6 >>> #2 0x00007f7a8d99baab in free () from /lib/libc.so.6 >>> #3 0x000000000077b08d in PQsendQuery () >>> #4 0x0000000000767c12 in PQSchedulerWrap__PQsendQuery (conn=3DCannot = >> access memory at address 0x7f7b848a4298 >>> ) >>> at ../AMD64_LINUX/PQSchedulerWrap.m3:210 >>> #5 0x0000000000763ce0 in UnsafeDatabase__MyExec (t=3DCannot access = >> memory at address 0x7f7b848a4328 >>> ) at ../src/UnsafeDatabase.m3:92 >>> #6 0x0000000000764a29 in UnsafeDatabase__ExecM (t=3DCannot access = >> memory at address 0x7f7b848a43f8 >>> ) at ../src/UnsafeDatabase.m3:239 >>> #7 0x0000000000764cee in UnsafeDatabase__TExecM (t=3DCannot access = >> memory at address 0x7f7b848a4528 >>> ) at ../src/UnsafeDatabase.m3:269 >>> #8 0x0000000000756638 in DesynchronizedDB__TransSExec (trans=3DCannot = >> access memory at address 0x7f7b848a45e8 >>> ) at ../src/DesynchronizedDB.m3:74 >>> #9 0x00000000005aeac6 in = >> DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 = >> (finalAttempt=3DCannot access memory at address 0x7f7b848a463f >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 >>> #10 0x00000000005ae639 in DBTable_gcoms_ordr_statusMonitor__ScanDirty = >> (cl=3DCannot access memory at address 0x7f7b848a4778 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 >>> #11 0x00000000005b171e in = >> DBTable_gcoms_ordr_statusMonitor__CheckForNew (cl=3DCannot access memory = >> at address 0x7f7b848a4958 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 >>> #12 0x00000000005b1382 in DBTable_gcoms_ordr_statusMonitor__Sync = >> (cl=3DCannot access memory at address 0x7f7b848a4ba8 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 >>> #13 0x00000000005b0e1c in DBTable_gcoms_ordr_statusMonitor__ApplyC = >> (cl=3DCannot access memory at address 0x7f7b848a4d58 >>> ) >>> at ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 =3D> = >> /home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 >>> #14 0x0000000000922eb5 in ThreadPosix__RunThread () at = >> ../src/thread/POSIX/ThreadPosix.m3:1174 >>> #15 0x00007f7a8d9697b0 in ?? () from /lib/libc.so.6 >>> #16 0x0000000000000000 in ?? () >>> (gdb)=20 From hendrik at topoi.pooq.com Thu Apr 21 15:19:04 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 09:19:04 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> Message-ID: <20110421131904.GB22222@topoi.pooq.com> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > Agreed. But CVSup used it and we were trying to be supportive. So, at present, CVSup presumably works with user threads. Maybe the cure is to fix CVSup instead of fork(). Why does CVSup needs a fork() that preserves all threads? Is it essential to its design, or incidental? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:20:54 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:20:54 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421131904.GB22222@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> Message-ID: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >> Agreed. But CVSup used it and we were trying to be supportive. > > So, at present, CVSup presumably works with user threads. > Maybe the cure is to fix CVSup instead of fork(). > > Why does CVSup needs a fork() that preserves all threads? Is > it essential to its design, or incidental? CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > -- hendrik From hendrik at topoi.pooq.com Thu Apr 21 16:28:26 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:28:26 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> Message-ID: <20110421142826.GA23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: > > On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: > > > On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: > >> Agreed. But CVSup used it and we were trying to be supportive. > > > > So, at present, CVSup presumably works with user threads. > > Maybe the cure is to fix CVSup instead of fork(). > > > > Why does CVSup needs a fork() that preserves all threads? Is > > it essential to its design, or incidental? > > CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. Does it need to use fork() at all? Wuld it suffice to use Modula 3's own thread system? -- hendrik From hosking at cs.purdue.edu Thu Apr 21 16:30:25 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 10:30:25 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421142826.GA23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: I'm not familiar with the architecture of CVSup. On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >> >> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >> >>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>> Agreed. But CVSup used it and we were trying to be supportive. >>> >>> So, at present, CVSup presumably works with user threads. >>> Maybe the cure is to fix CVSup instead of fork(). >>> >>> Why does CVSup needs a fork() that preserves all threads? Is >>> it essential to its design, or incidental? >> >> CVSup uses fork in an ill-defined way. fork is only well-defined for use as fork+exec. So, the fact that there are continuing threads is immaterial. CVSup wants to continue doing real work (without exec) after fork. > > Does it need to use fork() at all? Wuld it suffice to use Modula 3's > own thread system? > > -- hendrik From wagner at elegosoft.com Thu Apr 21 16:47:33 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 16:47:33 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> Message-ID: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Quoting Tony Hosking : > I'm not familiar with the architecture of CVSup. > > On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: > >> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>> >>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>> >>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>> >>>> So, at present, CVSup presumably works with user threads. >>>> Maybe the cure is to fix CVSup instead of fork(). >>>> >>>> Why does CVSup needs a fork() that preserves all threads? Is >>>> it essential to its design, or incidental? >>> >>> CVSup uses fork in an ill-defined way. fork is only well-defined >>> for use as fork+exec. So, the fact that there are continuing >>> threads is immaterial. CVSup wants to continue doing real work >>> (without exec) after fork. >> >> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >> own thread system? Not for real word server applications, where one problem in a user session would crash dozens of other sessions. At least this was what John Polstra reported. The additional safety of separate address spaces _has_ its advantages. I thought CVSup had been changed to work with pthreads though in CM3? Or do I misremember? Anyway, this shouldn't be our most important problem, but rather make System pthreads and also M3 user threads work in a reliable and fault-free way again. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From hendrik at topoi.pooq.com Thu Apr 21 16:54:27 2011 From: hendrik at topoi.pooq.com (Hendrik Boom) Date: Thu, 21 Apr 2011 10:54:27 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> Message-ID: <20110421145427.GC23127@topoi.pooq.com> On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: > Quoting Tony Hosking : > >> I'm not familiar with the architecture of CVSup. >> >> On Apr 21, 2011, at 10:28 AM, Hendrik Boom wrote: >> >>> On Thu, Apr 21, 2011 at 10:20:54AM -0400, Tony Hosking wrote: >>>> >>>> On Apr 21, 2011, at 9:19 AM, Hendrik Boom wrote: >>>> >>>>> On Wed, Apr 20, 2011 at 10:58:06AM -0400, Tony Hosking wrote: >>>>>> Agreed. But CVSup used it and we were trying to be supportive. >>>>> >>>>> So, at present, CVSup presumably works with user threads. >>>>> Maybe the cure is to fix CVSup instead of fork(). >>>>> >>>>> Why does CVSup needs a fork() that preserves all threads? Is >>>>> it essential to its design, or incidental? >>>> >>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>> for use as fork+exec. So, the fact that there are continuing >>>> threads is immaterial. CVSup wants to continue doing real work >>>> (without exec) after fork. >>> >>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>> own thread system? > > Not for real word server applications, where one problem in a user > session would crash dozens of other sessions. At least this was what > John Polstra reported. The additional safety of separate address > spaces _has_ its advantages. > > I thought CVSup had been changed to work with pthreads though in CM3? > Or do I misremember? > > Anyway, this shouldn't be our most important problem, but rather make > System pthreads and also M3 user threads work in a reliable and > fault-free way again. Once our only problems are with CVSup's abuse of fork(), I'd consider that done. -- hendrik From mika at async.caltech.edu Thu Apr 21 17:07:48 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Thu, 21 Apr 2011 08:07:48 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421150748.62E2B1A2078@async.async.caltech.edu> Hendrik Boom writes: ... >>>>> >>>>> CVSup uses fork in an ill-defined way. fork is only well-defined >>>>> for use as fork+exec. So, the fact that there are continuing >>>>> threads is immaterial. CVSup wants to continue doing real work >>>>> (without exec) after fork. >>>> >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real word server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > >Once our only problems are with CVSup's abuse of fork(), I'd >consider that done. Unfortunately, no. CVSup's ab/use of fork() only affects some software-engineering issues at the margin of the threading problem. It does do bad things to portability to have to support it. The Modula-3 way of doing what CVSup does is to use Process.Create to create a new process. If there is state in the current process that needs to be transmitted to the new process, Pickle the state and send it over a Pipe into the new process. Using fork() to do a kind of pseudo-shared-memory should not be necessary. I think. Mika From wagner at elegosoft.com Thu Apr 21 17:14:24 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 21 Apr 2011 17:14:24 +0200 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421145427.GC23127@topoi.pooq.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> Message-ID: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Quoting Hendrik Boom : > On Thu, Apr 21, 2011 at 04:47:33PM +0200, Olaf Wagner wrote: [...] >>>> Does it need to use fork() at all? Wuld it suffice to use Modula 3's >>>> own thread system? >> >> Not for real world server applications, where one problem in a user >> session would crash dozens of other sessions. At least this was what >> John Polstra reported. The additional safety of separate address >> spaces _has_ its advantages. >> >> I thought CVSup had been changed to work with pthreads though in CM3? >> Or do I misremember? >> >> Anyway, this shouldn't be our most important problem, but rather make >> System pthreads and also M3 user threads work in a reliable and >> fault-free way again. > > Once our only problems are with CVSup's abuse of fork(), I'd > consider that done. I think Mika has pointed out quite a few problems that haven't been solved yet (or I missed something, which may well be); he even contributed a sophisticated test program for validation of threads. If it hasn't been done yet, CVSup can probably easily be adapted to other thread-survival-during-fork patterns; we just should be able to provide a consistent specification and implementation for all platforms, for both threads and process semantics. However, I'll leave the office now for Easter holidays, and wish you all sunny and restful Easter days! Olaf PS: I'm still surprised that none of those recent problems has found its way into our fault tracker... -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 21 18:27:45 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 21 Apr 2011 17:27:45 +0100 (BST) Subject: [M3devel] wrapping postgresql library not enough In-Reply-To: <20110421151025.A8AC31A2078@async.async.caltech.edu> Message-ID: <980045.69138.qm@web29713.mail.ird.yahoo.com> Hi all: sure I can actually it work ?s with WeakRef, somehow interesting, is there a m3-db/m3postgresql or ? Besides the compilation problems, zeus3D package misses sources of at least 4 interface files and its not public dependences: http://www.cs.arizona.edu/~collberg/Research/Modula-3/m3sources/html/zeus3D/src/ Aux, Camera, G, Light perhaps they were exported as hide_Interface() with its own modules if so. They are a kind of helper or friend modules ti it, so they are missing in public three there. There is also a local copy of Obliq and Modula-3 packages of modules of models of lego tools, so it might be interesting for somebody to play with them, it's fairly a good size library written by an old M3 friend :), as are the algorithm animations done for zeus3D at DEC SRC. There is another one built package to animate distributed protocols called Zada, it might be interesting to plug it all, even there is a formal specification of the same problem, a Manufacturing Cell, local projects name was Didactic Manufacturing Cell thesis (all sources available for free just that Obliq is not working as far I can tell) Zada related relevant links (missing main web page is not under development any more, maybe we should ask them what they have about it and perhaps even their thesis) they had even audio: http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-agarwal.html http://ls4-www.cs.tu-dortmund.de/RVS/Exports/EB_PG225.ps.Z http://ls4-www.cs.tu-dortmund.de/RVS/V-DA/daa-kukasch.html So its would a good package animation example of lego tools, specifications, and protocols of all the framework, don't you think? Thanks in advance PD: just if you care of and have at 2 PM ET today there is a conference on embedded software verification, so it might be good to see what this guys are looking for, maybe we can give them a look of it? http://www.eetimes.com/design/eda-design/4215099/Using-verification-coverage-with-formal-analysis --- El jue, 21/4/11, Mika Nystrom escribi?: > De: Mika Nystrom > Asunto: Re: [M3devel] wrapping postgresql library not enough > Para: "Daniel Alejandro Benavides D." > Fecha: jueves, 21 de abril, 2011 10:10 > Hi Daniel, > > Sure I'd be happy to look at it (although I *may* already > have fixed my > problems, I don't know). Would it be OK to add the > PSQL driver you are talking > about to CM3? > > Can you give me a filename of some file you think would be > in zeus3d and I can > go looking in all my files for it? > > Mika > > "Daniel Alejandro Benavides D." writes: > >Hi Mika: > >I have a local copy of Daniel Solaz's 2001 PSQL driver, > if needed, just in = > >case you want it I can send right away to you. > >Besides: Did you know or know how to find out zeus3d > sources besides the pu= > >blic package? > > > >Thanks in advance > > > >--- El mi=E9, 20/4/11, Mika Nystrom > escribi=F3: > > > >> De: Mika Nystrom > >> Asunto: Re: [M3devel] wrapping postgresql library > not enough > >> Para: "Tony Hosking" > >> CC: m3devel at elegosoft.com > >> Fecha: mi=E9rcoles, 20 de abril, 2011 22:20 > >> Aha, I will try that if it happens > >> again. I wrapped more things that I think > >> might be iffy in my program. > >>=20 > >> But I have a feeling ThreadPosix__DumpEverybody > won't > >> work... the > >> application is probably deadlocked by having a > single > >> thread locking > >> against itself with some sort of pthread > lock. I > >> should look at the > >> source code of free. > >>=20 > >> Mika > >>=20 > >> Tony Hosking writes: > >> >Can you dump all the user threads at this > point?=20 > >> =3D > >> >ThreadPosix__DumpEverybody() should do > it. We > >> want to see where the =3D > >> >other threads might be. > >> > > >> >On Apr 20, 2011, at 10:53 PM, Mika Nystrom > wrote: > >> > > >> >> Hi again, > >> >>=3D20 > >> >> So, as I described before, I wrote a > program that > >> wraps all of =3D > >> >Critical > >> >> Mass's "postgresql" library (PQ.i3) with > >> Scheduler.DisableSwitching / > >> >> Scheduler.EnableSwitching. Yes, it > seems to > >> have reduced the =3D > >> >frequency > >> >> at which my application locks up, but it > has not > >> eliminated the =3D > >> >lockups. > >> >>=3D20 > >> >> User threads, AMD64_LINUX. > >> >>=3D20 > >> >> I'm not quite sure what to try > next. How can > >> I find out where malloc > >> >> and free might be used reentrantly, > assuming that > >> is the problem? > >> >>=3D20 > >> >> There may not be any reliable threading > in CM3 > >> anymore. Possibly it's > >> >> reliable on FreeBSD (without -pthread) > but I > >> wouldn't bet money on it. > >> >>=3D20 > >> >> Mika > >> >>=3D20 > >> >> (gdb) where > >> >> #0 0x00007f7a8da0102e in ?? () > from > >> /lib/libc.so.6 > >> >> #1 0x00007f7a8d99eb43 in ?? () > from > >> /lib/libc.so.6 > >> >> #2 0x00007f7a8d99baab in free () > from > >> /lib/libc.so.6 > >> >> #3 0x000000000077b08d in > PQsendQuery () > >> >> #4 0x0000000000767c12 in > >> PQSchedulerWrap__PQsendQuery (conn=3D3DCannot =3D > >> >access memory at address 0x7f7b848a4298 > >> >> ) > >> >> at > >> ../AMD64_LINUX/PQSchedulerWrap.m3:210 > >> >> #5 0x0000000000763ce0 in > >> UnsafeDatabase__MyExec (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4328 > >> >> ) at ../src/UnsafeDatabase.m3:92 > >> >> #6 0x0000000000764a29 in > >> UnsafeDatabase__ExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a43f8 > >> >> ) at ../src/UnsafeDatabase.m3:239 > >> >> #7 0x0000000000764cee in > >> UnsafeDatabase__TExecM (t=3D3DCannot access =3D > >> >memory at address 0x7f7b848a4528 > >> >> ) at ../src/UnsafeDatabase.m3:269 > >> >> #8 0x0000000000756638 in > >> DesynchronizedDB__TransSExec (trans=3D3DCannot > =3D > >> >access memory at address 0x7f7b848a45e8 > >> >> ) at ../src/DesynchronizedDB.m3:74 > >> >> #9 0x00000000005aeac6 in =3D > >> > >DBTable_gcoms_ordr_statusMonitor__ScanDirty__Attempt.1014 > >> =3D > >> >(finalAttempt=3D3DCannot access memory at > address > >> 0x7f7b848a463f > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:41 > >> >> #10 0x00000000005ae639 in > >> DBTable_gcoms_ordr_statusMonitor__ScanDirty =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4778 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:73 > >> >> #11 0x00000000005b171e in =3D > >> >DBTable_gcoms_ordr_statusMonitor__CheckForNew > >> (cl=3D3DCannot access memory =3D > >> >at address 0x7f7b848a4958 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:469 > >> >> #12 0x00000000005b1382 in > >> DBTable_gcoms_ordr_statusMonitor__Sync =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4ba8 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:445 > >> >> #13 0x00000000005b0e1c in > >> DBTable_gcoms_ordr_statusMonitor__ApplyC =3D > >> >(cl=3D3DCannot access memory at address > 0x7f7b848a4d58 > >> >> ) > >> >> at > >> ../AMD64_LINUX/DBTable_gcoms_ordr_statusMonitor.m3 > =3D3D> > >> =3D > >> > >/home/mika/t/calarm/ratsql/src/TableMonitor.mg:393 > >> >> #14 0x0000000000922eb5 in > ThreadPosix__RunThread > >> () at =3D > >> >../src/thread/POSIX/ThreadPosix.m3:1174 > >> >> #15 0x00007f7a8d9697b0 in ?? () from > >> /lib/libc.so.6 > >> >> #16 0x0000000000000000 in ?? () > >> >> (gdb)=3D20 > >> > From rcolebur at SCIRES.COM Thu Apr 21 19:48:57 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 13:48:57 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , <20110420055557.089101A207A@async.async.caltech.edu>, <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I concur with Mika ! We need to stick to the design tennants of Modula-3. Regards, Randy Coleburn -----Original Message----- From: Mika Nystrom [mailto:mika at async.caltech.edu] Sent: Wednesday, April 20, 2011 10:03 PM To: Jay K Cc: m3devel at elegosoft.com Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay K writes: >--_413fd7ab-4318-4696-9889-2407f5881625_ >Content-Type: text/plain; charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > > >> > soon afterwards in the child process=2C thus resetting all states. In t= >he >> > meantime=2C only a short list of async-signal-safe library routines are >> > promised to be available." > > >Any idea why? I don't know. > > >Is this such a burden? >We need a global lock order for it. Unreasonable? You mean that the application needs a global (partial) lock order for it? Somehow an application that wants to use facilities to fork-and-do-more-work has to call pthread_atfork with the order established, or have I misunderstood? That means every library that has locks has to register them somewhere? >I understand pthread_atfork is causing a problem with user threads=2C but > - It really ought not. The libc vs. libpthread design is broken. I believ= >e many systems > don't have this broken design=2C but I guess some do. The broken design= > is that > libc and libpthread implement some functions with the same name=2C and = >you get > to chose which one. And some of them might be thread-safe. > > > > In Modula-3=2C every application has *always* > > had to assume that there are other threads floating around. There is > > >This is the case in C too. >I can't assume any library I link to doesn't create some helper threads wit= >h pthreads. >pthreads should be assumed always in use=2C and there should be no ramifica= >tions of this. >I shouldn't have a choice e.g. of a malloc/free/strtok that aren't thread s= >afe vs. ones that are. Well there are versions of C that don't define threads at all (especially historical versions of C). A programmer using such a version might legitimately assume there are no threads except the main program. My comments about not supporting fork-and-do-more-work are based on the simple observation that the normal Modula-3 libraries provide a particular set of operations for thread control and for process creation, none of which allow fork-and-do-more-work. Which is why I say it is probably legitimate to assume that people who want to fork-and-do-more-work are on their own. I also think one of the valuable aspects of Modula-3 is that it defines a complete systems programming environment already with the facilities in Thread and Process, and that adding support for more facilities in m3core starts blurring the line between what is and is not Modula-3. This could turn out to be a problem if Modula-3 is in future again ported to systems without complete, modern C environments. By "problem" I mean an extra bifurcation of application programs: some programs for "POSIX Modula-3" and some for "other Modula-3". It was definitely a design goal of the language to provide interfaces that were simple and general enough that application code would be "write once, run anywhere"---and to do it in a smarter way than Java's sledgehammer approach of making everything pass through a virtual machine. In fact I remember it was a point of pride with SRC M3 that no conditional compilation was needed to account for OS and architecture differences even though the SRC distribution had been ported to a rather large variety of computers. Mika From jay.krell at cornell.edu Thu Apr 21 19:47:01 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 17:47:01 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421150748.62E2B1A2078@async.async.caltech.edu> References: , , <20110420055557.089101A207A@async.async.caltech.edu>, , <20110421131904.GB22222@topoi.pooq.com>, <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu>, <20110421142826.GA23127@topoi.pooq.com>, , <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com>, <20110421145427.GC23127@topoi.pooq.com>, <20110421150748.62E2B1A2078@async.async.caltech.edu> Message-ID: > Unfortunately, no. CVSup's ab/use of fork() only affects some > software-engineering issues at the margin of the threading problem. > It does do bad things to portability to have to support it. > > The Modula-3 way of doing what CVSup does is to use Process.Create to > create a new process. If there is state in the current process that > needs to be transmitted to the new process, Pickle the state and send > it over a Pipe into the new process. Using fork() to do a kind of > pseudo-shared-memory should not be necessary. I think. What cvsup does is more efficient. But I'm not sure if by much. You get a bunch of read-only shared state, including sharing physical memory. Most other forms of inter-process-communication will duplicate the data. What cvsup does is not portable to Windows. And is not efficient on Cygwin. Though it is portable to Interix and efficient there. The NT kernel supports fork() in the manner everyone is familiar with, but Win32 doesn't expose it. Anyway, we still don't know what are the problems currently, and they might exist on Win32 also. Or at least there are problems on Win64. The test program needs to be tried on 32bit Windows. And we need to consider removing use of GetThreadContext, and probably SuspendThread/ResumeThread as well. I understand the -pthread dependency might be undesirable in user threads and it should be easy to remove it, with the tradeoff that direct fork() won't do the right thing (though it will be racy and therefore often appear to work). - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Thu Apr 21 19:35:45 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 21 Apr 2011 13:35:45 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> References: <20110420055557.089101A207A@async.async.caltech.edu> <20110421131904.GB22222@topoi.pooq.com> <7043AAA1-FF3E-42D7-8271-E787260A55FC@cs.purdue.edu> <20110421142826.GA23127@topoi.pooq.com> <20110421164733.6qfakbls4ysggk0s@mx0.elegosoft.com> <20110421145427.GC23127@topoi.pooq.com> <20110421171424.ort6tlxeok4skw8k@mx0.elegosoft.com> Message-ID: fork + exec works fine and has always worked fine. fork + carry on is what Jay fixed with pthread_atfork. On Apr 21, 2011, at 11:14 AM, Olaf Wagner wrote: > If it hasn't been done yet, CVSup can probably easily be adapted to > other thread-survival-during-fork patterns; we just should be able > to provide a consistent specification and implementation for all > platforms, for both threads and process semantics. From jay.krell at cornell.edu Thu Apr 21 20:42:21 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 18:42:21 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Thu Apr 21 21:21:20 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 19:21:20 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: <20110421020247.31A0A1A2078@async.async.caltech.edu> References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu> Message-ID: I forgot to reply to this point, which keep coming up: > starts blurring the line between what is and is not Modula-3. This could > turn out to be a problem if Modula-3 is in future again ported to systems > without complete, modern C environments. By "problem" I mean an extra > bifurcation of application programs: some programs for "POSIX Modula-3" Cvsup won't work in such a system without a certain amount of work. Such a system requires a fairly significant amount of work. Nothing discussed here really touches on the work required. Such a system needs threads implemented in *some* fashion. Either pthreads, or sigaltstack, or jmpbuf smashing, or somehow natively in Modula-3, but ultimately, you need assembly and/or processor-specificness *somewhere*. You must save restore registers somewhere. You must setup a stack somewhere. You need preemption somehow (well, maybe not -- "cooperative suspend"). And you need to provide for file I/O somewhere, depending on what sort of program you want to run. And socket I/O, ditto. And keyboard/video/mouse/console functionality, ditto. You can use more or less of the facilities of the underlying system, giving you more or less functionality for free/cheap, obligating you to implement more or less of the functionality you require or desire. Modula-3 layers on top of Posix and Win32, with more or less thickness and state. Modula-3 was kind of interesting in that it provided threads with a certain amount of portability without requiring an underlying threading capability, with just timers for preemption and setjmp/longjmp for context save/restore. This is still somewhat interesting, e.g. maybe for DJGPP, but pthreads and Win32 threads are widespread now, and providing these features isn't particularly valuable or differentiating. You know -- why use Modula-3? One of the reasons in the past was maybe for portable threading. That is still a reason, but portable threading is not as difficult to write for one's self these days, merely by layering over pthreads and Win32. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 00:23:41 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , ,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , ,,<20110407190219.C00E31A2078@async.async.caltech.edu>, , ,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , ,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, , ,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , ,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , ,,<1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, ,,, ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, Message-ID: Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 01:15:04 2011 From: jay.krell at cornell.edu (Jay K) Date: Thu, 21 Apr 2011 23:15:04 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 01:28:59 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 00:28:59 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: Message-ID: <284553.92617.qm@web29701.mail.ird.yahoo.com> Hi all: one way of being us safe and capable of doing more than traditional modification of sources each time the system changes (which is basically almost everyone blindly agrees, sorry if I'm mad, but who else aside of us cares about this?) and avoid this dangerous path to near disaster of Modula-3 specifications is to selfhost in our threads. Modula-3 has been almost hard to break system, and even if Modula-3 is modified must remain there, no UNSAFE stuff imploded, no every one in its place, no dangerous threading with locks every 10, 20 30 milliseconds, no more, we are here to say again guys just make things clear and quick and simple, don't make us to change our implementations each time you are making its. I mean Linux KVM, Solaris Zones, Win HV, it isn't that hard to believe it, don't you think? Is about what guys there did with SPIN, I'm going back to that, but is useful for one reason, the only way to keep function stability and I mean that for us in some sense could be that, remember we are able to test stress a VM in such an environment as most as we would want, knowing every fault there is ones besides HyperThreading is making its presence in hardware assisted virtualization? then is a matter of what we want to achieve the next level is up to us. That said originally the DEC Unix already had emulation of threads, of C POSIX threads? (now called pthreads today), of Mach 3.0 layered ones and DEC OSF too and besides and own strends for more than the most of any system can think of. See: http://qstream.org/~krasic/cs508-2006/summaries/paper17/SpinOS.ppt Thanks in advance --- El jue, 21/4/11, Coleburn, Randy escribi?: De: Coleburn, Randy Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Para: "m3devel" Fecha: jueves, 21 de abril, 2011 17:23 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. ?Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.?? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages.? I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible.? ?I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?.? The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika.? It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform.? And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only.? Everything else is completely portable. ?Regards,Randy Coleburn ?From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 ?> I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcolebur at SCIRES.COM Fri Apr 22 05:24:02 2011 From: rcolebur at SCIRES.COM (Coleburn, Randy) Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, ,,,,<1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, ,,,,<20110407190219.C00E31A2078@async.async.caltech.edu>, ,,,,<59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, ,,,,<20110418164239.84CEB1A207A@async.async.caltech.edu>, ,,,,<0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, ,,,,<20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , , , , , , <20110419023058.4FBB11A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110420055557.089101A207A@async.async.caltech.edu>, , , , , , , <20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , Message-ID: Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I'm not advocating we "throw the baby out with the bath water", but if the "fork-and-do-more" pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay ________________________________ From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Regarding CVSup, I am probably not the one to ask how important it is because I have never used it. But, I say don't break everything else just to make CVSup work. I want the tenants of the "Green Book" to hold true for CM3 across all platforms. It would seem to me that CVSup needs to change; it is the "tail wagging the dog" at this point. Let's make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the "Green Book." I'm not saying we shouldn't be open to new ideas or discussion, but I for one don't want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a "systems language". The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes-the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards, Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Fri Apr 22 08:29:08 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 06:29:08 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 In-Reply-To: References: <818928D5-29FA-433E-9AC1-F57427CEC94F@m3w.org>, , , , , , <1B26BB93-EB7E-4F78-8187-14E766DDCA33@cs.purdue.edu>, , , , , , <20110407190219.C00E31A2078@async.async.caltech.edu>, , , , , , <59D8BCCC-12B5-4674-895C-08E6475A5020@cs.purdue.edu>, , , , , , <20110418164239.84CEB1A207A@async.async.caltech.edu>, , , , , , <0308C79B-BCCD-473A-B3AE-FD1E30DA9321@cs.purdue.edu>, , , , , , <20110418184540.C2D801A2078@async.async.caltech.edu>, , , , , , <1F0296C0-EC9F-42D2-BE8E-EFD221BE1B81@cs.purdue.edu>, , , ,,, , , ,,<20110419023058.4FBB11A207A@async.async.caltech.edu>, , ,,, , , , , , , , , <20110420055557.089101A207A@async.async.caltech.edu>, , , , , ,,, ,,<20110421020247.31A0A1A2078@async.async.caltech.edu>, , , , , , , , , Message-ID: > As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Exchanging Modula-3 for C has tremendously improved the maintainability, portability, correctness, size-in-source, and correctness-by-inspection of the system. Debuggability has improved as well, though hopefully that will "even out" -- hopefully Modula-3 code will become just as debuggable with stock gdb, and on systems without stabs support. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 23:24:02 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 Jay: Re CVSup, I agree that we need to get to the bottom of things as soon as possible. I?m not advocating we ?throw the baby out with the bath water?, but if the ?fork-and-do-more? pattern used by CVSup is causing problems, I vote we remove it until those problems can be resolved. Better to break one app, than a bunch. As for the use of C vs. Modula-3, it obvious that you and I have a difference of opinion. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 7:15 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 1) There is little evidence that supporting cvsup has been a bad thing. Maybe it is breaking user threads on poorly designed systems, and maybe that includes Linux, but it can be fixed while still supporting cvsup. The problems remain to be understood. cvsup and fork-and-do-more-work are very possibly just a red herring that everyone is rat-holing on. We really need to know what is wrong before jumping to conclusions about cvsup and fork-and-do-more-work. Very little has been done or said by anyone (including me!) except for speculation. IF IF IF we find cvsup and supporting fork-and-do-more-work are really a problem, then we should consider breaking them. But first we must understand what is going wrong. 2) We absolutely should use C, at least in certain places. Not using C caused major maintenance headaches and bugs in the system. > the more library stuff we can code in Modula-3 the better, Not necessarily. > as it gives us the platform independence, as expressed by Mika Absolutely not. The opposite is the truth. Coding more stuff, certain stuff, in Modula-3 gave us more platform dependence, not less. Language does not buy portability. Library use does. And someone has to implement the libraries. Layered on top of something. - Jay From: rcolebur at SCIRES.COM To: m3devel at elegosoft.com Date: Thu, 21 Apr 2011 18:23:41 -0400 Subject: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4Regarding CVSup, I am probably not the one to ask how important it is because I have never used it.But, I say don?t break everything else just to make CVSup work.I want the tenants of the ?Green Book? to hold true for CM3 across all platforms.It would seem to me that CVSup needs to change; it is the ?tail wagging the dog? at this point.Let?s make sure threading is fixed to work reliably again on all platforms; then go back and look at adjusting CVSup. Further, I think it safe to say that most of the folks on m3devel have bought into the philosophy and design tenants of Modula-3, as expressed in the ?Green Book.? I?m not saying we shouldn?t be open to new ideas or discussion, but I for one don?t want to spend a lot of time here debating the merits of other languages. I want to spend time here trying to ensure the stability, evolution, and continued availability of Modula-3 on as many platforms as possible. I also believe strongly that all developers should avoid resorting to coding something in C, because Modula-3 is a ?systems language?. The more library stuff we can code in Modula-3 the better, as it gives us the platform independence, as expressed by Mika. It is only when using the UNSAFE subset of the language that you should ever run into a situation where an implementation has to be changed to accommodate a different platform. And, then you know exactly where to go for the changes?the UNSAFE module and use of UNSAFE features only. Everything else is completely portable. Regards,Randy Coleburn From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K Sent: Thursday, April 21, 2011 2:42 PM To: Coleburn, Randy; m3devel Subject: RE: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 > I concur with Mika ! > We need to stick to the design tennants of Modula-3. ok...so..is someone volunteering to fix cvsup? Or are we willing to break it? And does this solve any larger problems? ie: are the changes to support cvsup really what broke anything? ok, yes, for now, they have broken user threads. > You mean that the application needs a global (partial) lock order > for it? Somehow an application that wants to use facilities to > fork-and-do-more-work has to call pthread_atfork with the order > established, or have I misunderstood? That means every library > that has locks has to register them somewhere? Yes. Not register the locks per se, but register callbacks. Those callbacks need to acquire/release locks though. Essentially what you said though. > Well there are versions of C that don't define threads at all (especially > historical versions of C). A programmer using such a version might legitimately > assume there are no threads except the main program. Hardly. Threads have been widespread for a long time now. Pretending they don't exist seems kind of dumb. > people who want to fork-and-do-more-work are > on their own. "on their own" meaning they have an arbitrary unbounded set of problems? We will completely ignore them? Will break cvsup? Is using pthread_atfork really so difficult?? > I also think one of the valuable aspects of Modula-3 is that it defines > a complete systems programming environment already with the facilities in > Thread and Process, and that adding support for more facilities in m3core Modula-3 is several things. It is a language. It is libraries. Maybe other things. The libraries might might be split into "green book" and others. To what extent should we support "others"? Just the fact that m3core exposes Unix.fork() gets us into "others". Anyway..I think there is this idea of Modula-3 being a closed system. Limit yourself to a smaller set of libraries and practises and ok. Go to far into the world of other libraries and things that work in C and can work in Modula-3, and things start breaking. It seems gray to me, what should work, what the overall approach should be. > bifurcation of application programs: some programs for "POSIX Modula-3" > and some for "other Modula-3". I believe this exists. Do we not have X-Windows apps in Modula-3 and Win32 apps in Modula-3? > It was definitely a design goal of the > language to provide interfaces that were simple and general enough that > application code would be "write once, run anywhere"---and to do it in a You can make a big push in this direction, but you'll never get there. People will always want more libraries. Some they will write brand new in Modula-3, good. Many others will already exist in C and C++ and they will want to reuse. > it was a point of pride with SRC M3 that no conditional compilation was needed to account for > OS and architecture differences even though the SRC distribution had > been ported to a rather large variety of computers. They did worse things instead. They had large swaths of duplicated code, per-target, that was almost the same per-target. Sometimes the code was code, sometimes declarations. The declarations were very tedious and error-prone to write and recieved no static checking. They had runtime checks for what the target was -- thus the problem where libm3 was tied to the exact list of targets available. Conditional compilation can be a good thing. - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From mika at async.caltech.edu Fri Apr 22 18:25:36 2011 From: mika at async.caltech.edu (Mika Nystrom) Date: Fri, 22 Apr 2011 09:25:36 -0700 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Message-ID: <20110422162536.E63631A2078@async.async.caltech.edu> Jay K writes: >--_f508cc8d-d8d9-47ec-89e0-fcca258f1fc1_ >Content-Type: text/plain; charset="Windows-1252" >Content-Transfer-Encoding: quoted-printable > > > > As for the use of C vs. Modula-3=2C it obvious that you and I have a dif= >ference of opinion. >Exchanging Modula-3 for C has tremendously improved the maintainability=2C = >portability=2C correctness=2C size-in-source=2C and correctness-by-inspecti= >on of the system. >Debuggability has improved as well=2C though hopefully that will "even out"= > -- hopefully Modula-3 code will become just as debuggable with stock gdb= >=2C and on systems without stabs support. > > - Jay In theory you are right. In practice, almost ten years ago now others at Caltech and I started to try to switch from PM3 to CM3 since it was obvious it was CM3 that was going to be maintained in future. The process still isn't over, as my frequent postings to this list bear witness to. My perspective is really very simple. I've been using Modula-3 as a tool to solve almost all programming problems I've had since about 1999, when I gave up C due to my inability to write a correct C program. I'm a user of the system and only reluctantly a peripheral maintainer. A few of my observations as a user are the following. -- Threads work perfectly in PM3. In CM3 it's very much ??? -- It took five years to get TEXT working acceptably (as in, no stack overflows!) in CM3. Now seems more or less OK, but Text.Hash is still possibly an order of magnitude slower than . -- I have no debugger that works for CM3. m3gdb is *very useful* for the old PM3. No it doesn't work perfectly but you can set breakpoints and print variables but that's what at least I use a debugger for. -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE are very slow (like orders of magnitude slower than for PM3). I looked at it once and realized that someone had walked all over RT0 with very heavy feet. Not a pretty sight, on my (long) to-do list. -- m3tk is broken owing to the introduction of LONGINT. This is not a peripheral issue when a lot of the code you produce is generated from other code (a process that improves programming productivity manyfold---when it works). As with all the other items on this list, m3tk has no trouble parsing the Modula-3 accepted by PM3. -- other LONGINT problems... -- It's taken me literally months of struggle to get my application working with CM3. The old PM3-compiled one traded half a billion dollars worth of stocks last year without a single glitch. The CM3-compiled version has been hanging every day except the last (so hopefully it's OK now...???) I'm literally losing sleep over this because it's running during European business hours. Only reason I got sleep last night is because it's Good Friday. Now I'm not blaming anyone in particular for these problems. In fact I think they mostly come from things that were done at Critical Mass, Inc. However there is a common thread. Just about all the issues I've run into have had *something* to do with someone's attempt to make Modula-3 more of "all things to all people" than it was. Its main strength was and is that it is a programming language with almost equal expressibility (both at the machine and data-structure level) as C++ with a *fifty* page definition in reasonably clear English---that's just a few pages longer than R5RS and in a bigger font to boot. Portability, library support, multicore use, comprehensive base datatypes, ... all those things are nice but not part of the "core mission" of the language. It makes sense to try to expand out and be all things to all people if you have the budget and development team for it. As it is now, all the improvements have done is threaten the survival of the programming language for any use whatsoever. Here's an experiment anyone on this list can try. Find a friend who doesn't use Modula-3 but is a computer programmer, software engineer, etc. Ask him to install CM3 and configure it so that the thread testing program works. See how long it takes before he gives up. Mika From jay.krell at cornell.edu Fri Apr 22 21:25:42 2011 From: jay.krell at cornell.edu (Jay K) Date: Fri, 22 Apr 2011 19:25:42 +0000 Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: <20110422162536.E63631A2078@async.async.caltech.edu> References: <20110422162536.E63631A2078@async.async.caltech.edu> Message-ID: Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Fri Apr 22 23:02:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Fri, 22 Apr 2011 22:02:28 +0100 (BST) Subject: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." In-Reply-To: Message-ID: <927697.40402.qm@web29714.mail.ird.yahoo.com> Hi all: if the problems were the thread package then we will be easyly solved for us, but instead the question how much we owe to each platforms depednence part, e.g: Cvsup was created with Pm3 and/or Ezm3, Pm3 with Linux almost perfect coverage of all source of incompatibilities and so, DEC SRC was make it freely available in any site without too much effort, though by doing it In Modula-3 made almost hard to erase fame of hard thing to do. Then Digital split ended with this efforts going on and meanwhile DEC SRC was being distributed as source compatible with pm3, or with cm3 even. Later Cm3 became an open product and everyone kept their objectives so don't think is easy to do that. Instead, there should be an easy way I Think to recover to be in that point. I mean: CM3 -> DEC SRC, DEC SRC -> PM3, and PM3 -> EZM3 -> Cvsup & Spin. I think there should be enough number of similar or almost equal modules same, so most of the time is relinking stuff. I think every one hast its own rights to believe the things they believe. I think what my old M3 Professor told me once before retired, Modula-3 didn't gain more users because the lack of an IDE, today there is not reason to say that, instead the market has grown a lot times faster then why should I believe it doesn't hold true (why we don't have Cm3 running a Cvsup server, why Cm3 is not running a Spin community, why Cm3 is not hosting several projects in web platforms, why Cm3 is not running Obliq3D stuff, and I could say more but but we don't want to be there I guess this is enough too much or little I don't know, we are not there still, Mika's projects, DEC SRC missing packages, zeus3D, etc). Let me know your thoughts and thanks in advance --- El vie, 22/4/11, Jay K escribi?: De: Jay K Asunto: Re: [M3devel] 5.8.6 LINUXLIBC6 breakage, kernel 2.6.23, glibc-2.6-4 message dated "Fri, 22 Apr 2011 06:29:08 -0000." Para: "Mika Nystrom" CC: "m3devel" Fecha: viernes, 22 de abril, 2011 14:25 Most of this is is not related to using C. Details below. > -- Threads work perfectly in PM3. In CM3 it's very much ??? PM3 only uses user threads, right? And the only problem with user threads on CM3 currently is that it depends on pthread_atfork recently and hopefully briefly, and libc/libpthread is broken on some systems and we have to workaround. > -- It took five years to get TEXT working acceptably (as in, no stack > overflows!) in CM3. Now seems more or less OK, but Text.Hash is > still possibly an order of magnitude slower than . Unrelated. I strongly suspect the problem here is in the library design. It is ok to have a readonly string type, but programmers should also have access to a writable/growable string/buffer type. See what C# and Java do. I suspect TEXTs problems stem from trying to make TEXT be both types in one. i.e. handling a lot of concatenation efficiently. > -- I have no debugger that works for CM3. m3gdb is *very useful* for > the old PM3. No it doesn't work perfectly but you can set breakpoints > and print variables but that's what at least I use a debugger for. CM3 m3gdb should be the same as it ever was. I've started work on making stock gdb worthwhile but it is mostly/all disabled currently, as exposing correct typeinfo to the backend has a tendency to break things. > -- CM3 has some weird performance issues; namely ISTYPE and TYPECASE > are very slow Unrelated. And maybe we should look into it. > -- m3tk is broken owing to the introduction of LONGINT. This is not a Unrelated. > -- other LONGINT problems... Unrelated. > In fact I think they mostly come from things that were done at Critical Mass, Inc. Yes, definitely at least some. > have had *something* to do with someone's attempt to make Modula-3 more > of "all things to all people" than it was. We should be able to do somewhat better overall.. PM3 also doesn't support pthreads or Win32 or work on any modern Linux system, right? ?- Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:14:49 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:14:49 +0000 Subject: [M3devel] pthread_atfork In-Reply-To: References: Message-ID: This is in now. Mika, can you try user threads again, and this time remove -pthread/-pthreads/-lpthread from your config files? Really, if user threads are to be supported, I wish they were easier to use -- no need to rebuild m3core or anything else, and specifiable in link or runtime commads. More details below: From: jay.krell at cornell.edu To: m3devel at elegosoft.com Subject: pthread_atfork Date: Tue, 19 Apr 2011 06:07:01 +0000 I propose that RTProcess__RegisterForkHandlers be implemented at least 3 times: ... manually for userthreads: ... take a lock, maintain the three lists/growable arrays, whatever probably implement it in Modula-3 [Jay April 23] done: implemented in C, locking via Disable/EnableSwitching, ok? And then, introduce RTProcess__Fork. Replace all calls to Unix.fork with it. Also implemented around 3 times. .. RTProcessWin32__Fork: assert(false) fork+exec must be implemented through higher level operations fork+do more work isn't possible [Jay April 23] instead, just not provided This doesn't help us figure out what is wrong with our pthreads implementation. It just makes user threads not require -pthread. [Jay April 23] still a major concern in need of significant investigation - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.krell at cornell.edu Sat Apr 23 12:19:25 2011 From: jay.krell at cornell.edu (Jay K) Date: Sat, 23 Apr 2011 10:19:25 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? Message-ID: I've asked before. I'm asking again. I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. @M3pthreads @M3userthreads or somesuch. I'd also like to be able to switch the default via a line in a m3makefile that builds a program. I know roughly what implementing this would look like. I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. I'll have to reread. Whatever problem it has, this change probably would too. I'm willing to implement this myself. I'm not asking for anyone else to do it. I understand it makes LOCK harder to efficiently inline. If/when LOCK becomes partly inlined, provide a switch? Thank you, - Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From hosking at cs.purdue.edu Sat Apr 23 21:04:16 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:04:16 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: <13CE2151-2CF8-4773-B32F-91E753256DAA@cs.purdue.edu> The problem is that you are forced to link with pthreads for the whole executable which may cause problems. Not sure why though. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From hosking at cs.purdue.edu Sat Apr 23 21:06:22 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Sat, 23 Apr 2011 15:06:22 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: Message-ID: Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. Sent from my iPhone On Apr 23, 2011, at 6:19 AM, Jay K wrote: > I've asked before. I'm asking again. > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > @M3pthreads @M3userthreads > or somesuch. > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > I know roughly what implementing this would look like. > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > I'll have to reread. > Whatever problem it has, this change probably would too. > > > I'm willing to implement this myself. > I'm not asking for anyone else to do it. > > > I understand it makes LOCK harder to efficiently inline. > If/when LOCK becomes partly inlined, provide a switch? > > > Thank you, > - Jay > From dabenavidesd at yahoo.es Sun Apr 24 19:51:57 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 18:51:57 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: Message-ID: <423962.44965.qm@web29707.mail.ird.yahoo.com> Hi all: In ref [2] it is claimed implemented stack walking exception handling for Modula-3 in SPINE using GCC 2.96 RT. Can you watch it? http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.8017 As also the strind schedulling for the SMP CPU Thanks in advance [1] D. Gough, S. Drew, K. J. Gough, and J. Ledermann, ?Implementing Zero Overhead Exception Handling,? 1995. [2] M. E. Fiuczynski, ?Safe and efficient resource sharing in component-based systems,? University of Washington, 2004. --- El s?b, 23/4/11, Tony Hosking escribi?: > De: Tony Hosking > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: "Jay K" > CC: "m3devel" > Fecha: s?bado, 23 de abril, 2011 14:06 > Good point about lock. I really think > this is the showstopper because compiled code will be fo a > particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K > wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and > pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a > line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look > like. > > > > > > I am a bit confused as to what problems > -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would > too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently > inline. > > If/when LOCK becomes partly inlined, provide a > switch? > > > > > > Thank you, > > - Jay > > > From dabenavidesd at yahoo.es Sun Apr 24 22:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 21:47:28 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ Message-ID: <319537.44680.qm@web29718.mail.ird.yahoo.com> Hi all: using google found this ftp directory (Modula2+ VAX): ftp://ftp.cisco.com/pub/rfc/DEC If DEC guys allowed to read and copy this is because they worked not just its products, but also their specifications, would be nice to see the Time modules explained in: /TIME_V1_0_0.PS If it's of enterprise and historical (academic) interest language could gain recognition from that, I mean they founded several open source projects back in time, which became later de-facto standards, it would be nice to see if this is of enough interest of their internal sources (can't forget Modula-3 was just one of them). They further developed Modula2+ into Modula2+Epsilon, which could be said to be a Modula3 in house brother if so. Furthermore there is a private source code library of internal design/layout chips components, later things could show they did watch to continue some of their work of prior language(s) or of its younger brother. See: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 Anyway this might be seen as a history on programming languages PhD project or so and computers if so (my CS professor told DEC machines were able to be specified to some large extent by its consumers and it seems by their producers too e.g like VAXstations). I never had the opportunity of doing something like that, perhaps computers have change, or like most people has changed, interesting ... Thanks in advance From dabenavidesd at yahoo.es Mon Apr 25 00:54:26 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Sun, 24 Apr 2011 23:54:26 +0100 (BST) Subject: [M3devel] Architecture descriptions of software/hardware in Modula-2+ In-Reply-To: <319537.44680.qm@web29718.mail.ird.yahoo.com> Message-ID: <34227.88904.qm@web29715.mail.ird.yahoo.com> Hi all: fortunately they had recognition was made by some sue, but for an issue; the claim they created a thing for the development of it: http://www.google.com/patents?id=PY0kAAAAEBAJ&zoom=4&pg=PA1#v=onepage&q&f=false Which is nice given that a patent is hold means that somehow everybody agrees with that I guess, but who *may not* be affected by it, exactly the counterexample of just another read my lips example :) of patents. Thanks in advance --- El dom, 24/4/11, Daniel Alejandro Benavides D. escribi?: > De: Daniel Alejandro Benavides D. > Asunto: [M3devel] Architecture descriptions of software/hardware in Modula-2+ > Para: m3devel at elegosoft.com > Fecha: domingo, 24 de abril, 2011 15:47 > Hi all: > using google found this ftp directory (Modula2+ VAX): > > ftp://ftp.cisco.com/pub/rfc/DEC > > If DEC guys allowed to read and copy this is because they > worked not just its products, but also their specifications, > would be nice to see the Time modules explained in: > /TIME_V1_0_0.PS > > If it's of enterprise and historical (academic) interest > language could gain recognition from that, I mean they > founded several open source projects back in time, which > became later de-facto standards, it would be nice to see if > this is of enough interest of their internal sources (can't > forget Modula-3 was just one of them). > > They further developed Modula2+ into Modula2+Epsilon, which > could be said to be a Modula3 in house brother if so. > > Furthermore there is a private source code library of > internal design/layout chips components, later things could > show they did watch to continue some of their work of prior > language(s) or of its younger brother. See: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.4844 > > Anyway this might be seen as a history on programming > languages PhD project or so and computers if so (my CS > professor told DEC machines were able to be specified to > some large extent by its consumers and it seems by their > producers too e.g like VAXstations). I never had the > opportunity of doing something like that, perhaps computers > have change, or like most people has changed, interesting > ... > > Thanks in advance > From rodney_bates at lcwb.coop Tue Apr 26 18:28:34 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 11:28:34 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <20110419115106.7E6A31A207A@async.async.caltech.edu> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> Message-ID: <4DB6F2B2.3090905@lcwb.coop> I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry more quickly than I can. On 04/19/2011 06:51 AM, Mika Nystrom wrote: > Hello m3devel, > > If anyone knows how to fix the following trivially, please go ahead. > > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal > > (errors reported by m3tk). > > The offending line is: > > INTERFACE BasicCtypes; > > IMPORT Word, Long; > > TYPE > (* the four signed integer types *) > signed_char = [-16_7f-1 .. 16_7f]; > short_int = [-16_7fff-1 .. 16_7fff]; > int = [-16_7fffffff-1 .. 16_7fffffff]; > long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; > long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) > > (* the four unsigned integer types *) > unsigned_char = [16_0 .. 16_ff]; > unsigned_short_int = [16_0 .. 16_ffff]; > unsigned_int = [16_0 .. 16_ffffffff]; > unsigned_long_int = Word.T; > unsigned_long_long = Long.T; > > (* the three floating types *) > float = REAL; > double = LONGREAL; > long_double = EXTENDED; > > (* char *) > char = signed_char; > > END BasicCtypes. > > I'll look at it myself, but not immediately (no time). Workaround for now. > > Mika > From jay.krell at cornell.edu Wed Apr 27 09:05:03 2011 From: jay.krell at cornell.edu (Jay K) Date: Wed, 27 Apr 2011 07:05:03 +0000 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: About inlining LOCK I was more echoing your past sentiment than giving my own, and I provided a mitigation: a) specifically the suggestion that the switchability be an option; it'd probably be on by default but people that want to squeeze out the extra performance could enable the inlining and lose the switchability About the need to link to pthread. Good point. I hadn't thought of that. BUT two points in possible rebuttal (not that is necessary an argument! more like a conversation) I'm sure that on some platforms there is no distinction. No separate broken libc.so and working libpthread.so. But yes, I realize the recent talk was about Linux, apparently it is broken. Solaris is probably better. Others I don't know, I'd have to check. Specifically, there should not be two separate fork implementations. Or two separate malloc implementations. The one and only fork must know about pthread_atfork. The one and only malloc must be thread safe, with respect to pthreads. (but not necessarily Modula-3 user-threads, that is our own problem). If that means fork and pthread_atfork have to be in the same .so, and/or that malloc and pthread_* have to be in the same .so, both of which are easy to believe are true, so be it. I shouldn't get one malloc via cc foo.c and a different one via cc foo.c -lpthread. The rare single-threaded process isn't worth optimizing. Isn't worth the headache. As well, though..and this isn't trivial, but I wonder if we should dispense with "static dynamic" linking, where we just call fork, malloc, etc., but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. That might address some of this libc vs. libpthread problem? But then the ABI rather than the source interface becomes needed to be known. Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? Hm. No. Nevermind. What is the problem again with using -pthread on Linux with user threads? So, maybe it pulls in an extra .so and has some small startup cost. Surely it should work? I'll have to dig in to the email and reproduce the bug. I'm of a mind now to remove the recent pthread_atfork stuff. But first I or someone has to understand the problem I was solving. Right now, I don't. The LOCK inlining problem seems very hypothetical right now. We don't have such a feature. Will we really ever? Is pthread_lockmutex partially inlined in C? Or have pthreads generally been implemented well enough that everyone is satisfied with no inlining? Or does Modula-3 sort of promote threads as lighter weight than they are in typical practise? And ditto for locking? You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, I suspect all the commercial Unices, had already presented and reasonably implemented the ideal threading interface, but all the free Unices were still holding out and either not providing pthreads or not providing them well.. at the time Modula-3 was slightly ahead of its time, and one could imagine not using the underlying platform directly because it was too slow or not full featured enough. But hasn't everything just about caught up by now (except OpenBSD with its user pthreads...) (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 years ago circa Solaris 2.10 was pretty modern..) That is -- are pthreads really worth us optimizing beyond? Don't all the C and C++ programmers and systems providers get it by now and have provided almost exactly what we need? Sorry, I'm rambling... Later, - Jay > From: hosking at cs.purdue.edu > Date: Sat, 23 Apr 2011 15:06:22 -0400 > To: jay.krell at cornell.edu > CC: m3devel at elegosoft.com > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > Sent from my iPhone > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > I've asked before. I'm asking again. > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > @M3pthreads @M3userthreads > > or somesuch. > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > I know roughly what implementing this would look like. > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > I'll have to reread. > > Whatever problem it has, this change probably would too. > > > > > > I'm willing to implement this myself. > > I'm not asking for anyone else to do it. > > > > > > I understand it makes LOCK harder to efficiently inline. > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > Thank you, > > - Jay > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rodney_bates at lcwb.coop Tue Apr 26 21:50:08 2011 From: rodney_bates at lcwb.coop (Rodney M. Bates) Date: Tue, 26 Apr 2011 14:50:08 -0500 Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB6F2B2.3090905@lcwb.coop> References: <20110419115106.7E6A31A207A@async.async.caltech.edu> <4DB6F2B2.3090905@lcwb.coop> Message-ID: <4DB721F0.3010101@lcwb.coop> On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > I just checked in a patch tp M3CExpValue.m3 in m3tk, in the head, which I believe will fix this > bug. I compiled it, but have not tested it, as I am sure Mika has this case to retry > more quickly than I can. > P.S.: Hats off to whoever adapted m3tk for LONGINT. There are places _everywhere_ that needed modification. The whole thing is about as pedantic as code gets. Layers upon layers of linguistic abstractions that do mostly zero semantic abstraction. I traced two declarations through 10 levels each of referrals to other interfaces, all but one of them just renames with the same simple name and different qualifying interfaces, before finally getting to something with some meat. The whole thing smells very strongly of the Ada/Diana culture of ASTs. > On 04/19/2011 06:51 AM, Mika Nystrom wrote: >> Hello m3devel, >> >> If anyone knows how to fix the following trivially, please go ahead. >> >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,45: type error in arguments to binary operator >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,25: bad numeric literal >> "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", line 18,51: bad numeric literal >> >> (errors reported by m3tk). >> >> The offending line is: >> >> INTERFACE BasicCtypes; >> >> IMPORT Word, Long; >> >> TYPE >> (* the four signed integer types *) >> signed_char = [-16_7f-1 .. 16_7f]; >> short_int = [-16_7fff-1 .. 16_7fff]; >> int = [-16_7fffffff-1 .. 16_7fffffff]; >> long_int = [-16_7fffffffffffffff -1 .. 16_7fffffffffffffff ]; >> long_long = [-16_7fffffffffffffffL-1L .. 16_7fffffffffffffffL]; (** HERE **) >> >> (* the four unsigned integer types *) >> unsigned_char = [16_0 .. 16_ff]; >> unsigned_short_int = [16_0 .. 16_ffff]; >> unsigned_int = [16_0 .. 16_ffffffff]; >> unsigned_long_int = Word.T; >> unsigned_long_long = Long.T; >> >> (* the three floating types *) >> float = REAL; >> double = LONGREAL; >> long_double = EXTENDED; >> >> (* char *) >> char = signed_char; >> >> END BasicCtypes. >> >> I'll look at it myself, but not immediately (no time). Workaround for now. >> >> Mika >> > From hosking at cs.purdue.edu Wed Apr 27 17:20:15 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Wed, 27 Apr 2011 11:20:15 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: On Apr 27, 2011, at 3:05 AM, Jay K wrote: > About inlining LOCK I was more echoing your past sentiment > than giving my own, and I provided a mitigation: > a) specifically the suggestion that the switchability be an option; > it'd probably be on by default but people that want to squeeze > out the extra performance could enable the inlining > and lose the switchability It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > About the need to link to pthread. > Good point. I hadn't thought of that. > BUT two points in possible rebuttal (not that is necessary > an argument! more like a conversation) > I'm sure that on some platforms there is no distinction. > No separate broken libc.so and working libpthread.so. > But yes, I realize the recent talk was about Linux, apparently it is broken. > Solaris is probably better. > Others I don't know, I'd have to check. > > > Specifically, there should not be two separate fork implementations. > Or two separate malloc implementations. > > > The one and only fork must know about pthread_atfork. > The one and only malloc must be thread safe, with respect to pthreads. > (but not necessarily Modula-3 user-threads, that is our own problem). > > > If that means fork and pthread_atfork have to be in the same .so, > and/or that malloc and pthread_* have to be in the same .so, both > of which are easy to believe are true, so be it. > > > I shouldn't get one malloc via cc foo.c and a different > one via cc foo.c -lpthread. > The rare single-threaded process isn't worth optimizing. > Isn't worth the headache. > > > As well, though..and this isn't trivial, but I wonder if we should > dispense with "static dynamic" linking, where we just call fork, malloc, etc., > but instead use "dynamic dynamic" ie. using dlopen/dlsym aggressively. > That might address some of this libc vs. libpthread problem? > But then the ABI rather than the source interface becomes needed to be known. > Possibly a problem..or possibly it is easy and direct with nothing tricky in most of the headers...? > > > Hm. No. Nevermind. > What is the problem again with using -pthread on Linux with user threads? > So, maybe it pulls in an extra .so and has some small startup cost. > Surely it should work? > I'll have to dig in to the email and reproduce the bug. > I'm of a mind now to remove the recent pthread_atfork stuff. > But first I or someone has to understand the problem I was solving. > Right now, I don't. > > > The LOCK inlining problem seems very hypothetical right now. > We don't have such a feature. Will we really ever? > > > Is pthread_lockmutex partially inlined in C? > Or have pthreads generally been implemented well enough that everyone is satisfied > with no inlining? > > > Or does Modula-3 sort of promote threads as lighter weight than they are > in typical practise? And ditto for locking? > > > You know, 15+ years ago when Modula-3, Win32, I suspect Solaris, > I suspect all the commercial Unices, had already presented and reasonably > implemented the ideal threading interface, but all the free Unices were still > holding out and either not providing pthreads or not providing them well.. > at the time Modula-3 was slightly ahead of its time, and one could imagine > not using the underlying platform directly because it was too slow > or not full featured enough. But hasn't everything just about caught up by > now (except OpenBSD with its user pthreads...) > (Yeah...15 years ago, Solaris was probably still a bit wonky..more like 6 > years ago circa Solaris 2.10 was pretty modern..) > > > That is -- are pthreads really worth us optimizing beyond? > Don't all the C and C++ programmers and systems providers get it by now > and have provided almost exactly what we need? > > > Sorry, I'm rambling... > > > Later, > - Jay > > > > From: hosking at cs.purdue.edu > > Date: Sat, 23 Apr 2011 15:06:22 -0400 > > To: jay.krell at cornell.edu > > CC: m3devel at elegosoft.com > > Subject: Re: [M3devel] ease of switching between pthreads and userthreads? > > > > Good point about lock. I really think this is the showstopper because compiled code will be fo a particular threading model. > > > > Sent from my iPhone > > > > On Apr 23, 2011, at 6:19 AM, Jay K wrote: > > > > > I've asked before. I'm asking again. > > > > > > > > > I'd like to be able to switch between userthreads and pthreads via a command line switch to the executable. > > > @M3pthreads @M3userthreads > > > or somesuch. > > > > > > > > > I'd also like to be able to switch the default via a line in a m3makefile that builds a program. > > > > > > > > > I know roughly what implementing this would look like. > > > > > > > > > I am a bit confused as to what problems -pthread/-pthreads/-lpthread causes. > > > I'll have to reread. > > > Whatever problem it has, this change probably would too. > > > > > > > > > I'm willing to implement this myself. > > > I'm not asking for anyone else to do it. > > > > > > > > > I understand it makes LOCK harder to efficiently inline. > > > If/when LOCK becomes partly inlined, provide a switch? > > > > > > > > > Thank you, > > > - Jay > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dabenavidesd at yahoo.es Wed Apr 27 23:53:31 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Wed, 27 Apr 2011 22:53:31 +0100 (BST) Subject: [M3devel] LONGINT -> more bugs in m3tk In-Reply-To: <4DB721F0.3010101@lcwb.coop> Message-ID: <689426.54766.qm@web29719.mail.ird.yahoo.com> Hi all: how does the arithmetic of LONGREAL affects that of the LONGINT, since both depend on that every BITSIZE should have comparable sizes, shouldn't it? I mean opaque types must be declared in both size LONGINT type and/or INTEGER size(s) to be able to be instantiated from memory in FPU in both cases, I know x87 is bigger size for its integers, besides what happens if the system doesn't provide that, should one consider implementing all (the) type conversions? Here: http://modula3.elegosoft.com/cm3/doc/reference/real-intf.html I think this one lets open the door for such conversions if I may say so: http://modula3.elegosoft.com/cm3/doc/reference/arithmetic.html I know one of the annoyances of old x86 FPU was to make any format conversion to CPU arithmetic thus converting from floats was very onerous in terms of losing your maximized precision in one assignment and bigger performance issues which made impractical for both sides, since initially every cycle was different and so, but I guess all programmer must be aware of that, but if the systems provide better performance like in any reasonable bigger machine, why not? I will give a check of decndumber to seek if they do that at least they had elegant exception handling Thanks in advance --- El mar, 26/4/11, Rodney M. Bates escribi?: > De: Rodney M. Bates > Asunto: Re: [M3devel] LONGINT -> more bugs in m3tk > Para: m3devel at elegosoft.com > Fecha: martes, 26 de abril, 2011 14:50 > > > On 04/26/2011 11:28 AM, Rodney M. Bates wrote: > > I just checked in a patch tp M3CExpValue.m3 in m3tk, > in the head, which I believe will fix this > > bug. I compiled it, but have not tested it, as I am > sure Mika has this case to retry > > more quickly than I can. > > > > P.S.: Hats off to whoever adapted m3tk for > LONGINT. There are places _everywhere_ that needed > modification. The whole thing is about as pedantic as > code gets. Layers upon layers of > linguistic abstractions that do mostly zero semantic > abstraction. I traced two declarations > through 10 levels each of referrals to other interfaces, > all but one of them just renames > with the same simple name and different qualifying > interfaces, before finally getting to > something with some meat. > > The whole thing smells very strongly of the Ada/Diana > culture of ASTs. > > > On 04/19/2011 06:51 AM, Mika Nystrom wrote: > >> Hello m3devel, > >> > >> If anyone knows how to fix the following > trivially, please go ahead. > >> > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,45: type error in arguments to binary operator > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,25: bad numeric literal > >> > "/usr/local/cm3/pkg/m3core/src/C/64BITS/BasicCtypes.i3", > line 18,51: bad numeric literal > >> > >> (errors reported by m3tk). > >> > >> The offending line is: > >> > >> INTERFACE BasicCtypes; > >> > >> IMPORT Word, Long; > >> > >> TYPE > >> (* the four signed integer types *) > >> signed_char = [-16_7f-1 .. 16_7f]; > >> short_int = [-16_7fff-1 .. 16_7fff]; > >> int = [-16_7fffffff-1 .. 16_7fffffff]; > >> long_int = [-16_7fffffffffffffff -1 .. > 16_7fffffffffffffff ]; > >> long_long = [-16_7fffffffffffffffL-1L .. > 16_7fffffffffffffffL]; (** HERE **) > >> > >> (* the four unsigned integer types *) > >> unsigned_char = [16_0 .. 16_ff]; > >> unsigned_short_int = [16_0 .. 16_ffff]; > >> unsigned_int = [16_0 .. 16_ffffffff]; > >> unsigned_long_int = Word.T; > >> unsigned_long_long = Long.T; > >> > >> (* the three floating types *) > >> float = REAL; > >> double = LONGREAL; > >> long_double = EXTENDED; > >> > >> (* char *) > >> char = signed_char; > >> > >> END BasicCtypes. > >> > >> I'll look at it myself, but not immediately (no > time). Workaround for now. > >> > >> Mika > >> > > > From wagner at elegosoft.com Thu Apr 28 09:29:14 2011 From: wagner at elegosoft.com (Olaf Wagner) Date: Thu, 28 Apr 2011 09:29:14 +0200 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: References: , Message-ID: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Quoting Tony Hosking : > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > >> About inlining LOCK I was more echoing your past sentiment >> than giving my own, and I provided a mitigation: >> a) specifically the suggestion that the switchability be an option; >> it'd probably be on by default but people that want to squeeze >> out the extra performance could enable the inlining >> and lose the switchability > > It is expected that LOCK will be implemented using CAS (and > equivalents) in the very near future. I have a student working on > it. This will be incompatible with the current user-threads > implementation of LOCK. It will be too painful to support both > using a compiler switch. Couldn't we change the existing user threads to use CAS for locks, too, in order to gain runtime switching of thread implementations? I have neither thought much about it nor looked at the implementation in recent years, so I hope it's no stupid question. Olaf -- Olaf Wagner -- elego Software Solutions GmbH Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 From dabenavidesd at yahoo.es Thu Apr 28 15:47:28 2011 From: dabenavidesd at yahoo.es (Daniel Alejandro Benavides D.) Date: Thu, 28 Apr 2011 14:47:28 +0100 (BST) Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: <212562.68766.qm@web29706.mail.ird.yahoo.com> Hi all: yes, that would be awesome ! :) instead of having the threads question of select one be able to switch, we can have sort of multiprocessor implementation capabilities and/or simple multi threading both for free in the RT like for special SPIN, however you should have migration capabilities between both address spaces and sort of a RT executive to control threads of both spaces: http://courses.cs.vt.edu/~cs5204/fall03/Papers/Migration/mathiske95migrating.pdf Similarly it would be good to see how much hard would it be to set distributed applications to get RT easily distributed in m3core, libm3, etc: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.9178 for something like the compiler, type checker, etc So any thread could live in a different machine in desired space --- El jue, 28/4/11, Olaf Wagner escribi?: > De: Olaf Wagner > Asunto: Re: [M3devel] ease of switching between pthreads and userthreads? > Para: m3devel at elegosoft.com > Fecha: jueves, 28 de abril, 2011 02:29 > Quoting Tony Hosking : > > > On Apr 27, 2011, at 3:05 AM, Jay K wrote: > > > >> About inlining LOCK I was more echoing your past > sentiment > >> than giving my own, and I provided a mitigation: > >> a) specifically the suggestion that the > switchability be an option; > >> it'd probably be on by default > but people that want to squeeze > >> out the extra performance could > enable the inlining > >> and lose the switchability > > > > It is expected that LOCK will be implemented using CAS > (and equivalents) in the very near future. I > have a student working on it. This will be > incompatible with the current user-threads > implementation of LOCK. It will be too painful to > support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for > locks, too, > in order to gain runtime switching of thread > implementations? > I have neither thought much about it nor looked at the > implementation > in recent years, so I hope it's no stupid question. > > Olaf > --Olaf Wagner -- elego Software Solutions GmbH > > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 > Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 > 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | > Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | > USt-IdNr: DE163214194 > > From hosking at cs.purdue.edu Thu Apr 28 16:22:36 2011 From: hosking at cs.purdue.edu (Tony Hosking) Date: Thu, 28 Apr 2011 10:22:36 -0400 Subject: [M3devel] ease of switching between pthreads and userthreads? In-Reply-To: <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> References: , <20110428092914.a7v6r4pzok4so0k8@mx0.elegosoft.com> Message-ID: On Apr 28, 2011, at 3:29 AM, Olaf Wagner wrote: > Quoting Tony Hosking : > >> On Apr 27, 2011, at 3:05 AM, Jay K wrote: >> >>> About inlining LOCK I was more echoing your past sentiment >>> than giving my own, and I provided a mitigation: >>> a) specifically the suggestion that the switchability be an option; >>> it'd probably be on by default but people that want to squeeze >>> out the extra performance could enable the inlining >>> and lose the switchability >> >> It is expected that LOCK will be implemented using CAS (and equivalents) in the very near future. I have a student working on it. This will be incompatible with the current user-threads implementation of LOCK. It will be too painful to support both using a compiler switch. > > Couldn't we change the existing user threads to use CAS for locks, too, > in order to gain runtime switching of thread implementations? > I have neither thought much about it nor looked at the implementation > in recent years, so I hope it's no stupid question. Hmm. Yes, I suppose we could. It might degrade performance of the user-level (uniprocessor) threads implementation. > > Olaf > -- > Olaf Wagner -- elego Software Solutions GmbH > Gustav-Meyer-Allee 25 / Geb?ude 12, 13355 Berlin, Germany > phone: +49 30 23 45 86 96 mobile: +49 177 2345 869 fax: +49 30 23 45 86 95 > http://www.elegosoft.com | Gesch?ftsf?hrer: Olaf Wagner | Sitz: Berlin > Handelregister: Amtsgericht Charlottenburg HRB 77719 | USt-IdNr: DE163214194 >