[M3devel] race conditions in lock-free code...
Daniel Alejandro Benavides D.
dabenavidesd at yahoo.es
Mon Aug 27 23:54:16 CEST 2012
Hi all:
I believe there shouldn't be so many for not being counted with fingers of one hand. The memory model was specified and model checked with TLA by Lamport and publicly available for bug finders contest.
However being an UNSAFE MODULE I suspect this was intended to explode that relaxed restriction. Also probable a C weak semantics could make things worse or better than they are, so ...
Thanks in advance
--- El lun, 27/8/12, Mika Nystrom <mika at async.caltech.edu> escribió:
De: Mika Nystrom <mika at async.caltech.edu>
Asunto: Re: [M3devel] race conditions in lock-free code...
Para: "Jay K" <jay.krell at cornell.edu>
CC: m3devel at elegosoft.com
Fecha: lunes, 27 de agosto, 2012 13:34
yeah I agree.
processor 1 performs...
x := alloc_new_text
x^ := character
return x
global := x
processor 2 performs
IF global # NIL THEN use(global) END
now if out of all this, x^ := character is held back, e.g., in a processor
write buffer, and if use(global) involves performing global^, the value
of global^ will be the value of global^ just after x := alloc_new_text
and before x^ := character.
I believe very aggressive architectures such as Alpha permit reordering
to that extent. If memory serves, on Alpha, if you do not perform a
Memory Barrier, pretty much nothing can be inferred on one processor
about the ordering of operations on another.
X86 I am not sure is that aggressive. In a hardware-synchronized
multiprocessor environment without barrier operations using e.g. the
Illinois/MESI protocol (doesn't X86?), at the point x^ := character is
performed, the cache coherence protocol has to guarantee the cache line
is in state Exclusive, which means it has to be evicted from the cache
in processor 2 and when use(global) comes around the protocol will
have to ensure it is in Shared state on processor 2. (Or Exclusive,
if P1 writes it back and invalidates.) This would guarantee that you
could not actually see a very old value for global^ at the same time
as a newer value for global. The protocol guarantees that the relative
ordering of writes is maintained across processors.
Of course I don't know whether Alphas are actually that aggressive in
practice. I'd suspect that a multiprocessor 21264 could be. But I do
agree it's a bug relative to the Alpha Architecture Manual. A very nasty
one at that.
Mika
Jay K writes:
>--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_
>Content-Type: text/plain; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
>It is NOT benign.
>The global can be written before the initialization of what it points to is=
> done.
>The result is wrong.
>
>
>It IS benign if multiple threads go and allocate separate multiple initiali=
>zed TEXTs.
>That's not the problem.
>
>
> - Jay
>
>
>> CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com
>> From: hosking at cs.purdue.edu
>> Subject: Re: [M3devel] race conditions in lock-free code...
>> Date: Mon=2C 27 Aug 2012 09:27:38 -0400
>> To: mika at async.caltech.edu
>>=20
>> So why is this anything other than benign? It's a cache and there's no pr=
>oblem if different threads get different texts. Don't cripple performance u=
>nnecessarily. The collector will reclaim any unreachable text.
>>=20
>> Sent from my iPad
>>=20
>> On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <mika at async.caltech.edu> =
>wrote:
>>=20
>> > Oh I see now.
>> >=20
>> > It's not that the processor reorders in any tricky way. It is that the
>> > write performed by Text8.New could be sitting around processor 1's cach=
>e
>> > indefinitely without actually getting written back to main memory.
>> > Processor 2 might see the new pointer=2C but when reading the memory po=
>inted
>> > to=2C would get uninitialized memory. I think this can happen on Alpha=
> but
>> > maybe not X86 (without further reorderings anyhow)?
>> >=20
>> > Any chance any of the pthreads bugs could be of this nature? How are t=
>he
>> > pthreads doing? I gathered some bugs were fixed but I haven't tried th=
>e
>> > system for a while.
>> >=20
>> > Mika
>> >=20
>> >=20
>> > Jay K writes:
>> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_
>> >> Content-Type: text/plain=3B charset=3D"iso-8859-1"
>> >> Content-Transfer-Encoding: quoted-printable
>> >>=20
>> >> The processor can reorder. Across call/ret at least in general.
>> >>=20
>> >>=20
>> >> The compiler can also reorder=3D2C if you have whole-program-optimizat=
>ion=3D2C =3D
>> >> link-time-code-gen (Microsoft "LTCG")=3D2C link-time-optimization (gcc=
> "LTO")=3D
>> >> . (We don't=3D2C but real world systems certainly do.)
>> >>=20
>> >>=20
>> >> - Jay
>> >>=20
>> >>=20
>> >>> To: jay.krell at cornell.edu=3D3B m3devel at elegosoft.com=3D3B mika at async.=
>caltech.=3D
>> >> edu
>> >>> Subject: Re: [M3devel] race conditions in lock-free code...
>> >>> Date: Sun=3D2C 26 Aug 2012 22:32:48 -0700
>> >>> From: mika at async.caltech.edu
>> >>> =3D20
>> >>> Sorry=3D2C I can't seem to quote your message.
>> >>> =3D20
>> >>> You say that Text8.New's result can be assigned to fromCharCache[c]
>> >>> before Text8.New is done running. How is that possible? Surely
>> >>> Text8.New has to calculate its result before it can be assigned
>> >>> anywhere. Yes possibly FromChar allocates space for the result of
>> >>> Text8.New but the assignment of whatever appears in that space can't
>> >>> happen until *after* Text8.New has run. Unless you are saying that
>> >>> the running of Text8.New and FromChar are intertwined somehow..?
>> >>> I know compilers do some odd things but that doesn't sound right...
>> >>> =3D20
>> >>> If Text8.New took fromCharCache[c] as a VAR parameter you might be
>> >>> right=3D2C though. But it doesn't...
>> >>> =3D20
>> >>> Am I missing something?
>> >>> =3D20
>> >>> Mika
>> >>> =3D20
>> >>> Mika Nystrom writes:
>> >>>>=20
>> >>>> Yeah it's a race condition=3D2C in theory. But is it important? If=
> you g=3D
>> >> et
>> >>>> the wrong copy=3D2C there will be two TEXTs representing one CHAR. =
>But no
>> >>>> one ever said FromChar was guaranteed to return the same pointer whe=
>n
>> >>>> you call it with the same CHAR...
>> >>>>=20
>> >>>> Mika
>> >>>>=20
>> >>>> Jay K writes:
>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_
>> >>>>> Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"
>> >>>>> Content-Transfer-Encoding: quoted-printable
>> >>>>>=20
>> >>>>> We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:
>> >>>>>=20
>> >>>>>=20
>> >>>>> VAR fromCharCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (=
>* 1-char t=3D
>> >> exts *)
>> >>>>>=20
>> >>>>>=20
>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D
>> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>> >>>>> BEGIN
>> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>> >>>>> buf [0] :=3D3D3D c=3D3D3B
>> >>>>> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B
>> >>>>> END=3D3D3B
>> >>>>> RETURN fromCharCache [c]
>> >>>>> END FromChar=3D3D3B
>> >>>>>=20
>> >>>>>=20
>> >>>>> It should be:
>> >>>>>=20
>> >>>>>=20
>> >>>>> PROCEDURE FromChar (c: CHAR): T =3D3D3D
>> >>>>> =3D3D0A=3D3D
>> >>>>> VAR buf: ARRAY [0..0] OF CHAR=3D3D3B
>> >>>>> =3D3D0A=3D3D
>> >>>>> BEGIN
>> >>>>> =3D3D0A=3D3D
>> >>>>> IF fromCharCache [c] =3D3D3D NIL THEN
>> >>>>> =3D3D0A=3D3D
>> >>>>> buf [0] :=3D3D3D c=3D3D3B
>> >>>>> =3D3D0A=3D3D
>> >>>>> WITH a =3D3D3D Text8.New (buf) DO
>> >>>>> MemoryBarrier()=3D3D3B
>> >>>>> fromCharCache[c] :=3D3D3D a=3D3D3B
>> >>>>> END=3D3D3B
>> >>>>> =3D3D0A=3D3D
>> >>>>> END=3D3D3B
>> >>>>> =3D3D0A=3D3D
>> >>>>> RETURN fromCharCache [c]
>> >>>>> =3D3D0A=3D3D
>> >>>>> END FromChar=3D3D3B
>> >>>>> =3D3D0A=3D3D
>> >>>>>=20
>> >>>>>=20
>> >>>>> to ensure that all of Text8.New() finishes before the assignment to=
> fro=3D
>> >> mCha=3D3D
>> >>>>> rCache[c] is made.
>> >>>>>=20
>> >>>>>=20
>> >>>>> Can the compiler somehow catch these?
>> >>>>> I fear they are a small epidemic.
>> >>>>> For a long time people didn't realize where all the compiler and pr=
>oces=3D
>> >> sor =3D3D
>> >>>>> could reorder.
>> >>>>>=20
>> >>>>>=20
>> >>>>> Do we have the right constructs by now to fix them?
>> >>>>> I think we do.
>> >>>>>=20
>> >>>>>=20
>> >>>>> - Jay
>> >>>>> =3D3D
>> >>>>>=20
>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_
>> >>>>> Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"
>> >>>>> Content-Transfer-Encoding: quoted-printable
>> >>>>>=20
>> >>>>> <html>
>> >>>>> <head>
>> >>>>> <style><!--
>> >>>>> .hmmessage P
>> >>>>> {
>> >>>>> margin:0px=3D3D3B
>> >>>>> padding:0px
>> >>>>> }
>> >>>>> body.hmmessage
>> >>>>> {
>> >>>>> font-size: 12pt=3D3D3B
>> >>>>> font-family:Calibri
>> >>>>> }
>> >>>>> --></style></head>
>> >>>>> <body class=3D3D3D'hmmessage'><div dir=3D3D3D'ltr'>We have race con=
>ditions=3D
>> >> =3D3D2C like =3D3D
>> >>>>> this=3D3D2C m3core/src/Text.m3:<br><br><br>VAR fromCharCache :=3D3D=
>3D ARRAY=3D
>> >> CHAR OF=3D3D
>> >>>>> T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<br><br><br>PROCEDURE Fr=
>omChar =3D
>> >> (c: CHAR=3D3D
>> >>>>> ): T =3D3D3D<br> =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B<b=
>r> =3D3D=3D
>> >> 3B BEGIN<br>=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N=
>IL THEN<br>&=3D
>> >> nbsp=3D3D3B =3D3D
>> >>>>> =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0] :=3D3D3D c=3D3D=
>3B<br> =3D3D3B=3D
>> >>  =3D3D3B =3D3D3B&n=3D3D
>> >>>>> bsp=3D3D3B =3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D=
>3D3B<br>&nbs=3D
>> >> p=3D3D3B =3D3D3B=3D3D
>> >>>>>  =3D3D3B END=3D3D3B<br> =3D3D3B =3D3D3B =3D3D3B RET=
>URN fromCharCa=3D
>> >> che [c]<br>&nb=3D3D
>> >>>>> sp=3D3D3B END FromChar=3D3D3B<br><br><br>It should be:<br><br><br>P=
>ROCEDURE=3D
>> >> FromCha=3D3D
>> >>>>> r (c: CHAR): T =3D3D3D<br>=3D3D0A=3D3D
>> >>>>>  =3D3D3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B<br>=3D3D0A=3D3D
>> >>>>>  =3D3D3B BEGIN<br>=3D3D0A=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B IF fromCharCache [c] =3D3D3D N=
>IL THEN<br>=3D
>> >> =3D3D0A=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B buf [0=
>] :=3D3D3D c=3D3D3=3D
>> >> B<br>=3D3D0A=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B WITH a=
> =3D3D3D Text8.N=3D
>> >> ew (buf) DO<br>&=3D3D
>> >>>>> nbsp=3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D=
>3D3B =3D3D3B M=3D
>> >> emoryBarrier()=3D3D3B<=3D3D
>> >>>>> br> =3D3D3B =3D3D3B =3D3D3B =3D3D3B =3D3D3B&nbs=
>p=3D3D3B =3D3D=3D
>> >> 3B fromCharCache[c=3D3D
>> >>>>> ] :=3D3D3D a=3D3D3B<br> =3D3D3B =3D3D3B =3D3D3B =3D=
>3D3B END=3D3D3B<br=3D
>> >>> =3D3D0A=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B END=3D3D3B<br>=3D3D0A=3D3D
>> >>>>>  =3D3D3B =3D3D3B =3D3D3B RETURN fromCharCache [c]<br>=
>=3D3D0A=3D3D
>> >>>>>  =3D3D3B END FromChar=3D3D3B<br>=3D3D0A=3D3D
>> >>>>> <br><br>to ensure that all of Text8.New() finishes before the assig=
>nmen=3D
>> >> t to=3D3D
>> >>>>> fromCharCache[c] is made.<br><br><br>Can the compiler somehow catch=
> th=3D
>> >> ese?=3D3D
>> >>>>> <br>I fear they are a small epidemic.<br>For a long time people did=
>n't =3D
>> >> real=3D3D
>> >>>>> ize where all the compiler and processor could reorder.<br><br><br>=
>Do w=3D
>> >> e ha=3D3D
>> >>>>> ve the right constructs by now to fix them?<br>I think we do.<br><b=
>r><b=3D
>> >> r> -=3D3D
>> >>>>> Jay<br> </div></body>
>> >>>>> </html>=3D3D
>> >>>>>=20
>> >>>>> --_5f23896e-68ee-44da-82dd-311cd8c58979_--
>> >> =3D
>> >>=20
>> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_
>> >> Content-Type: text/html=3B charset=3D"iso-8859-1"
>> >> Content-Transfer-Encoding: quoted-printable
>> >>=20
>> >> <html>
>> >> <head>
>> >> <style><!--
>> >> .hmmessage P
>> >> {
>> >> margin:0px=3D3B
>> >> padding:0px
>> >> }
>> >> body.hmmessage
>> >> {
>> >> font-size: 12pt=3D3B
>> >> font-family:Calibri
>> >> }
>> >> --></style></head>
>> >> <body class=3D3D'hmmessage'><div dir=3D3D'ltr'>The processor can reord=
>er. Acros=3D
>> >> s call/ret at least in general.<br><br><br>The compiler can also reord=
>er=3D2C=3D
>> >> if you have whole-program-optimization=3D2C link-time-code-gen (Micros=
>oft "L=3D
>> >> TCG")=3D2C link-time-optimization (gcc "LTO"). (We don't=3D2C but real=
> world sy=3D
>> >> stems certainly do.)<br><br><br> =3D3B- Jay<br><br><br><div><div i=
>d=3D3D"Sk=3D
>> >> yDrivePlaceholder"></div>>=3D3B To: jay.krell at cornell.edu=3D3B m3dev=
>el at elegos=3D
>> >> oft.com=3D3B mika at async.caltech.edu<br>>=3D3B Subject: Re: [M3devel]=
> race con=3D
>> >> ditions in lock-free code...<br>>=3D3B Date: Sun=3D2C 26 Aug 2012 22=
>:32:48 -0=3D
>> >> 700<br>>=3D3B From: mika at async.caltech.edu<br>>=3D3B <br>>=3D3B =
>Sorry=3D2C I =3D
>> >> can't seem to quote your message.<br>>=3D3B <br>>=3D3B You say tha=
>t Text8.N=3D
>> >> ew's result can be assigned to fromCharCache[c]<br>>=3D3B before Tex=
>t8.New =3D
>> >> is done running. How is that possible? Surely<br>>=3D3B Text8.New =
>has to =3D
>> >> calculate its result before it can be assigned<br>>=3D3B anywhere. =
>Yes pos=3D
>> >> sibly FromChar allocates space for the result of<br>>=3D3B Text8.New=
> but th=3D
>> >> e assignment of whatever appears in that space can't<br>>=3D3B happe=
>n until=3D
>> >> *after* Text8.New has run. Unless you are saying that<br>>=3D3B the=
> runni=3D
>> >> ng of Text8.New and FromChar are intertwined somehow..?<br>>=3D3B I =
>know co=3D
>> >> mpilers do some odd things but that doesn't sound right...<br>>=3D3B=
> <br>&g=3D
>> >> t=3D3B If Text8.New took fromCharCache[c] as a VAR parameter you might=
> be<br>=3D
>> >> >=3D3B right=3D2C though. But it doesn't...<br>>=3D3B <br>>=3D3=
>B Am I missin=3D
>> >> g something?<br>>=3D3B <br>>=3D3B Mika<br>>=3D3B <br>>=3D=
>3B Mika Nystr=3D
>> >> om writes:<br>>=3D3B >=3D3B<br>>=3D3B >=3D3BYeah it's a race c=
>ondition=3D2C i=3D
>> >> n theory. But is it important? If you get<br>>=3D3B >=3D3Bthe wr=
>ong copy=3D
>> >> =3D2C there will be two TEXTs representing one CHAR. But no<br>>=3D=
>3B >=3D3B=3D
>> >> one ever said FromChar was guaranteed to return the same pointer when<=
>br>&g=3D
>> >> t=3D3B >=3D3Byou call it with the same CHAR...<br>>=3D3B >=3D3B<=
>br>>=3D3B >=3D
>> >> =3D3B Mika<br>>=3D3B >=3D3B<br>>=3D3B >=3D3BJay K writes:<=
>br>>=3D3B >=3D
>> >> =3D3B>=3D3B--_5f23896e-68ee-44da-82dd-311cd8c58979_<br>>=3D3B >=
>=3D3B>=3D3BCon=3D
>> >> tent-Type: text/plain=3D3B charset=3D3D"iso-8859-1"<br>>=3D3B >=3D=
>3B>=3D3BConte=3D
>> >> nt-Transfer-Encoding: quoted-printable<br>>=3D3B >=3D3B>=3D3B<br=
>>>=3D3B >=3D
>> >> =3D3B>=3D3BWe have race conditions=3D3D2C like this=3D3D2C m3core/sr=
>c/Text.m3:<br=3D
>> >>> >=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B &g=
>t=3D3B>=3D3BVAR fromC=3D
>> >> harCache :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char tex=
>ts *)<br>&g=3D
>> >> t=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=
>=3D3B>=3D3BPROCEDURE Fr=3D
>> >> omChar (c: CHAR): T =3D3D3D<br>>=3D3B >=3D3B>=3D3B VAR buf: ARR=
>AY [0..0] OF =3D
>> >> CHAR=3D3D3B<br>>=3D3B >=3D3B>=3D3B BEGIN<br>>=3D3B >=3D3B&g=
>t=3D3B IF fromCh=3D
>> >> arCache [c] =3D3D3D NIL THEN<br>>=3D3B >=3D3B>=3D3B buf [0]=
> :=3D3D3D c=3D3D3=3D
>> >> B<br>>=3D3B >=3D3B>=3D3B fromCharCache[c] :=3D3D3D Text8.Ne=
>w (buf)=3D3D3B<=3D
>> >> br>>=3D3B >=3D3B>=3D3B END=3D3D3B<br>>=3D3B >=3D3B>=3D3=
>B RETURN fromCha=3D
>> >> rCache [c]<br>>=3D3B >=3D3B>=3D3B END FromChar=3D3D3B<br>>=3D=
>3B >=3D3B>=3D3B=3D
>> >> <br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3BIt should be=
>:<br>>=3D3B >=3D
>> >> =3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3=
>BPROCEDURE FromChar (=3D
>> >> c: CHAR): T =3D3D3D<br>>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D<br>>=3D=
>3B >=3D3B>=3D3B VA=3D
>> >> R buf: ARRAY [0..0] OF CHAR=3D3D3B<br>>=3D3B >=3D3B>=3D3B=3D3D0A=
>=3D3D<br>>=3D3B &=3D
>> >> gt=3D3B>=3D3B BEGIN<br>>=3D3B >=3D3B>=3D3B=3D3D0A=3D3D<br>>=
>=3D3B >=3D3B>=3D3B =3D
>> >> IF fromCharCache [c] =3D3D3D NIL THEN<br>>=3D3B >=3D3B>=3D3B=3D3=
>D0A=3D3D<br>>=3D
>> >> =3D3B >=3D3B>=3D3B buf [0] :=3D3D3D c=3D3D3B<br>>=3D3B >=
>=3D3B>=3D3B=3D3D0A=3D3D<=3D
>> >> br>>=3D3B >=3D3B>=3D3B WITH a =3D3D3D Text8.New (buf) DO<br=
>>>=3D3B >=3D
>> >> =3D3B>=3D3B MemoryBarrier()=3D3D3B<br>>=3D3B >=3D3B>=3D=
>3B fromCha=3D
>> >> rCache[c] :=3D3D3D a=3D3D3B<br>>=3D3B >=3D3B>=3D3B END=3D3D3=
>B<br>>=3D3B >=3D
>> >> =3D3B>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B END=3D3D3B<b=
>r>>=3D3B >=3D3B>=3D3B=3D
>> >> =3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B RETURN fromCharCache [c]<=
>br>>=3D3B >=3D
>> >> =3D3B>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B END FromChar=
>=3D3D3B<br>>=3D3B >=3D
>> >> =3D3B>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=
>=3D3B>=3D3B<br>>=3D3B &=3D
>> >> gt=3D3B>=3D3Bto ensure that all of Text8.New() finishes before the a=
>ssignment=3D
>> >> to fromCha=3D3D<br>>=3D3B >=3D3B>=3D3BrCache[c] is made.<br>>=
>=3D3B >=3D3B>=3D
>> >> =3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3BCan the=
> compiler somehow c=3D
>> >> atch these?<br>>=3D3B >=3D3B>=3D3BI fear they are a small epidem=
>ic.<br>>=3D
>> >> =3D3B >=3D3B>=3D3BFor a long time people didn't realize where all =
>the compile=3D
>> >> r and processor =3D3D<br>>=3D3B >=3D3B>=3D3Bcould reorder.<br>&g=
>t=3D3B >=3D3B&g=3D
>> >> t=3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3BDo we =
>have the right cons=3D
>> >> tructs by now to fix them?<br>>=3D3B >=3D3B>=3D3BI think we do.<=
>br>>=3D3B &=3D
>> >> gt=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=
>=3D3B - Jay<br>>=3D3B &=3D
>> >> gt=3D3B>=3D3B =3D3D<br>>=3D3B >=3D3B>=
>=3D3B<br>>=3D3B >=3D3B>=3D3B--_=3D
>> >> 5f23896e-68ee-44da-82dd-311cd8c58979_<br>>=3D3B >=3D3B>=3D3BCont=
>ent-Type: t=3D
>> >> ext/html=3D3B charset=3D3D"iso-8859-1"<br>>=3D3B >=3D3B>=3D3BCon=
>tent-Transfer-E=3D
>> >> ncoding: quoted-printable<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=
>=3D3B>=3D3B<=3D
>> >> =3D3Bhtml>=3D3B<br>>=3D3B >=3D3B>=3D3B<=3D3Bhead>=3D3B<br>=
>>=3D3B >=3D3B>=3D3B=3D
>> >> <=3D3Bstyle>=3D3B<=3D3B!--<br>>=3D3B >=3D3B>=3D3B.hmmessag=
>e P<br>>=3D3B >=3D
>> >> =3D3B>=3D3B{<br>>=3D3B >=3D3B>=3D3Bmargin:0px=3D3D3B<br>>=3D=
>3B >=3D3B>=3D3Bpadd=3D
>> >> ing:0px<br>>=3D3B >=3D3B>=3D3B}<br>>=3D3B >=3D3B>=3D3Bbody=
>.hmmessage<br>>=3D
>> >> =3D3B >=3D3B>=3D3B{<br>>=3D3B >=3D3B>=3D3Bfont-size: 12pt=3D=
>3D3B<br>>=3D3B >=3D
>> >> =3D3B>=3D3Bfont-family:Calibri<br>>=3D3B >=3D3B>=3D3B}<br>>=
>=3D3B >=3D3B>=3D3B=3D
>> >> -->=3D3B<=3D3B/style>=3D3B<=3D3B/head>=3D3B<br>>=3D3B >=
>=3D3B>=3D3B<=3D3Bbod=3D
>> >> y class=3D3D3D'hmmessage'>=3D3B<=3D3Bdiv dir=3D3D3D'ltr'>=3D3BWe=
> have race cond=3D
>> >> itions=3D3D2C like =3D3D<br>>=3D3B >=3D3B>=3D3Bthis=3D3D2C m3cor=
>e/src/Text.m3:<=3D
>> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BVAR fromCharCache :=
>=3D3D3D ARRAY CHAR =3D
>> >> OF=3D3D<br>>=3D3B >=3D3B>=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha=
>r texts *)<=3D3Bb=3D
>> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BPROCEDURE FromChar (c: CH=
>AR=3D3D<br>>=3D3B=3D
>> >> >=3D3B>=3D3B): T =3D3D3D<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B VAR=
> buf: ARRAY [0..0]=3D
>> >> OF CHAR=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr&g=
>t=3D3B=3D3D<br>>=3D
>> >> =3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3=
>Bnbsp=3D3D3B IF fromCha=3D
>> >> rCache [c] =3D3D3D NIL THEN<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=
>=3D3Bnbsp=3D3D<br>&g=3D
>> >> t=3D3B >=3D3B>=3D3B=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B=
>&=3D3Bnbsp=3D3D3B buf =3D
>> >> [0] :=3D3D3D c=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnb=
>sp=3D3D3B&=3D3Bnbsp=3D
>> >> =3D3D3B&=3D3Bn=3D3D<br>>=3D3B >=3D3B>=3D3Bbsp=3D3D3B&=3D3B=
>nbsp=3D3D3B fromCharCac=3D
>> >> he[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=
>=3D3D3B&=3D3Bnbsp=3D
>> >> =3D3D3B=3D3D<br>>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B END=3D3D=
>3B<=3D3Bbr>=3D3B&=3D
>> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B RETURN fromCh=
>arCache [c]<=3D3B=3D
>> >> br>=3D3B&=3D3Bnb=3D3D<br>>=3D3B >=3D3B>=3D3Bsp=3D3D3B END F=
>romChar=3D3D3B<=3D3Bb=3D
>> >> r>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BIt should be:<=3D3Bbr&g=
>t=3D3B<=3D3Bbr>=3D
>> >> =3D3B<=3D3Bbr>=3D3BPROCEDURE FromCha=3D3D<br>>=3D3B >=3D3B>=
>=3D3Br (c: CHAR): T =3D
>> >> =3D3D3D<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B&=
>=3D3Bnbsp=3D3D3B VAR buf:=3D
>> >> ARRAY [0..0] OF CHAR=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B =
>>=3D3B>=3D3B&=3D
>> >> =3D3Bnbsp=3D3D3B BEGIN<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B >=
>=3D3B>=3D3B&=3D3Bnbsp=3D
>> >> =3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B IF fromCharCache [c] =
>=3D3D3D NIL THEN&l=3D
>> >> t=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=
>=3D3D3B&=3D3Bnbsp=3D3D3B=3D
>> >> &=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B buf [0] :=
>=3D3D3D c=3D3D3B<=3D
>> >> =3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=
>=3D3D3B&=3D3Bnbsp=3D3D3B&=3D
>> >> amp=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B WITH a =3D=
>3D3D Text8.New (buf=3D
>> >> ) DO<=3D3Bbr>=3D3B&=3D3B=3D3D<br>>=3D3B >=3D3B>=3D3Bnbsp=
>=3D3D3B&=3D3Bnbsp=3D3D3=3D
>> >> B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3B=
>nbsp=3D3D3B&=3D3Bnbs=3D
>> >> p=3D3D3B MemoryBarrier()=3D3D3B<=3D3B=3D3D<br>>=3D3B >=3D3B>=
>=3D3Bbr>=3D3B&=3D3Bn=3D
>> >> bsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B=
>&=3D3Bnbsp=3D3D3B&am=3D
>> >> p=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<br>>=3D3B=
> >=3D3B>=3D3B] :=3D
>> >> =3D3D3D a=3D3D3B<=3D3Bbr>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D=
>3D3B&=3D3Bnbsp=3D3D3B&=3D
>> >> amp=3D3Bnbsp=3D3D3B END=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D=
>3B >=3D3B>=3D3B&=3D
>> >> =3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B END=3D3D3B<=
>=3D3Bbr>=3D3B=3D3D0A=3D3D=3D
>> >> <br>>=3D3B >=3D3B>=3D3B&=3D3Bnbsp=3D3D3B&=3D3Bnbsp=3D3D3B&=
>amp=3D3Bnbsp=3D3D3B RET=3D
>> >> URN fromCharCache [c]<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3B >=3D=
>3B>=3D3B&=3D3Bnb=3D
>> >> sp=3D3D3B END FromChar=3D3D3B<=3D3Bbr>=3D3B=3D3D0A=3D3D<br>>=3D3=
>B >=3D3B>=3D3B<=3D
>> >> =3D3Bbr>=3D3B<=3D3Bbr>=3D3Bto ensure that all of Text8.New() fin=
>ishes before =3D
>> >> the assignment to=3D3D<br>>=3D3B >=3D3B>=3D3B fromCharCache[c] i=
>s made.<=3D3B=3D
>> >> br>=3D3B<=3D3Bbr>=3D3B<=3D3Bbr>=3D3BCan the compiler somehow=
> catch these?=3D
>> >> =3D3D<br>>=3D3B >=3D3B>=3D3B<=3D3Bbr>=3D3BI fear they are a =
>small epidemic.&l=3D
>> >> t=3D3Bbr>=3D3BFor a long time people didn't real=3D3D<br>>=3D3B &g=
>t=3D3B>=3D3Bize=3D
>> >> where all the compiler and processor could reorder.<=3D3Bbr>=3D3B&=
>lt=3D3Bbr&=3D
>> >> gt=3D3B<=3D3Bbr>=3D3BDo we ha=3D3D<br>>=3D3B >=3D3B>=3D3Bve =
>the right construct=3D
>> >> s by now to fix them?<=3D3Bbr>=3D3BI think we do.<=3D3Bbr>=3D3=
>B<=3D3Bbr>=3D
>> >> =3D3B<=3D3Bbr>=3D3B -=3D3D<br>>=3D3B >=3D3B>=3D3B Jay<=3D3=
>Bbr>=3D3B =3D
>> >> <=3D3B/div>=3D3B<=3D3B/body>=3D3B<br>>=3D3B >=3D3B>=3D3B=
><=3D3B/html>=3D3B=3D
>> >> =3D3D<br>>=3D3B >=3D3B>=3D3B<br>>=3D3B >=3D3B>=3D3B--_5f23=
>896e-68ee-44da-82dd=3D
>> >> -311cd8c58979_--<br></div> </div></body>
>> >> </html>=3D
>> >>=20
>> >> --_86f22d85-5540-4565-bf73-232aaafdc94b_--
> =
>
>--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_
>Content-Type: text/html; charset="iso-8859-1"
>Content-Transfer-Encoding: quoted-printable
>
><html>
><head>
><style><!--
>.hmmessage P
>{
>margin:0px=3B
>padding:0px
>}
>body.hmmessage
>{
>font-size: 12pt=3B
>font-family:Calibri
>}
>--></style></head>
><body class=3D'hmmessage'><div dir=3D'ltr'>It is NOT benign.<br>The global =
>can be written before the initialization of what it points to is done.<br>T=
>he result is wrong.<br><br><br>It IS benign if multiple threads go and allo=
>cate separate multiple initialized TEXTs.<br>That's not the problem.<br><br=
>><br> =3B- Jay<br><br><br><div><div id=3D"SkyDrivePlaceholder"></div>&g=
>t=3B CC: jay.krell at cornell.edu=3B m3devel at elegosoft.com<br>>=3B From: hos=
>king at cs.purdue.edu<br>>=3B Subject: Re: [M3devel] race conditions in lock=
>-free code...<br>>=3B Date: Mon=2C 27 Aug 2012 09:27:38 -0400<br>>=3B T=
>o: mika at async.caltech.edu<br>>=3B <br>>=3B So why is this anything othe=
>r than benign? It's a cache and there's no problem if different threads get=
> different texts. Don't cripple performance unnecessarily. The collector w=
>ill reclaim any unreachable text.<br>>=3B <br>>=3B Sent from my iPad<br=
>>>=3B <br>>=3B On Aug 27=2C 2012=2C at 3:17 AM=2C Mika Nystrom <=3Bmi=
>ka at async.caltech.edu>=3B wrote:<br>>=3B <br>>=3B >=3B Oh I see now.=
><br>>=3B >=3B <br>>=3B >=3B It's not that the processor reorders in=
> any tricky way. It is that the<br>>=3B >=3B write performed by Text8.=
>New could be sitting around processor 1's cache<br>>=3B >=3B indefinite=
>ly without actually getting written back to main memory.<br>>=3B >=3B P=
>rocessor 2 might see the new pointer=2C but when reading the memory pointed=
><br>>=3B >=3B to=2C would get uninitialized memory. I think this can h=
>appen on Alpha but<br>>=3B >=3B maybe not X86 (without further reorderi=
>ngs anyhow)?<br>>=3B >=3B <br>>=3B >=3B Any chance any of the pthre=
>ads bugs could be of this nature? How are the<br>>=3B >=3B pthreads do=
>ing? I gathered some bugs were fixed but I haven't tried the<br>>=3B >=
>=3B system for a while.<br>>=3B >=3B <br>>=3B >=3B Mika<br>>=
>=3B >=3B <br>>=3B >=3B <br>>=3B >=3B Jay K writes:<br>>=3B >=
>=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_<br>>=3B >=3B>=3B C=
>ontent-Type: text/plain=3B charset=3D"iso-8859-1"<br>>=3B >=3B>=3B Co=
>ntent-Transfer-Encoding: quoted-printable<br>>=3B >=3B>=3B <br>>=3B=
> >=3B>=3B The processor can reorder. Across call/ret at least in genera=
>l.<br>>=3B >=3B>=3B <br>>=3B >=3B>=3B <br>>=3B >=3B>=3B T=
>he compiler can also reorder=3D2C if you have whole-program-optimization=3D=
>2C =3D<br>>=3B >=3B>=3B link-time-code-gen (Microsoft "LTCG")=3D2C li=
>nk-time-optimization (gcc "LTO")=3D<br>>=3B >=3B>=3B . (We don't=3D2C=
> but real world systems certainly do.)<br>>=3B >=3B>=3B <br>>=3B &g=
>t=3B>=3B <br>>=3B >=3B>=3B - Jay<br>>=3B >=3B>=3B <br>>=3B =
>>=3B>=3B <br>>=3B >=3B>=3B>=3B To: jay.krell at cornell.edu=3D3B m=
>3devel at elegosoft.com=3D3B mika at async.caltech.=3D<br>>=3B >=3B>=3B edu=
><br>>=3B >=3B>=3B>=3B Subject: Re: [M3devel] race conditions in loc=
>k-free code...<br>>=3B >=3B>=3B>=3B Date: Sun=3D2C 26 Aug 2012 22:3=
>2:48 -0700<br>>=3B >=3B>=3B>=3B From: mika at async.caltech.edu<br>>=
>=3B >=3B>=3B>=3B =3D20<br>>=3B >=3B>=3B>=3B Sorry=3D2C I can'=
>t seem to quote your message.<br>>=3B >=3B>=3B>=3B =3D20<br>>=3B =
>>=3B>=3B>=3B You say that Text8.New's result can be assigned to fromC=
>harCache[c]<br>>=3B >=3B>=3B>=3B before Text8.New is done running. =
> How is that possible? Surely<br>>=3B >=3B>=3B>=3B Text8.New has t=
>o calculate its result before it can be assigned<br>>=3B >=3B>=3B>=
>=3B anywhere. Yes possibly FromChar allocates space for the result of<br>&=
>gt=3B >=3B>=3B>=3B Text8.New but the assignment of whatever appears i=
>n that space can't<br>>=3B >=3B>=3B>=3B happen until *after* Text8.=
>New has run. Unless you are saying that<br>>=3B >=3B>=3B>=3B the r=
>unning of Text8.New and FromChar are intertwined somehow..?<br>>=3B >=
>=3B>=3B>=3B I know compilers do some odd things but that doesn't sound =
>right...<br>>=3B >=3B>=3B>=3B =3D20<br>>=3B >=3B>=3B>=3B If=
> Text8.New took fromCharCache[c] as a VAR parameter you might be<br>>=3B =
>>=3B>=3B>=3B right=3D2C though. But it doesn't...<br>>=3B >=3B&g=
>t=3B>=3B =3D20<br>>=3B >=3B>=3B>=3B Am I missing something?<br>&g=
>t=3B >=3B>=3B>=3B =3D20<br>>=3B >=3B>=3B>=3B Mika<br>>=
>=3B >=3B>=3B>=3B =3D20<br>>=3B >=3B>=3B>=3B Mika Nystrom writ=
>es:<br>>=3B >=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B =
>Yeah it's a race condition=3D2C in theory. But is it important? If you g=
>=3D<br>>=3B >=3B>=3B et<br>>=3B >=3B>=3B>=3B>=3B the wrong =
>copy=3D2C there will be two TEXTs representing one CHAR. But no<br>>=3B =
>>=3B>=3B>=3B>=3B one ever said FromChar was guaranteed to return th=
>e same pointer when<br>>=3B >=3B>=3B>=3B>=3B you call it with the=
> same CHAR...<br>>=3B >=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=
>=3B>=3B Mika<br>>=3B >=3B>=3B>=3B>=3B <br>>=3B >=3B>=
>=3B>=3B>=3B Jay K writes:<br>>=3B >=3B>=3B>=3B>=3B>=3B --_5=
>f23896e-68ee-44da-82dd-311cd8c58979_<br>>=3B >=3B>=3B>=3B>=3B>=
>=3B Content-Type: text/plain=3D3B charset=3D3D"iso-8859-1"<br>>=3B >=3B=
>>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable<br>>=
>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=
>=3B We have race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<br>=
>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B&g=
>t=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B VAR fromCharCache :=3D3D3D A=
>RRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char t=3D<br>>=3B >=3B>=3B=
> exts *)<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B&g=
>t=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B PROCEDURE FromCh=
>ar (c: CHAR): T =3D3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B VAR buf: A=
>RRAY [0..0] OF CHAR=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B BEGIN<b=
>r>>=3B >=3B>=3B>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL =
>THEN<br>>=3B >=3B>=3B>=3B>=3B>=3B buf [0] :=3D3D3D c=3D3D3B=
><br>>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=3D3D3D Tex=
>t8.New (buf)=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B<b=
>r>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c]<br>>=
>=3B >=3B>=3B>=3B>=3B>=3B END FromChar=3D3D3B<br>>=3B >=3B>=
>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B =
>>=3B>=3B>=3B>=3B>=3B It should be:<br>>=3B >=3B>=3B>=3B&g=
>t=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=
>=3B>=3B>=3B>=3B PROCEDURE FromChar (c: CHAR): T =3D3D3D<br>>=3B >=
>=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B=
>>=3B VAR buf: ARRAY [0..0] OF CHAR=3D3D3B<br>>=3B >=3B>=3B>=3B>=
>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B BEGIN<br>&g=
>t=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=
>=3B>=3B>=3B IF fromCharCache [c] =3D3D3D NIL THEN<br>>=3B >=3B>=
>=3B>=3B>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B=
> buf [0] :=3D3D3D c=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B =3D=
>3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B WITH a =3D3D3D Text8=
>.New (buf) DO<br>>=3B >=3B>=3B>=3B>=3B>=3B MemoryBarrier(=
>)=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B fromCharCache[c] :=
>=3D3D3D a=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B END=3D3D3B<br>=
>>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B&g=
>t=3B>=3B>=3B END=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B =3D3=
>D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B RETURN fromCharCache [c=
>]<br>>=3B >=3B>=3B>=3B>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=
>=3B>=3B>=3B>=3B END FromChar=3D3D3B<br>>=3B >=3B>=3B>=3B>=
>=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B =
>>=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B to=
> ensure that all of Text8.New() finishes before the assignment to fro=3D<br=
>>>=3B >=3B>=3B mCha=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B rCa=
>che[c] is made.<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B&=
>gt=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B Can the c=
>ompiler somehow catch these?<br>>=3B >=3B>=3B>=3B>=3B>=3B I fea=
>r they are a small epidemic.<br>>=3B >=3B>=3B>=3B>=3B>=3B For a=
> long time people didn't realize where all the compiler and proces=3D<br>&g=
>t=3B >=3B>=3B sor =3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B could =
>reorder.<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B&g=
>t=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B Do we have the r=
>ight constructs by now to fix them?<br>>=3B >=3B>=3B>=3B>=3B>=
>=3B I think we do.<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=
>=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B>=3B - Jay=
><br>>=3B >=3B>=3B>=3B>=3B>=3B =3D3D<br>=
>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=3B&g=
>t=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_<br>>=3B >=3B>=3B>=3B&=
>gt=3B>=3B Content-Type: text/html=3D3B charset=3D3D"iso-8859-1"<br>>=3B=
> >=3B>=3B>=3B>=3B>=3B Content-Transfer-Encoding: quoted-printable=
><br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=3B >=3B>=3B>=3B>=
>=3B>=3B <=3Bhtml>=3B<br>>=3B >=3B>=3B>=3B>=3B>=3B <=3Bh=
>ead>=3B<br>>=3B >=3B>=3B>=3B>=3B>=3B <=3Bstyle>=3B<=3B!=
>--<br>>=3B >=3B>=3B>=3B>=3B>=3B .hmmessage P<br>>=3B >=3B&g=
>t=3B>=3B>=3B>=3B {<br>>=3B >=3B>=3B>=3B>=3B>=3B margin:0p=
>x=3D3D3B<br>>=3B >=3B>=3B>=3B>=3B>=3B padding:0px<br>>=3B >=
>=3B>=3B>=3B>=3B>=3B }<br>>=3B >=3B>=3B>=3B>=3B>=3B body=
>.hmmessage<br>>=3B >=3B>=3B>=3B>=3B>=3B {<br>>=3B >=3B>=
>=3B>=3B>=3B>=3B font-size: 12pt=3D3D3B<br>>=3B >=3B>=3B>=3B&g=
>t=3B>=3B font-family:Calibri<br>>=3B >=3B>=3B>=3B>=3B>=3B }<b=
>r>>=3B >=3B>=3B>=3B>=3B>=3B -->=3B<=3B/style>=3B<=3B/he=
>ad>=3B<br>>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbody class=3D3D3D'h=
>mmessage'>=3B<=3Bdiv dir=3D3D3D'ltr'>=3BWe have race conditions=3D<br=
>>>=3B >=3B>=3B =3D3D2C like =3D3D<br>>=3B >=3B>=3B>=3B>=3B&=
>gt=3B this=3D3D2C m3core/src/Text.m3:<=3Bbr>=3B<=3Bbr>=3B<=3Bbr&g=
>t=3BVAR fromCharCache :=3D3D3D ARRAY=3D<br>>=3B >=3B>=3B CHAR OF=3D3D=
><br>>=3B >=3B>=3B>=3B>=3B>=3B T {NIL=3D3D2C ..}=3D3D3B (* 1-cha=
>r texts *)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BPROCEDURE FromChar =3D<=
>br>>=3B >=3B>=3B (c: CHAR=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=
>=3B ): T =3D3D3D<=3Bbr>=3B&=3Bnbsp=3D3D3B VAR buf: ARRAY [0..0] OF C=
>HAR=3D3D3B<=3Bbr>=3B&=3Bnbsp=3D3D=3D<br>>=3B >=3B>=3B 3B BEGIN=
><=3Bbr>=3B=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D=
>3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL T=
>HEN<=3Bbr>=3B&=3B=3D<br>>=3B >=3B>=3B nbsp=3D3D3B&=3Bnbsp=
>=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B =3D3D3B&=3Bnbsp=3D3D3B&am=
>p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&a=
>mp=3Bnbsp=3D3D3B=3D<br>>=3B >=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D=
>3D3B&=3Bn=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B bsp=3D3D3B&=
>=3Bnbsp=3D3D3B fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3Bbr>=
>=3B&=3Bnbs=3D<br>>=3B >=3B>=3B p=3D3D3B&=3Bnbsp=3D3D3B=3D3D<br>=
>>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr=
>>=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RETURN fromCha=
>rCa=3D<br>>=3B >=3B>=3B che [c]<=3Bbr>=3B&=3Bnb=3D3D<br>>=3B=
> >=3B>=3B>=3B>=3B>=3B sp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=
><=3Bbr>=3B<=3Bbr>=3BIt should be:<=3Bbr>=3B<=3Bbr>=3B<=3B=
>br>=3BPROCEDURE=3D<br>>=3B >=3B>=3B FromCha=3D3D<br>>=3B >=3B&g=
>t=3B>=3B>=3B>=3B r (c: CHAR): T =3D3D3D<=3Bbr>=3B=3D3D0A=3D3D<br>=
>>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B VAR buf: ARRAY [0.=
>.0] OF CHAR=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B&g=
>t=3B>=3B &=3Bnbsp=3D3D3B BEGIN<=3Bbr>=3B=3D3D0A=3D3D<br>>=3B >=
>=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=
>=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN<=3Bbr>=3B=3D<br>>=3B &g=
>t=3B>=3B =3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbs=
>p=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=
>=3D3D3B buf [0] :=3D3D3D c=3D3D3=3D<br>>=3B >=3B>=3B B<=3Bbr>=3B=
>=3D3D0A=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&am=
>p=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B WITH =
>a =3D3D3D Text8.N=3D<br>>=3B >=3B>=3B ew (buf) DO<=3Bbr>=3B&=
>=3B=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B nbsp=3D3D3B&=3Bnbsp=3D=
>3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3=
>B&=3Bnbsp=3D3D3B M=3D<br>>=3B >=3B>=3B emoryBarrier()=3D3D3B<=3B=
>=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B br>=3B&=3Bnbsp=3D3D3B&a=
>mp=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=
>=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=3D<br>>=3B >=3B>=3B 3B fromCharCache[c=
>=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B ] :=3D3D3D a=3D3D3B<=3Bbr&=
>gt=3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D=
>3B END=3D3D3B<=3Bbr=3D<br>>=3B >=3B>=3B>=3B =3D3D0A=3D3D<br>>=
>=3B >=3B>=3B>=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=
>=3Bnbsp=3D3D3B END=3D3D3B<=3Bbr>=3B=3D3D0A=3D3D<br>>=3B >=3B>=3B&=
>gt=3B>=3B>=3B &=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B&=3Bnbsp=3D3D3B RE=
>TURN fromCharCache [c]<=3Bbr>=3B=3D3D0A=3D3D<br>>=3B >=3B>=3B>=
>=3B>=3B>=3B &=3Bnbsp=3D3D3B END FromChar=3D3D3B<=3Bbr>=3B=3D3D0A=
>=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B <=3Bbr>=3B<=3Bbr>=3B=
>to ensure that all of Text8.New() finishes before the assignmen=3D<br>>=
>=3B >=3B>=3B t to=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B fromCha=
>rCache[c] is made.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BCan the compile=
>r somehow catch th=3D<br>>=3B >=3B>=3B ese?=3D3D<br>>=3B >=3B>=
>=3B>=3B>=3B>=3B <=3Bbr>=3BI fear they are a small epidemic.<=3B=
>br>=3BFor a long time people didn't =3D<br>>=3B >=3B>=3B real=3D3D<=
>br>>=3B >=3B>=3B>=3B>=3B>=3B ize where all the compiler and pro=
>cessor could reorder.<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3BDo w=3D<br>&=
>gt=3B >=3B>=3B e ha=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B ve th=
>e right constructs by now to fix them?<=3Bbr>=3BI think we do.<=3Bbr&=
>gt=3B<=3Bbr>=3B<=3Bb=3D<br>>=3B >=3B>=3B r>=3B -=3D3D<br>>=
>=3B >=3B>=3B>=3B>=3B>=3B Jay<=3Bbr>=3B =
> <=3B/div>=3B<=3B/body>=3B<br>>=3B >=3B>=3B>=3B>=3B>=
>=3B <=3B/html>=3B=3D3D<br>>=3B >=3B>=3B>=3B>=3B>=3B <br>>=
>=3B >=3B>=3B>=3B>=3B>=3B --_5f23896e-68ee-44da-82dd-311cd8c58979_=
>--<br>>=3B >=3B>=3B =3D<br>>=3B >=3B>=
>=3B <br>>=3B >=3B>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_<br>>=
>=3B >=3B>=3B Content-Type: text/html=3B charset=3D"iso-8859-1"<br>>=
>=3B >=3B>=3B Content-Transfer-Encoding: quoted-printable<br>>=3B >=
>=3B>=3B <br>>=3B >=3B>=3B <=3Bhtml>=3B<br>>=3B >=3B>=3B &=
>lt=3Bhead>=3B<br>>=3B >=3B>=3B <=3Bstyle>=3B<=3B!--<br>>=3B=
> >=3B>=3B .hmmessage P<br>>=3B >=3B>=3B {<br>>=3B >=3B>=3B =
>margin:0px=3D3B<br>>=3B >=3B>=3B padding:0px<br>>=3B >=3B>=3B }=
><br>>=3B >=3B>=3B body.hmmessage<br>>=3B >=3B>=3B {<br>>=3B &=
>gt=3B>=3B font-size: 12pt=3D3B<br>>=3B >=3B>=3B font-family:Calibri=
><br>>=3B >=3B>=3B }<br>>=3B >=3B>=3B -->=3B<=3B/style>=3B=
><=3B/head>=3B<br>>=3B >=3B>=3B <=3Bbody class=3D3D'hmmessage'&g=
>t=3B<=3Bdiv dir=3D3D'ltr'>=3BThe processor can reorder. Acros=3D<br>>=
>=3B >=3B>=3B s call/ret at least in general.<=3Bbr>=3B<=3Bbr>=
>=3B<=3Bbr>=3BThe compiler can also reorder=3D2C=3D<br>>=3B >=3B>=
>=3B if you have whole-program-optimization=3D2C link-time-code-gen (Microso=
>ft "L=3D<br>>=3B >=3B>=3B TCG")=3D2C link-time-optimization (gcc "LTO=
>"). (We don't=3D2C but real world sy=3D<br>>=3B >=3B>=3B stems certai=
>nly do.)<=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B&=3Bnbsp=3D3B- Jay<=
>=3Bbr>=3B<=3Bbr>=3B<=3Bbr>=3B<=3Bdiv>=3B<=3Bdiv id=3D3D"Sk=
>=3D<br>>=3B >=3B>=3B yDrivePlaceholder">=3B<=3B/div>=3B&=3Bg=
>t=3D3B To: jay.krell at cornell.edu=3D3B m3devel at elegos=3D<br>>=3B >=3B>=
>=3B oft.com=3D3B mika at async.caltech.edu<=3Bbr>=3B&=3Bgt=3D3B Subject=
>: Re: [M3devel] race con=3D<br>>=3B >=3B>=3B ditions in lock-free cod=
>e...<=3Bbr>=3B&=3Bgt=3D3B Date: Sun=3D2C 26 Aug 2012 22:32:48 -0=3D<=
>br>>=3B >=3B>=3B 700<=3Bbr>=3B&=3Bgt=3D3B From: mika at async.cal=
>tech.edu<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Sorry=3D2=
>C I =3D<br>>=3B >=3B>=3B can't seem to quote your message.<=3Bbr>=
>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B You say that Text8.N=3D<br>&=
>gt=3B >=3B>=3B ew's result can be assigned to fromCharCache[c]<=3Bbr&=
>gt=3B&=3Bgt=3D3B before Text8.New =3D<br>>=3B >=3B>=3B is done run=
>ning. How is that possible? Surely<=3Bbr>=3B&=3Bgt=3D3B Text8.New =
>has to =3D<br>>=3B >=3B>=3B calculate its result before it can be ass=
>igned<=3Bbr>=3B&=3Bgt=3D3B anywhere. Yes pos=3D<br>>=3B >=3B>=
>=3B sibly FromChar allocates space for the result of<=3Bbr>=3B&=3Bgt=
>=3D3B Text8.New but th=3D<br>>=3B >=3B>=3B e assignment of whatever a=
>ppears in that space can't<=3Bbr>=3B&=3Bgt=3D3B happen until=3D<br>&=
>gt=3B >=3B>=3B *after* Text8.New has run. Unless you are saying that&l=
>t=3Bbr>=3B&=3Bgt=3D3B the runni=3D<br>>=3B >=3B>=3B ng of Text8.=
>New and FromChar are intertwined somehow..?<=3Bbr>=3B&=3Bgt=3D3B I k=
>now co=3D<br>>=3B >=3B>=3B mpilers do some odd things but that doesn'=
>t sound right...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bg=3D<br>&=
>gt=3B >=3B>=3B t=3D3B If Text8.New took fromCharCache[c] as a VAR param=
>eter you might be<=3Bbr>=3B=3D<br>>=3B >=3B>=3B &=3Bgt=3D3B ri=
>ght=3D2C though. But it doesn't...<=3Bbr>=3B&=3Bgt=3D3B <=3Bbr>=
>=3B&=3Bgt=3D3B Am I missin=3D<br>>=3B >=3B>=3B g something?<=3Bb=
>r>=3B&=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika<=3Bbr>=3B&=
>amp=3Bgt=3D3B <=3Bbr>=3B&=3Bgt=3D3B Mika Nystr=3D<br>>=3B >=3B&g=
>t=3B om writes:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&am=
>p=3Bgt=3D3B &=3Bgt=3D3BYeah it's a race condition=3D2C i=3D<br>>=3B &g=
>t=3B>=3B n theory. But is it important? If you get<=3Bbr>=3B&=3B=
>gt=3D3B &=3Bgt=3D3Bthe wrong copy=3D<br>>=3B >=3B>=3B =3D2C there =
>will be two TEXTs representing one CHAR. But no<=3Bbr>=3B&=3Bgt=3D3=
>B &=3Bgt=3D3B=3D<br>>=3B >=3B>=3B one ever said FromChar was guara=
>nteed to return the same pointer when<=3Bbr>=3B&=3Bg=3D<br>>=3B &g=
>t=3B>=3B t=3D3B &=3Bgt=3D3Byou call it with the same CHAR...<=3Bbr&g=
>t=3B&=3Bgt=3D3B &=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<=
>br>>=3B >=3B>=3B =3D3B Mika<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=
>=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3BJay K writes:<=3Bbr>=3B=
>&=3Bgt=3D3B &=3Bgt=3D<br>>=3B >=3B>=3B =3D3B&=3Bgt=3D3B--_5f=
>23896e-68ee-44da-82dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=
>=3D3B&=3Bgt=3D3BCon=3D<br>>=3B >=3B>=3B tent-Type: text/plain=3D3B=
> charset=3D3D"iso-8859-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B=
>gt=3D3BConte=3D<br>>=3B >=3B>=3B nt-Transfer-Encoding: quoted-printab=
>le<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a=
>mp=3Bgt=3D3B &=3Bgt=3D<br>>=3B >=3B>=3B =3D3B&=3Bgt=3D3BWe have=
> race conditions=3D3D2C like this=3D3D2C m3core/src/Text.m3:<=3Bbr=3D<br>=
>>=3B >=3B>=3B>=3B &=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3B=
>br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=
>=3D3B &=3Bgt=3D3B&=3Bgt=3D3BVAR fromC=3D<br>>=3B >=3B>=3B harCa=
>che :=3D3D3D ARRAY CHAR OF T {NIL=3D3D2C ..}=3D3D3B (* 1-char texts *)<=
>=3Bbr>=3B&=3Bg=3D<br>>=3B >=3B>=3B t=3D3B &=3Bgt=3D3B&=3Bg=
>t=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=
>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROCEDURE Fr=3D<br>>=3B >=
>=3B>=3B omChar (c: CHAR): T =3D3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=
>=3D3B&=3Bgt=3D3B VAR buf: ARRAY [0..0] OF =3D<br>>=3B >=3B>=3B CH=
>AR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B BEGIN&l=
>t=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B IF fromCh=3D<br>=
>>=3B >=3B>=3B arCache [c] =3D3D3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3=
>B &=3Bgt=3D3B&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3=3D<br>>=3B &=
>gt=3B>=3B B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B =
> fromCharCache[c] :=3D3D3D Text8.New (buf)=3D3D3B<=3B=3D<br>>=3B >=3B=
>>=3B br>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=
>=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCha=3D=
><br>>=3B >=3B>=3B rCache [c]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D=
>3B&=3Bgt=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=
>=3D3B&=3Bgt=3D3B=3D<br>>=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B =
>&=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=
>=3Bgt=3D3BIt should be:<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=3B =
>>=3B>=3B =3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B=
>&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BPROC=
>EDURE FromChar (=3D<br>>=3B >=3B>=3B c: CHAR): T =3D3D3D<=3Bbr>=
>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&am=
>p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B VA=3D<br>>=3B >=3B>=3B R bu=
>f: ARRAY [0..0] OF CHAR=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a=
>mp=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D<br>>=3B &=
>gt=3B>=3B gt=3D3B&=3Bgt=3D3B BEGIN<=3Bbr>=3B&=3Bgt=3D3B &=
>=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=
>=3D3B&=3Bgt=3D3B =3D<br>>=3B >=3B>=3B IF fromCharCache [c] =3D3D=
>3D NIL THEN<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=
>=3D3D<=3Bbr>=3B&=3Bgt=3D<br>>=3B >=3B>=3B =3D3B &=3Bgt=3D3B=
>&=3Bgt=3D3B buf [0] :=3D3D3D c=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=
>amp=3Bgt=3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3B=3D<br>>=3B >=3B>=3B br&=
>gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B WITH a =3D3D3D Text8.=
>New (buf) DO<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=3B >=3B>=
>=3B =3D3B&=3Bgt=3D3B MemoryBarrier()=3D3D3B<=3Bbr>=3B&=3Bg=
>t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCha=3D<br>>=3B >=3B>=
>=3B rCache[c] :=3D3D3D a=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=
>amp=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>&g=
>t=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D=
>3B &=3Bgt=3D3B&=3Bgt=3D3B END=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=
>amp=3Bgt=3D3B&=3Bgt=3D3B=3D<br>>=3B >=3B>=3B =3D3D0A=3D3D<=3Bbr&=
>gt=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B RETURN fromCharCache [c=
>]<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=3B >=3B>=3B =3D3B&=
>=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=
>=3D3B END FromChar=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=
>=3B >=3B>=3B =3D3B&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3=
>B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a=
>mp=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D<br>>=3B >=3B>=3B =
>gt=3D3B&=3Bgt=3D3Bto ensure that all of Text8.New() finishes before the =
>assignment=3D<br>>=3B >=3B>=3B to fromCha=3D3D<=3Bbr>=3B&=3Bgt=
>=3D3B &=3Bgt=3D3B&=3Bgt=3D3BrCache[c] is made.<=3Bbr>=3B&=3Bgt=
>=3D3B &=3Bgt=3D3B&=3Bgt=3D<br>>=3B >=3B>=3B =3D3B<=3Bbr>=3B=
>&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am=
>p=3Bgt=3D3B&=3Bgt=3D3BCan the compiler somehow c=3D<br>>=3B >=3B>=
>=3B atch these?<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BI f=
>ear they are a small epidemic.<=3Bbr>=3B&=3Bgt=3D<br>>=3B >=3B&g=
>t=3B =3D3B &=3Bgt=3D3B&=3Bgt=3D3BFor a long time people didn't realiz=
>e where all the compile=3D<br>>=3B >=3B>=3B r and processor =3D3D<=
>=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bcould reorder.<=3Bb=
>r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bg=3D<br>>=3B >=3B>=3B t=3D=
>3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&a=
>mp=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BDo we have the right cons=3D<br>&g=
>t=3B >=3B>=3B tructs by now to fix them?<=3Bbr>=3B&=3Bgt=3D3B &a=
>mp=3Bgt=3D3B&=3Bgt=3D3BI think we do.<=3Bbr>=3B&=3Bgt=3D3B &=
>=3B=3D<br>>=3B >=3B>=3B gt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=
>=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D=
>3B&=3Bgt=3D3B - Jay<=3Bbr>=3B&=3Bgt=3D3B &=3B=3D<br>>=3B >=
>=3B>=3B gt=3D3B&=3Bgt=3D3B =3D3D<=3Bbr>=3B=
>&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &am=
>p=3Bgt=3D3B&=3Bgt=3D3B--_=3D<br>>=3B >=3B>=3B 5f23896e-68ee-44da-8=
>2dd-311cd8c58979_<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BC=
>ontent-Type: t=3D<br>>=3B >=3B>=3B ext/html=3D3B charset=3D3D"iso-885=
>9-1"<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3BContent-Transf=
>er-E=3D<br>>=3B >=3B>=3B ncoding: quoted-printable<=3Bbr>=3B&=
>=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3B=
>gt=3D3B&=3Bgt=3D3B&=3Blt=3D<br>>=3B >=3B>=3B =3D3Bhtml&=3Bgt=
>=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3=
>Bhead&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3=
>B=3D<br>>=3B >=3B>=3B &=3Blt=3D3Bstyle&=3Bgt=3D3B&=3Blt=3D3B=
>!--<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B.hmmessage P<=
>=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=3B >=3B>=3B =3D3B&=3Bg=
>t=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bmargin:0px=
>=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bpadd=3D<br>=
>>=3B >=3B>=3B ing:0px<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=
>=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bbody.hm=
>message<=3Bbr>=3B&=3Bgt=3D<br>>=3B >=3B>=3B =3D3B &=3Bgt=3D=
>3B&=3Bgt=3D3B{<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bf=
>ont-size: 12pt=3D3D3B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D<br>>=3B &g=
>t=3B>=3B =3D3B&=3Bgt=3D3Bfont-family:Calibri<=3Bbr>=3B&=3Bgt=3D=
>3B &=3Bgt=3D3B&=3Bgt=3D3B}<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B=
>&=3Bgt=3D3B=3D<br>>=3B >=3B>=3B --&=3Bgt=3D3B&=3Blt=3D3B/sty=
>le&=3Bgt=3D3B&=3Blt=3D3B/head&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D=
>3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3Bbod=3D<br>>=3B >=3B>=3B =
>y class=3D3D3D'hmmessage'&=3Bgt=3D3B&=3Blt=3D3Bdiv dir=3D3D3D'ltr'&am=
>p=3Bgt=3D3BWe have race cond=3D<br>>=3B >=3B>=3B itions=3D3D2C like =
>=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bthis=3D3D2C m=
>3core/src/Text.m3:&=3Blt=3D<br>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=
>&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BVAR fromCharCac=
>he :=3D3D3D ARRAY CHAR =3D<br>>=3B >=3B>=3B OF=3D3D<=3Bbr>=3B&=
>=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B T {NIL=3D3D2C ..}=3D3D3B (* 1-char =
>texts *)&=3Blt=3D3Bb=3D<br>>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=
>=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BPROCEDURE FromChar (c: C=
>HAR=3D3D<=3Bbr>=3B&=3Bgt=3D3B=3D<br>>=3B >=3B>=3B &=3Bgt=3D=
>3B&=3Bgt=3D3B): T =3D3D3D&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnb=
>sp=3D3D3B VAR buf: ARRAY [0..0]=3D<br>>=3B >=3B>=3B OF CHAR=3D3D3B&am=
>p=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr=
>&=3Bgt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D<br>>=3B >=3B>=3B =3D3B =
>&=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=
>=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B IF fromCha=3D<br>>=3B >=3B>=3B rCac=
>he [c] =3D3D3D NIL THEN&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D=
>3D3B&=3Bamp=3D3Bnbsp=3D3D<=3Bbr>=3B&=3Bg=3D<br>>=3B >=3B>=
>=3B t=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am=
>p=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf =3D<br>>=3B >=3B&=
>gt=3B [0] :=3D3D3D c=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbs=
>p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D<br>>=3B >=3B&g=
>t=3B =3D3D3B&=3Bamp=3D3Bn=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3=
>B&=3Bgt=3D3Bbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCac=3D<br>>=
>=3B >=3B>=3B he[c] :=3D3D3D Text8.New (buf)=3D3D3B&=3Blt=3D3Bbr&=
>=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D<br>>=3B >=3B=
>>=3B =3D3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3=
>B&=3Bamp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3B=
>amp=3D<br>>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&am=
>p=3Bamp=3D3Bnbsp=3D3D3B RETURN fromCharCache [c]&=3Blt=3D3B=3D<br>>=3B=
> >=3B>=3B br&=3Bgt=3D3B&=3Bamp=3D3Bnb=3D3D<=3Bbr>=3B&=3Bgt=
>=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bsp=3D3D3B END FromChar=3D3D3B&=3Blt=3D=
>3Bb=3D<br>>=3B >=3B>=3B r&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=
>&=3Blt=3D3Bbr&=3Bgt=3D3BIt should be:&=3Blt=3D3Bbr&=3Bgt=3D3B&a=
>mp=3Blt=3D3Bbr&=3Bgt=3D<br>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=
>=3Bgt=3D3BPROCEDURE FromCha=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B=
>&=3Bgt=3D3Br (c: CHAR): T =3D<br>>=3B >=3B>=3B =3D3D3D&=3Blt=3D=
>3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&a=
>mp=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B VAR buf:=3D<br>>=3B >=3B>=3B A=
>RRAY [0..0] OF CHAR=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bb=
>r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D<br>>=3B &=
>gt=3B>=3B =3D3Bnbsp=3D3D3B BEGIN&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3=
>D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnb=
>sp=3D<br>>=3B >=3B>=3B =3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D=
>3Bnbsp=3D3D3B IF fromCharCache [c] =3D3D3D NIL THEN&=3Bl=3D<br>>=3B &g=
>t=3B>=3B t=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=
>amp=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D=
>3D3B=3D<br>>=3B >=3B>=3B &=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbs=
>p=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B buf [0] :=3D3D3D c=3D3D3B&=3Blt=3D<br=
>>>=3B >=3B>=3B =3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3B=
>gt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D=
>3Bnbsp=3D3D3B&=3B=3D<br>>=3B >=3B>=3B amp=3D3Bnbsp=3D3D3B&=3Bam=
>p=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B WITH a =3D3D3D Text8.New (buf=
>=3D<br>>=3B >=3B>=3B ) DO&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3=
>B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bnbsp=3D3D3B&=
>amp=3Bamp=3D3Bnbsp=3D3D3=3D<br>>=3B >=3B>=3B B&=3Bamp=3D3Bnbsp=3D3=
>D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=
>=3D3D3B&=3Bamp=3D3Bnbs=3D<br>>=3B >=3B>=3B p=3D3D3B MemoryBarrier(=
>)=3D3D3B&=3Blt=3D3B=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=
>=3Bgt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bn=3D<br>>=3B >=3B>=3B bsp=3D=
>3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=
>=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bam=3D<br>>=3B >=3B>=3B p=3D3B=
>nbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B fromCharCache[c=3D3D<=3Bbr>=3B&am=
>p=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B] :=3D<br>>=3B >=3B>=3B =3D3D=
>3D a=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Ba=
>mp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3B=3D<br>>=3B >=3B>=
>=3B amp=3D3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3=
>D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B&=3Bamp=3D<br>=
>>=3B >=3B>=3B =3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D=
>3Bnbsp=3D3D3B END=3D3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D=3D<br>&g=
>t=3B >=3B>=3B <=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B=
>&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3D3B&=3Bamp=3D3Bnbsp=3D3=
>D3B RET=3D<br>>=3B >=3B>=3B URN fromCharCache [c]&=3Blt=3D3Bbr&=
>=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=
>=3D3B&=3Bamp=3D3Bnb=3D<br>>=3B >=3B>=3B sp=3D3D3B END FromChar=3D3=
>D3B&=3Blt=3D3Bbr&=3Bgt=3D3B=3D3D0A=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=
>amp=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D<br>>=3B >=3B>=3B =3D3Bbr&=
>=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3Bto ensure that all of Text8.New() f=
>inishes before =3D<br>>=3B >=3B>=3B the assignment to=3D3D<=3Bbr>=
>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B fromCharCache[c] is made.&am=
>p=3Blt=3D3B=3D<br>>=3B >=3B>=3B br&=3Bgt=3D3B&=3Blt=3D3Bbr&=
>=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BCan the compiler somehow catch thes=
>e?=3D<br>>=3B >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D=
>3B&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BI fear they are a small epide=
>mic.&=3Bl=3D<br>>=3B >=3B>=3B t=3D3Bbr&=3Bgt=3D3BFor a long tim=
>e people didn't real=3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3B=
>gt=3D3Bize=3D<br>>=3B >=3B>=3B where all the compiler and processor c=
>ould reorder.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3B=3D<br>&g=
>t=3B >=3B>=3B gt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D3BDo we ha=3D3D<=3Bb=
>r>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3Bve the right construct=3D=
><br>>=3B >=3B>=3B s by now to fix them?&=3Blt=3D3Bbr&=3Bgt=3D3B=
>I think we do.&=3Blt=3D3Bbr&=3Bgt=3D3B&=3Blt=3D3Bbr&=3Bgt=3D<br=
>>>=3B >=3B>=3B =3D3B&=3Blt=3D3Bbr&=3Bgt=3D3B -=3D3D<=3Bbr>=
>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B Jay&=3Blt=3D3Bbr&=3Bgt=
>=3D3B =3D<br>>=3B >=3B>=3B &=3Blt=3D3B/div=
>&=3Bgt=3D3B&=3Blt=3D3B/body&=3Bgt=3D3B<=3Bbr>=3B&=3Bgt=3D3B=
> &=3Bgt=3D3B&=3Bgt=3D3B&=3Blt=3D3B/html&=3Bgt=3D3B=3D<br>>=3B=
> >=3B>=3B =3D3D<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3=
>B<=3Bbr>=3B&=3Bgt=3D3B &=3Bgt=3D3B&=3Bgt=3D3B--_5f23896e-68ee-=
>44da-82dd=3D<br>>=3B >=3B>=3B -311cd8c58979_--<=3Bbr>=3B<=3B/di=
>v>=3B <=3B/div>=3B<=3B/body>=3B<br>>=3B=
> >=3B>=3B <=3B/html>=3B=3D<br>>=3B >=3B>=3B <br>>=3B >=3B=
>>=3B --_86f22d85-5540-4565-bf73-232aaafdc94b_--<br></div> </di=
>v></body>
></html>=
>
>--_5c33c9ca-3a7d-4c39-ab66-ab9c151d1539_--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20120827/0f52ebd2/attachment-0002.html>
More information about the M3devel
mailing list