[M3devel] Question about TEXTs under CM3

Tony Hosking hosking at cs.purdue.edu
Fri Jul 27 19:23:43 CEST 2007 

On Jul 27, 2007, at 12:40 PM, Tony Hosking wrote:

>
> On Jul 27, 2007, at 11:04 AM, Mika Nystrom wrote:
>
>> Hi Tony,
>>
>> Sorry it took me a day to check up on this.  I lost access to my
>> 10.4 system and got my 10.3 back from Apple.
>>
>> You are right, your fix solved the pickle problem.  Thank you very  
>> much,
>> as always!  Yes I realized it was doing something very tricky when I
>> looked at the code...
>>
>> I also tested all my various software, and some of the CM3  
>> programs, on
>> 10.3.  I can report the following:
>>
>> 1. The crash in Juno's virtual machine is gone
>> 2. The threads taking 100% no longer do (I think this was the same  
>> bug that
>>    caused.. deadlock was it?.. on 10.4)
>> 3. And of course the pickling is working great.
>
> Great!
>
>> But...
>>
>> 4. Bresenham still locks up for me.  It does this, more often than  
>> not,
>> on both 10.3 and 10.4.  I'm really surprised it never does it for  
>> you!
>> (Although it is true it sometimes work.)
>
> Yes, this problem seems to manifest only on Darwin, and I am seeing  
> the issue with Bresenham as you do.  I think it is a VBT issue that  
> I have isolated to the following small program, which works fine  
> for me on Linux, but which has always caused problems on Darwin  
> locking up and not responding (I strongly suspect the Darwin X  
> implementation rather than the threads system, but need still to  
> diagnose the issue):
>
> MODULE Main;
>
> IMPORT ButtonVBT, HighlightVBT, VBT, TextVBT, Filter, Text,  
> Trestle, BorderedVBT,
>        Compiler, RTIO;
>
> PROCEDURE Flip(b: ButtonVBT.T; <*UNUSED*> READONLY cd:  
> VBT.MouseRec) RAISES {}=
>   BEGIN
>     WITH ch = Filter.Child(b), txt = TextVBT.Get(ch) DO
>       IF Text.Equal(txt, "Alpha") THEN
>         TextVBT.Put(ch, "Beta")
>       ELSE
>         TextVBT.Put(ch, "Alpha")
>       END
>     END
>   END Flip;
>
> VAR
>   button := ButtonVBT.New(TextVBT.New("Alpha"), Flip);
>   root := BorderedVBT.New(button);
>
> BEGIN
>   RTIO.PutText(Compiler.ThisFile()); RTIO.PutChar('\n');
>   RTIO.PutText(Compiler.ThisPath()); RTIO.PutChar('\n');
>   RTIO.PutInt(Compiler.ThisLine()); RTIO.PutChar('\n');
>   RTIO.Flush();
>   Trestle.Install(root);
>   Trestle.AwaitDelete(root);
> END Main.

The fix for this program is to set Xquartz defaults as follows (see  
the manual to see how to do this):

defaults write com.apple.x11 wm_click_through -bool true

I have still not diagnosed the issue with mentor running Bresenham.

>
>
>> 5. I am still using @M3noincremental to run ktok, but it's the *only*
>> program that seems to require it.
>
> I would like to figure this one out soon, as it really is a worry  
> to me -- I am concerned that the new incremental GC support be as  
> robust as possible and this is the only outstanding issue with it  
> of which I am aware.
>
>>
>> Almost everything looks really good!
>>
>>     Mika
>>
>> Tony Hosking writes:
>>> I'm about to check in the fix.  I have tested this with your example
>>> program and things work fine.
>>>
>>> On Jul 26, 2007, at 11:26 AM, Tony Hosking wrote:
>>>
>>>> On closer inspection, it is even messier than that.  In the old PM3
>>>> the text that is constructed is allocated in UNTRACED storage
>>>> (M3TextWithHeader is an UNTRACED REF RECORD...) so it can't be
>>>> GC'd, and looks (to the GC) like an (old-style) text literal
>>>> allocated outside the heap.  It is harder for us to fake up a text
>>>> literal like this with the new TEXT setup since we'd need to
>>>> allocate, copy, and loophole something that looks like a text
>>>> literal (an object) in the untraced heap.  The best alternative is
>>>> to fix the pickler with a special that handles Text8Cstrings.  I
>>>> think this is the cleanest approach.  Mika, please try the
>>>> following program, which adds a special for Text8CString.   I will
>>>> put this code into the builtin specials for pickles.
>>>>
>>>> On Jul 26, 2007, at 9:47 AM, Tony Hosking wrote:
>>>>
>>>>> Looks like we need to fix M3toC.StoT so that it works the same as
>>>>> old PM3.  The old code works because it constructs a text *in* the
>>>>> heap as an ARRAY OF CHAR, that just happens to have its payload
>>>>> (the array contents) outside the heap.  (Arrays in M3 contain a
>>>>> reference to their data).  We can play the same trick for
>>>>> Text8CString and get things to work properly for you.  I will make
>>>>> this fix and check it in.
>>>>>
>>>>>
>>>>> On Jul 26, 2007, at 8:40 AM, Mika Nystrom wrote:
>>>>>
>>>>>> Ok, I am about to answer my own email.  Here's a little program I
>>>>>> wrote:
>>>>>>
>>>>>> MODULE Main;
>>>>>> IMPORT Pickle, IO, Params, TextWr, TextRd;
>>>>>>
>>>>>> VAR wr := NEW(TextWr.T).init();
>>>>>>     toPickle := "pickle this";
>>>>>> BEGIN
>>>>>>   IF Params.Count > 1 THEN toPickle := Params.Get(1) END;
>>>>>>
>>>>>>   Pickle.Write(wr,toPickle);
>>>>>>
>>>>>>   IO.Put("pickled \""&toPickle&"\"\n");
>>>>>>   IO.Put("read back \""&
>>>>>>          Pickle.Read(NEW(TextRd.T).init(TextWr.ToText(wr)))
>>>>>>          &"\"\n");
>>>>>> END Main.
>>>>>>
>>>>>> === On FreeBSD4 with my ancient, creaky PM3:
>>>>>>
>>>>>> (64)trs80:~/ptest/src>../FreeBSD4/pickleit
>>>>>> pickled "pickle this"
>>>>>> read back "pickle this"
>>>>>> (65)trs80:~/ptest/src>../FreeBSD4/pickleit pickle\ that
>>>>>> pickled "pickle that"
>>>>>> read back "pickle that"
>>>>>>
>>>>>> === On PPC_DARWIN with the latest CM3:
>>>>>>
>>>>>> [QT:~/ptest/src] mika% ../PPC_DARWIN/pickleit
>>>>>> pickled "pickle this"
>>>>>> read back "pickle this"
>>>>>> [QT:~/ptest/src] mika% ../PPC_DARWIN/pickleit pickle\ that
>>>>>> pickled "pickle that"
>>>>>>
>>>>>>
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    Segmentation violation - possible attempt to  
>>>>>> dereference NIL
>>>>>> ***
>>>>>>
>>>>>> Abort
>>>>>>
>>>>>> === On FreeBSD4 with the latest CM3:
>>>>>>
>>>>>> (73)rover:~/ptest/src>../FreeBSD4/pickleit
>>>>>> pickled "pickle this"
>>>>>> read back "pickle this"
>>>>>> (74)rover:~/ptest/src>../FreeBSD4/pickleit pickle\ that
>>>>>> pickled "pickle that"
>>>>>>
>>>>>>
>>>>>> ***
>>>>>> *** runtime error:
>>>>>> ***    Segmentation violation - possible attempt to  
>>>>>> dereference NIL
>>>>>> ***
>>>>>>
>>>>>> Abort
>>>>>>
>>>>>> ============
>>>>>>
>>>>>> Diagnosis:
>>>>>>
>>>>>> The code I mentioned in a previous email declares a  
>>>>>> Text8CString.T
>>>>>> to be of type TEXT OBJECT str: Ctypes.char_star; END .  Elsewhere
>>>>>> in the system, Ctypes.char_star is specifically declared to be an
>>>>>> "UNTRACED REF Ctypes.char".  According to the specification of
>>>>>> Pickle, an UNTRACED REF is pickled as NIL.
>>>>>>
>>>>>> Generally speaking, you don't see many Text8CString.Ts in the
>>>>>> system.
>>>>>> This one comes in via Params.Get, which in turn calls  
>>>>>> RTArgs.GetArg,
>>>>>> which in turn calls M3toC.StoT.
>>>>>>
>>>>>> StoT is generally not something you want to call, but it's  
>>>>>> supposed
>>>>>> to be OK here because you're just passing around argv, which  
>>>>>> won't
>>>>>> change.  Generally speaking, it ought to be OK to use to convert
>>>>>> C strings whose addresses aren't going to change during the  
>>>>>> lifetime
>>>>>> of the program.
>>>>>>
>>>>>> I think it is *totally unacceptable* that Params.Get returns a
>>>>>> "poisoned" TEXT.  There is absolutely nothing in any of the
>>>>>> interfaces
>>>>>> that warns of this fact, and it is surprising to say the least.
>>>>>> There is also no simple way of "copying" TEXTs, as it shouldn't
>>>>>> ever be required!  Finally, there's no way of checking whether a
>>>>>> TEXT is a Text8CString.T without importing Text8CString, which is
>>>>>> an UNSAFE INTERFACE, which of course is illegal in a safe  
>>>>>> MODULE!!
>>>>>>
>>>>>> What does baffle me a bit is that the code works on the old PM3.
>>>>>> It uses the old TextF, which declares
>>>>>>
>>>>>>   TEXT = BRANDED Text.Brand REF ARRAY OF CHAR;
>>>>>>
>>>>>> The old (from PM3, I think it's 1.1.15) M3toC has:
>>>>>>
>>>>>> PROCEDURE StoT (s: Ctypes.char_star): TEXT =
>>>>>>   VAR t := NEW (M3TextWithHeader);
>>>>>>   BEGIN
>>>>>>     t.header.typecode := RT0.TextTypecode;
>>>>>>     t.body.chars      := LOOPHOLE (s, ADDRESS);
>>>>>>     t.body.length     := 1 + Cstring.strlen (s);
>>>>>>     RETURN LOOPHOLE (ADR (t.body), TEXT);
>>>>>>   END StoT;
>>>>>>
>>>>>> I'm not entirely sure why the old code works---why isn't the
>>>>>> M3TextWithHeader garbage-collected?   The pickler doesn't seem to
>>>>>> know that the result of LOOPHOLE (ADR (t.body), TEXT) is special.
>>>>>> In fact the pickler doesn't seem to know anything about TEXTs
>>>>>> at all.
>>>>>>
>>>>>> I see several possible ways of solving the problem.  One is to
>>>>>> permit M3toC.StoT to remain broken (since M3toC is an UNSAFE
>>>>>> interface, there's no "legal" reason not to do that) and make  
>>>>>> sure
>>>>>> Params.Get (and everything else that could remotely be called
>>>>>> "standard") doesn't use M3toC.StoT---oh and to leave some very
>>>>>> prominent warning signs, both in M3toC and Text8CString that  
>>>>>> "here
>>>>>> be demons".  Another is to revert to the SRC method of  
>>>>>> LOOPHOLEing
>>>>>> the C strings.  I never liked the CM3 TEXTs, and I like them even
>>>>>> less now; I disliked them from the start because they broke  
>>>>>> pickle
>>>>>> compatibility with SRC M3, and now I find that they aren't even
>>>>>> compatible with themselves.
>>>>>>
>>>>>> Modula-3's strengths have always been its utter simplicity
>>>>>> and bullet-proof reliability.  This stuff, where some objects are
>>>>>> more "serializable" than others, reminds me of Java!
>>>>>>
>>>>>> Does anyone have an opinion as to how this problem ought to be
>>>>>> solved?
>>>>>>
>>>>>>      Mika
>>>>>>
>>>>>> Mika Nystrom writes:
>>>>>>> Hello everyone,
>>>>>>>
>>>>>>> Here's a random question that hopefully someone can answer...
>>>>>>>
>>>>>>> Is it true that certain TEXTs can't be Pickled under CM3?
>>>>>>>
>>>>>>> Unless there's some magic I don't understand going on, I am
>>>>>>> thinking
>>>>>>> that would be the implication of:
>>>>>>>
>>>>>>> UNSAFE INTERFACE Text8CString;
>>>>>>> ...
>>>>>>> TYPE
>>>>>>>  T <: Public;
>>>>>>>  Public = TEXT OBJECT
>>>>>>>    str: Ctypes.char_star;
>>>>>>>  END;
>>>>>>>  (* The array contains the characters of the text followed by a
>>>>>>> '\000'. *)
>>>>>>> ...
>>>>>>>
>>>>>>> I hope someone can set me straight and tell me that no, the
>>>>>>> situation
>>>>>>> isn't as dire as that---some clever thing somewhere makes it
>>>>>>> possible
>>>>>>> to Pickle all objects of type TEXT...
>>>>>>>
>>>>>>>     Mika

More information about the M3devel mailing list