[M3devel] Question about TEXTs under CM3

[Mika Nystrom]

Ok, I am about to answer my own email.  Here's a little program I wrote:

MODULE Main;
IMPORT Pickle, IO, Params, TextWr, TextRd;

VAR wr := NEW(TextWr.T).init();
    toPickle := "pickle this";
BEGIN
  IF Params.Count > 1 THEN toPickle := Params.Get(1) END;

  Pickle.Write(wr,toPickle);

  IO.Put("pickled \""&toPickle&"\"\n");
  IO.Put("read back \""& 
         Pickle.Read(NEW(TextRd.T).init(TextWr.ToText(wr)))
         &"\"\n");
END Main.

=== On FreeBSD4 with my ancient, creaky PM3:

(64)trs80:~/ptest/src>../FreeBSD4/pickleit 
pickled "pickle this"
read back "pickle this"
(65)trs80:~/ptest/src>../FreeBSD4/pickleit pickle\ that
pickled "pickle that"
read back "pickle that"

=== On PPC_DARWIN with the latest CM3:

[QT:~/ptest/src] mika% ../PPC_DARWIN/pickleit
pickled "pickle this"
read back "pickle this"
[QT:~/ptest/src] mika% ../PPC_DARWIN/pickleit pickle\ that
pickled "pickle that"


***
*** runtime error:
***    Segmentation violation - possible attempt to dereference NIL
***

Abort

=== On FreeBSD4 with the latest CM3:

(73)rover:~/ptest/src>../FreeBSD4/pickleit 
pickled "pickle this"
read back "pickle this"
(74)rover:~/ptest/src>../FreeBSD4/pickleit pickle\ that
pickled "pickle that"


***
*** runtime error:
***    Segmentation violation - possible attempt to dereference NIL
***

Abort

============

Diagnosis:

The code I mentioned in a previous email declares a Text8CString.T
to be of type TEXT OBJECT str: Ctypes.char_star; END .  Elsewhere
in the system, Ctypes.char_star is specifically declared to be an
"UNTRACED REF Ctypes.char".  According to the specification of
Pickle, an UNTRACED REF is pickled as NIL.

Generally speaking, you don't see many Text8CString.Ts in the system.
This one comes in via Params.Get, which in turn calls RTArgs.GetArg,
which in turn calls M3toC.StoT.

StoT is generally not something you want to call, but it's supposed
to be OK here because you're just passing around argv, which won't
change.  Generally speaking, it ought to be OK to use to convert
C strings whose addresses aren't going to change during the lifetime
of the program.

I think it is *totally unacceptable* that Params.Get returns a
"poisoned" TEXT.  There is absolutely nothing in any of the interfaces
that warns of this fact, and it is surprising to say the least.
There is also no simple way of "copying" TEXTs, as it shouldn't
ever be required!  Finally, there's no way of checking whether a
TEXT is a Text8CString.T without importing Text8CString, which is
an UNSAFE INTERFACE, which of course is illegal in a safe MODULE!!

What does baffle me a bit is that the code works on the old PM3.
It uses the old TextF, which declares

  TEXT = BRANDED Text.Brand REF ARRAY OF CHAR;

The old (from PM3, I think it's 1.1.15) M3toC has:

PROCEDURE StoT (s: Ctypes.char_star): TEXT =
  VAR t := NEW (M3TextWithHeader);
  BEGIN
    t.header.typecode := RT0.TextTypecode;
    t.body.chars      := LOOPHOLE (s, ADDRESS);
    t.body.length     := 1 + Cstring.strlen (s);
    RETURN LOOPHOLE (ADR (t.body), TEXT);
  END StoT;

I'm not entirely sure why the old code works---why isn't the
M3TextWithHeader garbage-collected?   The pickler doesn't seem to
know that the result of LOOPHOLE (ADR (t.body), TEXT) is special.
In fact the pickler doesn't seem to know anything about TEXTs
at all.

I see several possible ways of solving the problem.  One is to
permit M3toC.StoT to remain broken (since M3toC is an UNSAFE
interface, there's no "legal" reason not to do that) and make sure
Params.Get (and everything else that could remotely be called
"standard") doesn't use M3toC.StoT---oh and to leave some very
prominent warning signs, both in M3toC and Text8CString that "here
be demons".  Another is to revert to the SRC method of LOOPHOLEing
the C strings.  I never liked the CM3 TEXTs, and I like them even
less now; I disliked them from the start because they broke pickle
compatibility with SRC M3, and now I find that they aren't even
compatible with themselves.

Modula-3's strengths have always been its utter simplicity
and bullet-proof reliability.  This stuff, where some objects are
more "serializable" than others, reminds me of Java!

Does anyone have an opinion as to how this problem ought to be
solved?

     Mika

Mika Nystrom writes:
>Hello everyone,
>
>Here's a random question that hopefully someone can answer...
>
>Is it true that certain TEXTs can't be Pickled under CM3?
>
>Unless there's some magic I don't understand going on, I am thinking
>that would be the implication of:
>
>UNSAFE INTERFACE Text8CString;
>...
>TYPE
>  T <: Public;
>  Public = TEXT OBJECT
>    str: Ctypes.char_star;
>  END;
>  (* The array contains the characters of the text followed by a '\000'. *)
>...
>
>I hope someone can set me straight and tell me that no, the situation
>isn't as dire as that---some clever thing somewhere makes it possible
>to Pickle all objects of type TEXT... 
>
>     Mika