[M3devel] TextLiteral/MaxBytes 32bit/64bit pickling

[Rodney M. Bates]

On 08/04/2013 10:12 PM, Jay K wrote:
> Ideally I could have a TextLiteral on a 64bit machine larger than 2GB or 4GB.
> I would not be able to unpickle it on a 32bit machine and that would be ok.
>
>
> There a long standing problem though anyway in the frontend where INTEGER
> is used in place of TInt.T or LONGINT, giving 32bit limits to 64bit targets.

I presume this, plus the fact that the compiler lays everything out in _bits_
is why the DIV BITSIZE(Byte)?

>
>
> I put in a hack actually to change MaxBytes to be a 32bit limit, to fix
> cross compiling from 32bit to 64bit. Maybe this is something different though.
>

I have done the same thing in my local directories, but for a different reason.
It gives a consistent fingerprint to TextLiteral.T, so it can cross-pickled, so
to speak, 32<->64.

I also located the two fingerprints for the two sizes and am now mapping
between them in Pickles, so that old pickles, generated on either machine, can
be read.

But this suggests another strategy.  In the case of TextLiteral.T, the static
size of the array is neither correct nor depended on.  So we could just hard-code
one of the fingerprints (probably the 32-bit one) in the compiler and always
emit it rather than computing it from the type.  That would keep pickles
happy and allow the actual declared size to be adjusted to suit whatever other
criteria mattered.

>
> I realize that is quite a TextLiteral but I think, generally, if it fits in target address space, it should work.
> Heck, we should think about "persistant disk based" data structures that don't even fit in memory.
> e.g. file systems, movies, etc.
>
>
>   - Jay
>
>
>  > Date: Sat, 3 Aug 2013 00:25:54 +0000
>  > To: m3commit at elegosoft.com
>  > From: rodney at elego.de
>  > Subject: [M3commit] CVS Update: cm3
>  >
>  > CVSROOT: /usr/cvs
>  > Changes by: rodney at birch. 13/08/03 00:25:54
>  >
>  > Modified files:
>  > cm3/m3-libs/libm3/src/pickle/ver2/: Tag: devel_unicode
>  > Pickle2.m3
>  >
>  > Log message:
>  > An almost solution to the problem of TextLiteral.T's having MaxBytes
>  > dependent on word size, making pickling of TextLiteral.T fail across
>  > different word sizes.
>  >
>  > Hard code the fingerprint for the 64-bit case and translate it to
>  > the one for 32-bit systems, in pickles. Also vice-versa.
>  >
>  > The best long-term solution also involves making TextLiteral.MaxBytes
>  > word-size-independent. This could invalidate some preexisting pickles,
>  > but only if they contain heap objects with a field or element with
>  > declared static type TextLiteral.T, which seems quite unlikely. If
>  > such fields/methods are declared as TEXT, the actual allocated type of
>  > their referents can be TextLiteral.T and all will work.
>  >