[M3devel] platform-independent packing/alignment?
Coleburn, Randy
rcolebur at SCIRES.COM
Mon Feb 1 21:02:49 CET 2010
Right, you can't put it back in the same state; you just create a new one.
Here is an example showing a case where I had a field named "PrivateMutex" that was embedded in an object that needed to be pickled.
TYPE
PrivateMutex = MUTEX BRANDED Brand & ".PrivateMutex" OBJECT END;
(*|***********************************************************************************)
(* Pickle Procedures *)
(*|***********************************************************************************)
TYPE
Special = Pickle.Special OBJECT
OVERRIDES
write := WriteSpecial;
read := ReadSpecial;
END; (* Special *)
PROCEDURE WriteSpecial (self: Special;
<*UNUSED*>ref: REFANY;
<*UNUSED*>writer: Pickle.Writer
)
RAISES {Pickle.Error,
<*NOWARN*>Wr.Failure, Thread.Alerted} =
(* Special pickle writer for LiteConfig.PrivateMutex objects. *)
BEGIN (* WriteSpecial *)
IF self.sc = TYPECODE(PrivateMutex)
THEN
(* omit writing the mutex field *)
ELSE
RAISE Pickle.Error("LiteConfig.WriteSpecial asked to process unrecognized typecode."); (* should never happen *)
END; (* if *)
END WriteSpecial;
PROCEDURE ReadSpecial (self: Special;
reader: Pickle.Reader;
id: Pickle.RefID
): REFANY
RAISES {Pickle.Error,
<*NOWARN*>Rd.EndOfFile, Rd.Failure, Thread.Alerted} =
(* Special pickle reader for LiteConfig.PrivateMutex objects. *)
BEGIN (* ReadSpecial *)
IF self.sc = TYPECODE(PrivateMutex)
THEN
WITH m = NEW(PrivateMutex)
DO
reader.noteRef(m, id);
RETURN m;
END; (* with *)
ELSE
RAISE Pickle.Error("LiteConfig.ReadSpecial asked to process unrecognized typecode."); (* should never happen *)
END; (* if *)
END ReadSpecial;
BEGIN (* LiteConfig *)
Pickle.RegisterSpecial(NEW(Special, sc := TYPECODE(PrivateMutex)));
END LiteConfig.
From: jayk123 at hotmail.com [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Monday, February 01, 2010 2:47 PM
To: Coleburn, Randy; m3devel
Subject: RE: [M3devel] platform-independent packing/alignment?
You *might* imagine pickling a mutex, held or not, like to
"freeze"/"suspend" a running system, possibly to move to
another system, where it is "resumed".
But in reality, that won't work.
If changing alignment/padding won't break pickles, I again
suggest that "max_align" be raised to at least 64 for all architectures.
So that double and longint are aligned.
Or that max_align be thrown out entirely and have the architectures
all declare things correctly..which they likely already are.
> recreates the MUTEX in the environment of the reading program
Just with "NEW(MUTEX)" right?
No attempt to put it back in the same state, I assume.
- Jay
> From: rcolebur at SCIRES.COM
> To: m3devel at elegosoft.com
> Date: Mon, 1 Feb 2010 14:37:48 -0500
> Subject: Re: [M3devel] platform-independent packing/alignment?
>
> For MUTEX, you need to write a special pickler procedure that deals with them the way you want. For most of my code, I simply include a tag on output (writing) that is interpreted on input (reading) by my special pickler code that recreates the MUTEX in the environment of the reading program. If anyone wants the code, let me know and I'll send an example.
> Regards,
> Randy Coleburn
>
> -----Original Message-----
> From: Dragiša Durić [mailto:dragisha at m3w.org]
> Sent: Monday, February 01, 2010 3:42 AM
> To: Mika Nystrom
> Cc: m3devel at elegosoft.com
> Subject: Re: [M3devel] platform-independent packing/alignment?
>
> MUTEX is primitive type whose typecode is fixed by compiler, IIRC - that
> code being same even if changes were made to it's basic structure... Or
> not... On the second thought, MUTEX is basically fixed, but it's
> revealations can vary. Thus making it incompatible over
> pickles/versions.
>
> I've not looked at my code/data yet, but MUTEX was only an idea what
> went... It is OODBMS package, and mutex is not used for persistent data,
> only for tables/indexes. Thus, problem (at leat for me) remains and it
> is not pickled MUTEX.
>
> On Mon, 2010-02-01 at 00:19 -0800, Mika Nystrom wrote:
> > For what it's worth...
> >
> > I recall someone (Dragisa?) just complained that he was having problems
> > unpickling data structures...
> >
> > I have never had much luck pickling and unpickling MUTEXes, using a few
> > (not very different) M3 compilers. I just leave them out of my pickles.
> > It makes the pickles a lot more compatible across versions. I'm not
> > talking recent CM3 versions here but a variety of old versions.
> > Implementors have seemed to modify MUTEX quite a bit. What's the point
> > of pickling it anyhow? If it's an active mutex the unpickling will
> > surely lead to garbage, if there are wait queues and things.. then you
> > wind up un/pickling Thread structures too.
> >
> > Mika
> >
> > Tony Hosking writes:
> > >
> > >--Apple-Mail-4--1058110347
> > >Content-Transfer-Encoding: quoted-printable
> > >Content-Type: text/plain;
> > > charset=utf-8
> > >
> > >That shouldn't affect things since the pickler picks apart the =
> > >structures and ships the individual primitives. Right?
> > >
> > >On 1 Feb 2010, at 02:41, Jay K wrote:
> > >
> > >> Let's say I have:
> > >> =20
> > >> =20
> > >> TYPE FOO =3D RECORD a: LONGFLOAT; b: UINT32; END;
> > >> TYPE FOOA =3D ARRAY [0..1] OF FOO;
> > >> =20
> > >> =20
> > >> And I have a platform with max_align =3D 32 and a platform with =
> > >max_align =3D 64.
> > >> One platform will make FOOA be 24 bytes, the other 32.
> > >> One platform will pad out FOO to be align the 8 byte LONGFLOAT, one =
> > >won't.
> > >> Can that pickle be read/written by the two platforms?
> > >> And how does the code know/detect the difference?
> > >> That is, if a platform changed from max_align =3D 32 to 64, would the =
> > >pickle code work just
> > >> as well on the "same" platform as on the "different" platforms?
> > >> =20
> > >> =20
> > >> I very recently changed NT386 max_align from 32 to 64, and I believe
> > >> the remaining active platforms with max_align =3D 32 should also be =
> > >64.
> > >> It would "only" affect pickles with LONGINT or LONGFLOAT, as well
> > >> as "lining up" such structs with C.
> > >> =20
> > >> =20
> > >> Such alignment shold make us not need the extra "unaligned double" =
> > >switch
> > >> on most platforms (some mystery still on some platforms) as well as
> > >> perhaps be needed for some atomic operations, esp. maybe on
> > >> LONGINT on 32bit x86.
> > >> =20
> > >> =20
> > >> - Jay
> > >>=20
> > >> =20
> > >> > From: dragisha at m3w.org
> > >> > To: rodney_bates at lcwb.coop
> > >> > Date: Sun, 31 Jan 2010 22:14:47 +0100
> > >> > CC: m3devel at elegosoft.com
> > >> > Subject: Re: [M3devel] platform-independent packing/alignment?
> > >> >=20
> > >> > I've not changed my code, that is for sure.... But now I am not sure
> > >> > some parent type (esp MUTEX here an there) was not changed... I'll =
> > >take
> > >> > a look on this again sometime soon and report my findings.
> > >> >=20
> > >> > Thanks for clues.
> > >> >=20
> > >> > On Sun, 2010-01-31 at 14:00 -0600, Rodney M. Bates wrote:
> > >> > >=20
> > >> > > Dragi=C5=A1a Duri=C4=87 wrote:
> > >> > > > I've asked this before, but didn't catch answer:
> > >> > > >=20
> > >> > > > On Sun, 2010-01-31 at 12:21 +0000, Jay K wrote:
> > >> > > >> I suggest we can probably get by with platform-independent
> > >> > > >> packing/alignment settings.
> > >> > > >=20
> > >> > > > Some time ago I've used pickles (CM3) to save some data... My =
> > >program
> > >> > > > does not read that pickle anymore....=20
> > >> > >=20
> > >> > > And you are certain your program that tries to read still contains
> > >> > > exact structurally equivalent types to all the types in the =
> > >pickle?
> > >> > >=20
> > >> > > >=20
> > >> > > > Someone remembers moment when this incompatible changes were =
> > >made?=20
> > >> > --=20
> > >> > Dragi=C5=A1a Duri=C4=87 <dragisha at m3w.org>
> > >> >=20
> > >
> > >
> > >--Apple-Mail-4--1058110347
> > >Content-Transfer-Encoding: quoted-printable
> > >Content-Type: text/html;
> > > charset=utf-8
> > >
> > ><html><head><base href=3D"x-msg://140/"></head><body style=3D"word-wrap: =
> > >break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
> > >after-white-space; ">That shouldn't affect things since the pickler =
> > >picks apart the structures and ships the individual primitives. =
> > > Right?<div>
> > ><br><div><div>On 1 Feb 2010, at 02:41, Jay K wrote:</div><br =
> > >class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><span =
> > >class=3D"Apple-style-span" style=3D"border-collapse: separate; =
> > >font-family: Helvetica; font-size: medium; font-style: normal; =
> > >font-variant: normal; font-weight: normal; letter-spacing: normal; =
> > >line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; =
> > >white-space: normal; widows: 2; word-spacing: 0px; =
> > >-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
> > >0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
> > >auto; -webkit-text-stroke-width: 0px; "><div class=3D"hmmessage" =
> > >style=3D"font-size: 10pt; font-family: Verdana; ">Let's say I =
> > >have:<br> <br> <br>TYPE FOO =3D RECORD a: LONGFLOAT; b: =
> > >UINT32; END;<br>TYPE FOOA =3D ARRAY [0..1] OF =
> > >FOO;<br> <br> <br>And I have a platform with max_align =
> > >=3D 32 and a platform with max_align =3D 64.<br>One platform will =
> > >make FOOA be 24 bytes, the other 32.<br> One platform will =
> > >pad out FOO to be align the 8 byte LONGFLOAT, one won't.<br>Can that =
> > >pickle be read/written by the two platforms?<br>And how does the code =
> > >know/detect the difference?<br>That is, if a platform changed from =
> > >max_align =3D 32 to 64, would the pickle code work just<br>as =
> > >well on the "same" platform as on the "different" =
> > >platforms?<br> <br> <br>I very recently changed NT386 =
> > >max_align from 32 to 64, and I believe<br>the remaining active platforms =
> > >with max_align =3D 32 should also be 64.<br>It would "only" affect =
> > >pickles with LONGINT or LONGFLOAT, as well<br>as "lining up" such =
> > >structs with C.<br> <br> <br>Such alignment shold make us not =
> > >need the extra "unaligned double" switch<br>on most platforms (some =
> > >mystery still on some platforms) as well as<br>perhaps be needed for =
> > >some atomic operations, esp. maybe on<br>LONGINT on 32bit =
> > >x86.<br> <br> <br> - Jay<br><br> <br>> From:<span =
> > >class=3D"Apple-converted-space"> </span><a =
> > >href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org</a><br>> To:<span =
> > >class=3D"Apple-converted-space"> </span><a =
> > >href=3D"mailto:rodney_bates at lcwb.coop">rodney_bates at lcwb.coop</a><br>> =
> > >Date: Sun, 31 Jan 2010 22:14:47 +0100<br>> CC:<span =
> > >class=3D"Apple-converted-space"> </span><a =
> > >href=3D"mailto:m3devel at elegosoft.com">m3devel at elegosoft.com</a><br>> =
> > >Subject: Re: [M3devel] platform-independent =
> > >packing/alignment?<br>><span =
> > >class=3D"Apple-converted-space"> </span><br>> I've not changed =
> > >my code, that is for sure.... But now I am not sure<br>> some parent =
> > >type (esp MUTEX here an there) was not changed... I'll take<br>> a =
> > >look on this again sometime soon and report my findings.<br>><span =
> > >class=3D"Apple-converted-space"> </span><br>> Thanks for =
> > >clues.<br>><span class=3D"Apple-converted-space"> </span><br>> =
> > >On Sun, 2010-01-31 at 14:00 -0600, Rodney M. Bates wrote:<br>> =
> > >><span class=3D"Apple-converted-space"> </span><br>> > =
> > >Dragi=C5=A1a Duri=C4=87 wrote:<br>> > > I've asked this before, =
> > >but didn't catch answer:<br>> > ><span =
> > >class=3D"Apple-converted-space"> </span><br>> > > On Sun, =
> > >2010-01-31 at 12:21 +0000, Jay K wrote:<br>> > >> I suggest =
> > >we can probably get by with platform-independent<br>> > >> =
> > >packing/alignment settings.<br>> > ><span =
> > >class=3D"Apple-converted-space"> </span><br>> > > Some =
> > >time ago I've used pickles (CM3) to save some data... My program<br>> =
> > >> > does not read that pickle anymore....<span =
> > >class=3D"Apple-converted-space"> </span><br>> ><span =
> > >class=3D"Apple-converted-space"> </span><br>> > And you are =
> > >certain your program that tries to read still contains<br>> > =
> > >exact structurally equivalent types to all the types in the =
> > >pickle?<br>> ><span =
> > >class=3D"Apple-converted-space"> </span><br>> > ><span =
> > >class=3D"Apple-converted-space"> </span><br>> > > Someone =
> > >remembers moment when this incompatible changes were made?<span =
> > >class=3D"Apple-converted-space"> </span><br>> --<span =
> > >class=3D"Apple-converted-space"> </span><br>> Dragi=C5=A1a =
> > >Duri=C4=87 <<a =
> > >href=3D"mailto:dragisha at m3w.org">dragisha at m3w.org</a>><br>><span =
> > >class=3D"Apple-converted-space"> </span><br></div></span></blockquote=
> > >></div><br></div></body></html>=
> > >
> > >--Apple-Mail-4--1058110347--
> --
> Dragiša Durić <dragisha at m3w.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100201/74d2d953/attachment-0002.html>
More information about the M3devel
mailing list