[M3devel] <*LAZYALIGN*>

[Darko]

I agree that pickles should not break. I'm not convinced this  
alignment change break pickles since it doesn't do anything but change  
the padding of fields in structures, which is going to change between  
platforms anyway. Or maybe pickles don't support bit fields anyway. Do  
pickles use RTTipe exclusively or does it do its own hacking too?


On 19/02/2008, at 3:52 PM, Randy Coleburn wrote:

> Sorry, but I've never checked to see how pickles represent "BITS FOR".
> I've used BITS FOR only on rare occasion.
>
> My concern about "breaking" is that if a change is made to the  
> encoding of pickles, then all other programs that use pickles would  
> need to be rebuilt in order to remain compatible.  I seem to recall  
> that there is a version number that is put into the encoding to help  
> with compatibility and of course we already have pickle, pickle2,  
> netobj, and netobj2.  Perhaps your change could be structured in  
> such a way as to still understand the older variants for  
> compatibility.
>
> Here is a scenario:  Suppose I have a program that uses pickles and/ 
> or netobj.  A version is built and put into an embedded system.   
> This system shares pickles/netobj with another client program.   
> After the embedded system is fielded, a change is made to the  
> encoding of pickles.  New features are added to the client program  
> and it is rebuilt.  Now the client program can't share pickles with  
> the embedded system.
>
> Scenario2:  A program uses pickles to store data in files.  If the  
> pickle structure is changed, the program will no longer be able to  
> read its old files without some sort of transformation function on  
> all the files.
>
> Regards,
> Randy
>
> >>> "Darko" <darko at darko.org> 2/18/2008 8:44 PM >>>
> Actually rather than compilain rather grumpily, maybe I can just  
> make the
> changes required to RTTipe and do the work to reomve the pragma. I can
> work with Randy to ensure it doesn't break pickles, there is no  
> reason why
> it should.
>
> On tha subject Randy, I assume pickles encode the number of BITS FOR
> for a field? If not it should.
>
>
> > We seem to be going through these same point several times. I'm  
> not sure
> > what the difficulty is, I'm just repeating myself.
> >
> > Keep in mind this applies *only* to packed structures, ie BIT FOR  
> or bit
> > fields.
> >
> > The change I put in libralised the *M3* alignment rules so that  
> BITS FOR
> > fields in structures would align on byte boundries if possible  
> instead of
> > restricting them to word alignment generally. GCC happily  
> generates code
> > for this. There may be restrictions in GCC's C as to how you can  
> arrange
> > bit fields but I don't see what they have to do with M3.
> >
> > This is absolutely essentail for using the native APIs on Mac OS X  
> and its
> > removal would make M3 pointless for use on the Mac, at least more  
> me.
> >
> > This should be the default behviour. If you are using BITS FOR it's
> > becuase you want to arrange the fields in a record in particular  
> way. Why
> > then arbitrarliy pad out the fields? If performance is an issue,  
> the user
> > should be using appropriate field bit sizes. The implementation  
> rule for
> > BITS FOR in M3 is implementation dependent, there are no language  
> issues.
> >
> > The LAZYALIGN pragma (that is, the prgama itself) is something Olaf
> > created and had nothing to do with me. I diagreed with it and  
> never used
> > it in the version of the compiler I ran. I support its removal.
> >
> >
> >
> >> On Feb 18, 2008, at 3:01 PM, Darko wrote:
> >>
> >>> The alignment behaviour is absolutely crucial to programming
> >>> natively in Mac OS X and should be kept. I have a great need for  
> it.
> >>>
> >>> The PRGAMA is of no use and can be removed.
> >>>
> >>> Does anyone have any objections to making this alignment behaviour
> >>> standard?
> >>
> >> What do you mean make LAZYALIGN standard?  Why wouldn't we go with
> >> the standard gcc-backend alignment?  Perhaps I don't understand  
> what
> >> it is you are doing or trying to do.  Again, please remind me  
> what it
> >> is that LAZYALIGN does and why it is needed.
> >>
> >>> On 19/02/2008, at 4:36 AM, Tony Hosking wrote:
> >>>
> >>>> Can someone remind me again why we need LAZYALIGN?  I concur with
> >>>> Randy that if it is rarely used and moreso breaks things I would
> >>>> argue to abandon it.
> >>>>
> >>>> On Feb 18, 2008, at 10:56 AM, Randy Coleburn wrote:
> >>>>
> >>>>> I use pickles extensively and they are used also by network  
> objects.
> >>>>> I've never used LAZYALIGN.
> >>>>> My vote is that we don't break something (pickles/netobj) to add
> >>>>> support for something that is rarely used.
> >>>>> Regards,
> >>>>> Randy
> >>>>>
> >>>>> >>> "Rodney M. Bates" <rodney.bates at wichita.edu> 2/15/2008 5:06
> >>>>> PM >>>
> >>>>> I'll put it in comments in the code.  I am sure I can fix it to
> >>>>> handle LAZYALIGN too,  just not sure whether I can do it without
> >>>>> requiring existing pickle files to be regenerated and/or  
> existing
> >>>>> code that reads/writes pickles to need recompilation.
> >>>>>
> >>>>> Anybody on this list who has code that might be affected if  
> pickles
> >>>>> or compiled code were invalidated by a change?
> >>>>>
> >>>>> I do propose we change the way of telling the compiler to do
> >>>>> LAZYALIGN so that it is a property of a type and affects all
> >>>>> variables of that type.
> >>>>>
> >>>>> Olaf Wagner wrote:
> >>>>> > Perhaps we should check-in this description somewhere near the
> >>>>> > actual code? Or is there enough documentation already?
> >>>>> >
> >>>>> > Olaf
> >>>>> >
> >>>>> > PS: Based on your description, I'd say we should abandon
> >>>>> LAZYALIGN.
> >>>>> >     Or at least put a big sticker on that it will break  
> pickles.
> >>>>> >
> >>>>> > Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:
> >>>>> >
> >>>>> >> The word "Packing" in RTPacking is perhaps misleading.  Using
> >>>>> BITSIZE,
> >>>>> >> etc. only works for getting object layouts as on the machine
> >>>>> executing
> >>>>> >> the code, which is all that is needed when writing a pickle.
> >>>>> >>
> >>>>> >> When reading, Pickle code needs to know the layouts of a type
> >>>>> both as
> >>>>> >> it is on the reading machine and as it was on the machine  
> that
> >>>>> wrote
> >>>>> >> the pickle.  The type description that the compiler  
> generates is
> >>>>> >> excerpted and contains no field displacements, just lists of
> >>>>> field
> >>>>> >> types (which are either recursive type descriptions or  
> builtin
> >>>>> types).
> >>>>> >> So it is independent of word sizes, etc.
> >>>>> >>
> >>>>> >> Pickles regenerates the displacements using the few target
> >>>>> machine
> >>>>> >> characteristics in a RTPacking.T  It traverses a type
> >>>>> description and
> >>>>> >> simultaneously computes two sets of field displacements, both
> >>>>> as they
> >>>>> >> are on the reading machine and on the writing machine.  For
> >>>>> the latter,
> >>>>> >> the value of RTPacking.T is (after a compact bit encoding)
> >>>>> stored in the
> >>>>> >> header of the pickle file.  For the former, it's gotten by
> >>>>> techniques
> >>>>> >> like
> >>>>> >> using BITSIZE.  This is actually all done in RTTipe, part of
> >>>>> m3core, and
> >>>>> >> called by Pickle code.
> >>>>> >>
> >>>>> >> This is very fragile.  RTTipe has to duplicate the compiler's
> >>>>> layout
> >>>>> >> behavior.  There is no shared code.  Making it common would
> >>>>> involve
> >>>>> >> quite a bit of rework, as the two use substantially different
> >>>>> data
> >>>>> >> structure and code organization.  It will be obvious what  
> kind
> >>>>> of bit
> >>>>> >> damage could occur if the two algorithms didn't agree.
> >>>>> >>
> >>>>> >> This is why I am obsessing over LAZYALIGN.  I have been
> >>>>> comparing the
> >>>>> >> field displacement computations in RTTipe and in the
> >>>>> compiler.  The
> >>>>> >> former is oblivious to LAZYALIGN.
> >>>>> >>
> >>>>> >> Note that all this is required even without any packing of
> >>>>> small fields
> >>>>> >> within words.  E.G., a record with two INTEGER fields,  
> pickled
> >>>>> on a
> >>>>> >> 32-bit machine and unpickled on a 64.
> >>>>> >>
> >>>>> >> Tony Hosking wrote:
> >>>>> >>
> >>>>> >>> Rodney,
> >>>>> >>>
> >>>>> >>> Why does there need to be an entry for LONGINT in  
> RTPacking?  I
> >>>>> >>> would  have thought all packing is done on word-sized units
> >>>>> anyway.
> >>>>> >>>  Each  side of the connection can check BITSIZE(LONGINT) to
> >>>>> figure
> >>>>> >>> out what  to do presumably no differently from the way
> >>>>> INTEGER is
> >>>>> >>> communicated  between 32-bit and 64-bit machines.  Am I  
> missing
> >>>>> >>> something?
> >>>>> >>>
> >>>>> >>> -- Tony
> >>>>> >>>
> >>>>> >>> On Feb 15, 2008, at 10:51 AM, Tony Hosking wrote:
> >>>>> >>>
> >>>>> >>>> Ah, OK.  I don't much delve in pickle-land.  Anything I can
> >>>>> help with?
> >>>>> >>>>
> >>>>> >>>> On Feb 14, 2008, at 11:02 PM, Rodney M. Bates wrote:
> >>>>> >>>>
> >>>>> >>>>> 1) RTPacking.T and needs to have a separate size for  
> LONGINT,
> >>>>> >>>>>   (which can vary independently of the size of INTEGER).
> >>>>> >>>>> 2) Variable length values of subrange bounds found in type
> >>>>> >>>>>   descriptions (in TipeDesc.i3) can now have values beyond
> >>>>> >>>>>   what the native word size can represent.
> >>>>> >>>>> 3) RTType.Kind.Longint (which I presume you, Tony, added
> >>>>> recently)
> >>>>> >>>>>   is not handled by Pickles.
> >>>>> >>>>>
> >>>>> >>>>> I have done some coding on this, but have been  
> interrupted.
> >>>>> >>>>>
> >>>>> >>>>> Tony Hosking wrote:
> >>>>> >>>>>
> >>>>> >>>>>> Why is LONGINT for pickles not yet supported?
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> --
> >>>>> >>>>>  
> -------------------------------------------------------------
> >>>>> >>>>> Rodney M. Bates, retired assistant professor
> >>>>> >>>>> Dept. of Computer Science, Wichita State University
> >>>>> >>>>> Wichita, KS 67260-0083
> >>>>> >>>>> 316-978-3922
> >>>>> >>>>> rodney.bates at wichita.edu
> >>>>> >>>
> >>>>> >>>
> >>>>> >>>
> >>>>> >>>
> >>>>> >>
> >>>>> >> --
> >>>>> >> -------------------------------------------------------------
> >>>>> >> Rodney M. Bates, retired assistant professor
> >>>>> >> Dept. of Computer Science, Wichita State University
> >>>>> >> Wichita, KS 67260-0083
> >>>>> >> 316-978-3922
> >>>>> >> rodney.bates at wichita.edu
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>>
> >>>>> --
> >>>>> -------------------------------------------------------------
> >>>>> Rodney M. Bates, retired assistant professor
> >>>>> Dept. of Computer Science, Wichita State University
> >>>>> Wichita, KS 67260-0083
> >>>>> 316-978-3922
> >>>>> rodney.bates at wichita.edu
> >>>>>
> >>>>
> >>
> >>
> >
> >
> >
>
>
>