[M3devel] <*LAZYALIGN*>
Mika Nystrom
mika at async.caltech.edu
Fri Feb 22 22:20:52 CET 2008
Hmm, can't you always write code to force whatever bit pattern you
like, if you absolutely insist on it? One could read the specification
that way too.
Mika
Darko writes:
>I think you may be reading that definition too literally. Any
>processor is going to data alignment rules so 'adjacent' is subject to
>what can be implemented on the processor. The problem I have is that
>the existing code doesn't pack to the limits of the processor, but to
>some arbitrary alignment. Thus my comment about viewing so-called lazy
>alignment as a fix to an alignment bug.
>
>
>On 23/02/2008, at 2:35 AM, Rodney M. Bates wrote:
>
>>
>>
>> Darko wrote:
>>> There wouldn't be any explicit expression. in the source, but
>>> there would be a flag in Target.i3 for turning it off and on for
>>> different platforms.
>>> Although the details may be a bit off, the principal is this:
>>> before byte alignment change:
>>> RECORD
>>> a: BITS 8 FOR 0..255; (* takes 32 bits *)
>>> b: BITS 32 FOR INTEGER; (* takes 32 bits *)
>>
>> ^If the compiler's alignment rules are not lazy, then this violates
>> a language rule and should not compile. From 2.2.5, Packed types:
>>
>> "...variables of type T that occur in records, objects, or arrays will
>> occupy exactly n bits and be packed adjacent to the preceding field or
>> element."
>>
>> To get the padding between a and b, you would have to either put it in
>> with an explicit pad field, or remove the BITS 32 from the type of b.
>>
>>> c: BITS 16 FOR 0..65000; (* takes 32 bits *)
>>> END;
>>> after byte alignment change:
>>> RECORD
>>> a: BITS 8 FOR 0..255; (* takes 8 bits *)
>>> b: BITS 32 FOR INTEGER; (* takes 32 bits *)
>>> c: BITS 16 FOR 0..65000; (* takes 16 bits *)
>>> END;
>>> On 19/02/2008, at 3:40 PM, Tony Hosking wrote:
>>>> Thanks for the explanation! How does one express the alignment
>>>> you support?
>>>>
>>>> It does sound like <*LAZYALIGN*> can go away.
>>>>
>>>> On Feb 18, 2008, at 8:44 PM, Darko wrote:
>>>>
>>>>> Actually rather than compilain rather grumpily, maybe I can just
>>>>> make the
>>>>> changes required to RTTipe and do the work to reomve the pragma.
>>>>> I can
>>>>> work with Randy to ensure it doesn't break pickles, there is no
>>>>> reason why
>>>>> it should.
>>>>>
>>>>> On tha subject Randy, I assume pickles encode the number of BITS
>>>>> FOR
>>>>> for a field? If not it should.
>>>>>
>>>>>
>>>>>> We seem to be going through these same point several times. I'm
>>>>>> not sure
>>>>>> what the difficulty is, I'm just repeating myself.
>>>>>>
>>>>>> Keep in mind this applies *only* to packed structures, ie BIT
>>>>>> FOR or bit
>>>>>> fields.
>>>>>>
>>>>>> The change I put in libralised the *M3* alignment rules so that
>>>>>> BITS FOR
>>>>>> fields in structures would align on byte boundries if possible
>>>>>> instead of
>>>>>> restricting them to word alignment generally. GCC happily
>>>>>> generates code
>>>>>> for this. There may be restrictions in GCC's C as to how you
>>>>>> can arrange
>>>>>> bit fields but I don't see what they have to do with M3.
>>>>>>
>>>>>> This is absolutely essentail for using the native APIs on Mac OS
>>>>>> X and its
>>>>>> removal would make M3 pointless for use on the Mac, at least
>>>>>> more me.
>>>>>>
>>>>>> This should be the default behviour. If you are using BITS FOR
>>>>>> it's
>>>>>> becuase you want to arrange the fields in a record in
>>>>>> particular way. Why
>>>>>> then arbitrarliy pad out the fields? If performance is an
>>>>>> issue, the user
>>>>>> should be using appropriate field bit sizes. The implementation
>>>>>> rule for
>>>>>> BITS FOR in M3 is implementation dependent, there are no
>>>>>> language issues.
>>>>>>
>>>>>> The LAZYALIGN pragma (that is, the prgama itself) is something
>>>>>> Olaf
>>>>>> created and had nothing to do with me. I diagreed with it and
>>>>>> never used
>>>>>> it in the version of the compiler I ran. I support its removal.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Feb 18, 2008, at 3:01 PM, Darko wrote:
>>>>>>>
>>>>>>>> The alignment behaviour is absolutely crucial to programming
>>>>>>>> natively in Mac OS X and should be kept. I have a great need
>>>>>>>> for it.
>>>>>>>>
>>>>>>>> The PRGAMA is of no use and can be removed.
>>>>>>>>
>>>>>>>> Does anyone have any objections to making this alignment
>>>>>>>> behaviour
>>>>>>>> standard?
>>>>>>>
>>>>>>>
>>>>>>> What do you mean make LAZYALIGN standard? Why wouldn't we go
>>>>>>> with
>>>>>>> the standard gcc-backend alignment? Perhaps I don't
>>>>>>> understand what
>>>>>>> it is you are doing or trying to do. Again, please remind me
>>>>>>> what it
>>>>>>> is that LAZYALIGN does and why it is needed.
>>>>>>>
>>>>>>>> On 19/02/2008, at 4:36 AM, Tony Hosking wrote:
>>>>>>>>
>>>>>>>>> Can someone remind me again why we need LAZYALIGN? I concur
>>>>>>>>> with
>>>>>>>>> Randy that if it is rarely used and moreso breaks things I
>>>>>>>>> would
>>>>>>>>> argue to abandon it.
>>>>>>>>>
>>>>>>>>> On Feb 18, 2008, at 10:56 AM, Randy Coleburn wrote:
>>>>>>>>>
>>>>>>>>>> I use pickles extensively and they are used also by network
>>>>>>>>>> objects.
>>>>>>>>>> I've never used LAZYALIGN.
>>>>>>>>>> My vote is that we don't break something (pickles/netobj) to
>>>>>>>>>> add
>>>>>>>>>> support for something that is rarely used.
>>>>>>>>>> Regards,
>>>>>>>>>> Randy
>>>>>>>>>>
>>>>>>>>>>>>> "Rodney M. Bates" <rodney.bates at wichita.edu> 2/15/2008 5:06
>>>>>>>>>>
>>>>>>>>>> PM >>>
>>>>>>>>>> I'll put it in comments in the code. I am sure I can fix it
>>>>>>>>>> to
>>>>>>>>>> handle LAZYALIGN too, just not sure whether I can do it
>>>>>>>>>> without
>>>>>>>>>> requiring existing pickle files to be regenerated and/or
>>>>>>>>>> existing
>>>>>>>>>> code that reads/writes pickles to need recompilation.
>>>>>>>>>>
>>>>>>>>>> Anybody on this list who has code that might be affected if
>>>>>>>>>> pickles
>>>>>>>>> or compiled code were invalidated by a change?
>>>>>>>>>>
>>>>>>>>>> I do propose we change the way of telling the compiler to do
>>>>>>>>>> LAZYALIGN so that it is a property of a type and affects all
>>>>>>>>>> variables of that type.
>>>>>>>>>>
>>>>>>>>>> Olaf Wagner wrote:
>>>>>>>>>>
>>>>>>>>>>> Perhaps we should check-in this description somewhere near
>>>>>>>>>>> the
>>>>>>>>>>> actual code? Or is there enough documentation already?
>>>>>>>>>>>
>>>>>>>>>>> Olaf
>>>>>>>>>>>
>>>>>>>>>>> PS: Based on your description, I'd say we should abandon
>>>>>>>>>>
>>>>>>>>>> LAZYALIGN.
>>>>>>>>>>
>>>>>>>>>>> Or at least put a big sticker on that it will break
>>>>>>>>>>> pickles.
>>>>>>>>>>>
>>>>>>>>>>> Quoting "Rodney M. Bates" <rodney.bates at wichita.edu>:
>>>>>>>>>>>
>>>>>>>>>>>> The word "Packing" in RTPacking is perhaps misleading.
>>>>>>>>>>>> Using
>>>>>>>>>>
>>>>>>>>>> BITSIZE,
>>>>>>>>>>
>>>>>>>>>>>> etc. only works for getting object layouts as on the machine
>>>>>>>>>>
>>>>>>>>>> executing
>>>>>>>>>>
>>>>>>>>>>>> the code, which is all that is needed when writing a pickle.
>>>>>>>>>>>>
>>>>>>>>>>>> When reading, Pickle code needs to know the layouts of a
>>>>>>>>>>>> type
>>>>>>>>>>
>>>>>>>>>> both as
>>>>>>>>>>
>>>>>>>>>>>> it is on the reading machine and as it was on the machine
>>>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>> wrote
>>>>>>>>>>
>>>>>>>>>>>> the pickle. The type description that the compiler
>>>>>>>>>>>> generates is
>>>>>>>>>>>> excerpted and contains no field displacements, just lists of
>>>>>>>>>>
>>>>>>>>>> field
>>>>>>>>>>
>>>>>>>>>>>> types (which are either recursive type descriptions or
>>>>>>>>>>>> builtin
>>>>>>>>>>
>>>>>>>>>> types).
>>>>>>>>>>
>>>>>>>>>>>> So it is independent of word sizes, etc.
>>>>>>>>>>>>
>>>>>>>>>>>> Pickles regenerates the displacements using the few target
>>>>>>>>>>
>>>>>>>>>> machine
>>>>>>>>>>
>>>>>>>>>>>> characteristics in a RTPacking.T It traverses a type
>>>>>>>>>>
>>>>>>>>>> description and
>>>>>>>>>>
>>>>>>>>>>>> simultaneously computes two sets of field displacements,
>>>>>>>>>>>> both
>>>>>>>>>>
>>>>>>>>>> as they
>>>>>>>>>>
>>>>>>>>>>>> are on the reading machine and on the writing machine. For
>>>>>>>>>>
>>>>>>>>>> the latter,
>>>>>>>>>>
>>>>>>>>>>>> the value of RTPacking.T is (after a compact bit encoding)
>>>>>>>>>>
>>>>>>>>>> stored in the
>>>>>>>>>>
>>>>>>>>>>>> header of the pickle file. For the former, it's gotten by
>>>>>>>>>>
>>>>>>>>>> techniques
>>>>>>>>>>
>>>>>>>>>>>> like
>>>>>>>>>>>> using BITSIZE. This is actually all done in RTTipe, part of
>>>>>>>>>>
>>>>>>>>>> m3core, and
>>>>>>>>>>
>>>>>>>>>>>> called by Pickle code.
>>>>>>>>>>>>
>>>>>>>>>>>> This is very fragile. RTTipe has to duplicate the
>>>>>>>>>>>> compiler's
>>>>>>>>>>
>>>>>>>>>> layout
>>>>>>>>>>
>>>>>>>>>>>> behavior. There is no shared code. Making it common would
>>>>>>>>>>
>>>>>>>>>> involve
>>>>>>>>>>
>>>>>>>>>>>> quite a bit of rework, as the two use substantially
>>>>>>>>>>>> different
>>>>>>>>>>
>>>>>>>>>> data
>>>>>>>>>>
>>>>>>>>>>>> structure and code organization. It will be obvious what
>>>>>>>>>>>> kind
>>>>>>>>>>
>>>>>>>>>> of bit
>>>>>>>>>>
>>>>>>>>>>>> damage could occur if the two algorithms didn't agree.
>>>>>>>>>>>>
>>>>>>>>>>>> This is why I am obsessing over LAZYALIGN. I have been
>>>>>>>>>>
>>>>>>>>>> comparing the
>>>>>>>>>>
>>>>>>>>>>>> field displacement computations in RTTipe and in the
>>>>>>>>>>
>>>>>>>>>> compiler. The
>>>>>>>>>>
>>>>>>>>>>>> former is oblivious to LAZYALIGN.
>>>>>>>>>>>>
>>>>>>>>>>>> Note that all this is required even without any packing of
>>>>>>>>>>
>>>>>>>>>> small fields
>>>>>>>>>>
>>>>>>>>>>>> within words. E.G., a record with two INTEGER fields,
>>>>>>>>>>>> pickled
>>>>>>>>>>
>>>>>>>>>> on a
>>>>>>>>>>
>>>>>>>>>>>> 32-bit machine and unpickled on a 64.
>>>>>>>>>>>>
>>>>>>>>>>>> Tony Hosking wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Rodney,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Why does there need to be an entry for LONGINT in
>>>>>>>>>>>>> RTPacking? I
>>>>>>>>>>>>> would have thought all packing is done on word-sized units
>>>>>>>>>>
>>>>>>>>>> anyway.
>>>>>>>>>>
>>>>>>>>>>>>> Each side of the connection can check BITSIZE(LONGINT) to
>>>>>>>>>>
>>>>>>>>>> figure
>>>>>>>>>>
>>>>>>>>>>>>> out what to do presumably no differently from the way
>>>>>>>>>>
>>>>>>>>>> INTEGER is
>>>>>>>>>>
>>>>>>>>>>>>> communicated between 32-bit and 64-bit machines. Am I
>>>>>>>>>>>>> missing
>>>>>>>>>>>>> something?
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- Tony
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Feb 15, 2008, at 10:51 AM, Tony Hosking wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ah, OK. I don't much delve in pickle-land. Anything I
>>>>>>>>>>>>>> can
>>>>>>>>>>
>>>>>>>>>> help with?
>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Feb 14, 2008, at 11:02 PM, Rodney M. Bates wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) RTPacking.T and needs to have a separate size for
>>>>>>>>>>>>>>> LONGINT,
>>>>>>>>>>>>>>> (which can vary independently of the size of INTEGER).
>>>>>>>>>>>>>>> 2) Variable length values of subrange bounds found in
>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>> descriptions (in TipeDesc.i3) can now have values beyond
>>>>>>>>>>>>>>> what the native word size can represent.
>>>>>>>>>>>>>>> 3) RTType.Kind.Longint (which I presume you, Tony, added
>>>>>>>>>>
>>>>>>>>>> recently)
>>>>>>>>>>
>>>>>>>>>>>>>>> is not handled by Pickles.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have done some coding on this, but have been
>>>>>>>>>>>>>>> interrupted.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Tony Hosking wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Why is LONGINT for pickles not yet supported?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>>>>>>> Rodney M. Bates, retired assistant professor
>>>>>>>>>>>>>>> Dept. of Computer Science, Wichita State University
>>>>>>>>>>>>>>> Wichita, KS 67260-0083
>>>>>>>>>>>>>>> 316-978-3922
>>>>>>>>>>>>>>> rodney.bates at wichita.edu
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>>>> Rodney M. Bates, retired assistant professor
>>>>>>>>>>>> Dept. of Computer Science, Wichita State University
>>>>>>>>>>>> Wichita, KS 67260-0083
>>>>>>>>>>>> 316-978-3922
>>>>>>>>>>>> rodney.bates at wichita.edu
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>> Rodney M. Bates, retired assistant professor
>>>>>>>>>> Dept. of Computer Science, Wichita State University
>>>>>>>>>> Wichita, KS 67260-0083
>>>>>>>>>> 316-978-3922
>>>>>>>>>> rodney.bates at wichita.edu
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>> --
>> -------------------------------------------------------------
>> Rodney M. Bates, retired assistant professor
>> Dept. of Computer Science, Wichita State University
>> Wichita, KS 67260-0083
>> 316-978-3922
>> rodney.bates at wichita.edu
More information about the M3devel
mailing list