[M3devel] 64bit INTEGERs, WIDECHAR: language specified or configuration/target dependent?

[Elmar Stellnberger]

Thread forked from: Re: [M3devel] 32bit host 64bit target TextLiteral 
recurring problem

Am 04.06.15 um 18:35 schrieb Rodney M. Bates:
>
>
> On 06/04/2015 09:01 AM, Elmar Stellnberger wrote:
>> Am 03.06.15 um 03:58 schrieb Jay K:
>>> We cannot cross from 32bit host to 64bit target.
>>>
>>>
>>> Ok to commit this?
>>>
>>>
>>
>> Compatibility between the 32bit and the 64bit version of the same 
>> program would be the minimum I expected from a pickle.
>> To me this is just another argument for a language specified value 
>> range of INTEGER.
>> It makes certain things easier.
>>
>> My suggestion was:
>> TYPE OFFSET = BITS BITSIZE(ADDRESS) FOR INTEGER; (and INTEGER = BITS 
>> 32 FOR INTEGER for 32 and 64bit targets)
>> while also allowing BITS 16 FOR INTEGER (if that works for WIDECHAR I 
>> believe there is no reason to forbid it for INTEGER)
>> BITS 64 FOR INTEGER would only work on x64 systems.
>>
>
> What you want is already pretty much here.  Just declare your desired 
> type yourself:
>
> TYPE Int32T = [-16_7FFFFFFF-1 .. 16_7FFFFFFF];
   This should be good news. However my suggestion was to make Int32.T 
the default
as this will be somewhat faster for the average application which is 
data intensive (just
think of image processing) which will never be rewritten to use Int32.T 
instead of INT.
All of our current and old programs being prepared to run on a 32bit as 
well as a 64bit
target should compile and run without any problem. There is one single 
exception
where the compiler would throw an error: when using INTEGER for address 
arithmetics
in unsafe modules (error is: can only LOOPHOLE between types of the same 
size, or
some way similar as I have already tested this). There we would need the 
so called
targetint/offset.
   If I understand things right the following two things would need to 
get implemented
to allow a default int size of 32 for existing applications (be it the 
default or just an
additional compiler option as the size of WIDECHAR already is):
* automatic range adaption for packed ints (i.e. those with BITS .. FOR)
* allowing non 32bit alignment of INTs in memory (as is already the case 
for WIDE[CHAR]s)
>
> It will always have this range and occupy 32 bits, regardless of the 
> native word size.  Pickles
> will always save and reread it with exactly this range, even between 
> machines of different
> word size.
So the target size of the pickle is already determined by the respective 
INT subrange, right?
Something that should to my believe be well documented as one could 
easily like to extend
an enum with 255 members to more or equal than 256 an then wonder why 
the new an the
old file formats become incompatible. To circumvent such a problem we 
would likely need
to allow BITS 8*anything FOR INTEGER and take the bitcount specified in 
BITS for in order to
determine the target size of an object instead of semi-automatically 
readjusting it all the
time.
>
> The only thing that will differ from 32- to 64-bit machine is the size 
> of intermediate results in
> expressions.  If you are already _not_ being careful about avoiding 
> overflows, there are probably
> cases where the results of silent overflows would differ with machine 
> word size.  Not sure what
> you would really want here.  The unsigned arithmetic operations in 
> Word are explicitly defined
> to silently do binary twos-complement wrap-around when overflows 
> happen.  The language does not
> define this for the builtin signed operators.  In code where I care 
> about overflows, I never assume
> this.  It is probably machine-dependent.
Yes, I believe it would be good like this. No real 32bit INT with 32bit 
arithmetics but just restricting
the range for Int32.T when it gets written into memory.
>
> Or, you can import it: Int32.T. is already declared, in m3core, with 
> this range.
>
> However, Int32.T specifies BITS 32 FOR ...  I strongly suggest this is 
> almost never what you really want.
> The only place the BITS specification makes any difference is when you 
> declare a record or object
> field of this type.  In all other situations, BITS 32 FOR .. has _no 
> effect_.  The only purpose of
> BITS is so you can force the layout of a record.  In that case, since 
> the range needs exactly 32
> bits anyway, even if you omit the BITS, a field of this type will 
> still always occupy 32 bits.
Nonetheless it is currently not possible to declare a variable of a 
32bit type globally:

  TYPE Pixel = BITS 32 FOR INTEGER;

unluckily does not work as we can have no global variable of type Pixel.
This is not only un-orthogonal but also a very practical problem; 
Suggesting that we call
procedures with a parameter of type pixel we will never be able to 
change that type f.i. to
a packed record with members red, green, blue alpha if there are places 
in our main
program where we need to declare VAR pix1, pix2 : INTEGER just because 
Pixel is a packed
type being refused as global variable.

>
> But, with the BITS specification, the compiler is not allowed to 
> insert alignment padding, (if it
> were, you could not fully specify the layout) and if the field comes 
> at a place that is not 32-bit
> aligned, you will either get a compile-time error or maybe a compiler 
> will accept it and handle
> unaligned field access (it is not required to--ours does not), neither 
> of which is probably what
> you want.  Without BITS, the compiler will pad as needed.  With BITS, 
> you will have to insert explicit
> padding for alignment yourself as needed, which can be different for 
> every field of this type of every
> record/object.  And later adding/removing/changing any prior field can 
> upset it and require you to
> rework the padding.  Moreover, I am sure there are simple cases where 
> the required padding differs
> between 32-bit and 64-bit machines.
On the long term I would personally even advocate an ALIGN directive:
  f.i. TYPE SSEdata = ALIGN 128 FOR RECORD ... END;

It would be very handy to have this for tapping the full power of the 
vectorization units
of current processors. Manually re-aligning heap allocated objects in an 
unsafe module
is not a very satisfying alternative (no local/global/stack allocation, 
memory waste and
thus a certain performance loss if there are many wholes). However this 
would possibly
also make changes to our garbage collector necessary as it currently 
only guarantees
word alignment. Finally it would also be possible to write ALIGN 
BITSIZE(TARGETINT) ....


>
> Having been bitten by this more than once, I now only put BITS..FOR as 
> the anonymous type of a specific
> field, never in a named type to be used in multiple places.
A less bitchy and more orthogonal Modula-3 compiler supporting integer 
globals and
locals of non word size and alignment would be a real pleasure, do you 
consent Rodney?
>
> (Actually, BITS does the same thing when used as the type of an array 
> element, but cases where this
> matters are rare.  If the bit count evenly divides the word size, 
> there won't be any alignment padding
> needed, and BITS won't matter.  Otherwise, the compiler is likely to 
> refuse, unless the entire array
> fits in a single machine word.  I do have a couple of such cases.)
>
>> The value range - bitsize correlation problem could be solved easily 
>> by automatically extending and reducing the value
>> range of an integer type to what its binary representation allows as 
>> long as no explicit range has ever been specified for
>> any of its supertypes (i.e. BITS 16 FOR INTEGER = BITS 16 FOR 
>> [-32768..+32767]).
>>
>> Finally it needs to be said that the argument that target size 
>> arithmetics is always the fastest which was often used in
>> here can be very wrong. It does not hold for the 64bit systems of 
>> today because the CPU is no more the bottleneck. In
>> a fact now the memory hierarchy has become the new bottleneck (which 
>> means that using more memory for the same
>> tends to be much slower).
>>
>> Regards,
>> Elmar
>>
>