[M3devel] [modula3/cm3] Packed set literal generation issue (#21)

Tue Sep 26 13:03:21 CEST 2017

> If you use low-level techniques, there are going to be a lot more things to address in source code to make things independent of word size and endianness.

I suppose you’re also going to have to deal with native byte sizes and addressing unit sizes...

Also, what did you mean when you previously mentioned “more complicated variations to adapt to different endianness”?

— JC

-----Original Message-----
From: Rodney M. Bates [mailto:rodney_bates at lcwb.coop] 
Sent: Tuesday, September 26, 2017 0:15
To: JC Chu <jcchu at acm.org>; rodney.m.bates at acm.org
Cc: m3devel at elegosoft.com
Subject: Re: [modula3/cm3] Packed set literal generation issue (#21)

On 09/25/2017 06:25 AM, JC Chu wrote:
>> I don't think Word and Long will help, since, as far as types are concerned, Word.T = INTEGER and Long.T = LONGINT.  They mainly just provide operators that apply unsigned interpretation to the bits and do other bit-twiddling.
>
> I was really more concerned about the bit layout of sets: just which bit of SET OF [0..31] {0} should be 1?  We know it’s supposed to be whichever bit that represents the first element of ARRAY [0..31] OF BITS 1 FOR BOOLEAN, but the language definition doesn’t tell us that, either.
>
> So in the end I guess sets are just sets, suitable when you want a nice syntax and don’t have to care about the representation.

Yes, Most of Modula3's builtin types are more abstract in a few ways than in many
languages.

> Bit sets as used in other languages are really words, whose bits can be inspected and manipulated by using Word.Extract and Word.Insert, respectively.

But you can alternatively do it with bit twiddling in Modula3 too, but the language
is inviting you to use the more abstract types and operators.  If you use low-level
techniques, there are going to be a lot more things to address in source code to make
things independent of word size and endianness.  I have written a fair amount of code
like this, and it requires a *lot* of attention to details that you could forget about
otherwise

>> Unfortunately, the compiler currently insists every set must be a full native INTEGER size, whether you try to size it yourself with BITS or let the compiler do it.
>
> That’s weird.  Here on my host INTEGER and LONGINT are 32- and 64-bit, respectively, and the type BITS BITSIZE(LONGINT) FOR [0..-1 + BITSIZE(LONGINT)] is accepted, and 64 bits and 8 bytes in size.The type BITS 24 FOR [0..16777215] is also valid, and 24 bits and 3 bytes in size.  (On the other hand, the type RECORD s: BITS BITSIZE(LONGINT) FOR [0..-1 + BITSIZE(LONGINT)] END is unaccepted, which I think is allowed by the language definition.)
>

 From 2.2.5:

The values allowed for n are implementation-dependent. An illegal value for n is a static error.
The legality of a packed type can depend on its context; for example, an implementation could
prohibit packed integers from spanning word boundaries.

So the language pretty much allows a compiler to refuse anything it doesn't want to do, as long
as it gives a compile error.  Probably it accepts the BITS 64 for a scalar (which it is not required
to actually honor, but not inside a record, where it honoring it would entail spanning a word
boundary.  Just my speculative attempt to read the original compiler developers' minds.

> — JC
>
> -----Original Message-----
> From: Rodney M. Bates [mailto:rodney_bates at lcwb.coop]
> Sent: Monday, September 25, 2017 5:00
> To: JC Chu <jcchu at acm.org>; modula3/cm3 <reply+00adcf3441dcbaa2d5083d68d816543de0cf3bafdae19c0292cf0000000115d94f8892a169ce0f650920 at reply.github.com>
> Cc: rodney.m.bates at acm.org; m3devel at elegosoft.com
> Subject: Re: [modula3/cm3] Packed set literal generation issue (#21)
>
>
>
> On 09/24/2017 07:04 AM, JC Chu wrote:
>>> Are you aware that the BITS 32 here has no effect on the size of the global variable?
>>
>>
>>
>> Yes.  I was looking for a bit set type that’s guaranteed to have the same representation as some integer type, such as BITS BITSIZE(INTEGER) FOR [0..-1 + BITSIZE(INTEGER)] for INTEGER.  But that guarantee doesn’t seem to exist, according to the language definition.  Seems like the correct way is to use Word interface (but then Long (the corresponding interface for LONGINT) doesn’t seem to be usable).
>>
>>
>
> Yes, the language considers scalar variable sizes (locals, globals, parameters) to be
> the implementation's business.  However, for low-level coding, you can wrap what you
> want inside a record, usually with only one field, and access the field:
>
>   TYPE SetT = SET OF [0..31];
>   TYPE IntSizedSetT = RECORD S : BITS BITSIZE(INTEGER) FOR SetT END;
>   VAR Set := IntSizedSetT { SetT { 0 } };
>   VAR Int : INTEGER;
>
>   BEGIN
>      Int := LOOPHOLE(Set.S, INTEGER);
>    END
>
> Any mistakes or failure of the size self-adaptation code will give a
> compile-time error on the LOOPHOLE, due to unequal sizes.
>
> I have used this technique a number of times in low-level code to make
> it self-adapt to different word sizes, and more complicated variations
> to adapt to different endianness.  The compiler and runtime do this in
> some places. In some other places, they just make assumptions about
> what the compiler will do that happen to be correct currently, but are
> not guaranteed.  I think not all of them would give compile-time errors
> if/when the assumptions changed.  I have fixed a few of them, but not
> systematically.
>
> I don't think Word and Long will help, since, as far as types are concerned,
> Word.T = INTEGER and Long.T = LONGINT.  They mainly just provide operators
> that apply unsigned interpretation to the bits and do other bit-twiddling.
> I do use Word.T instead of INTEGER where appropriate in declarations just
> as documentation of the way they will be used.  Otherwise (anyway?) it gets
> tedious keeping track of when to use '+' and when to use Word.Plus.
>
> Unfortunately, the compiler currently insists every set must be a full native
> INTEGER size, whether you try to size it yourself with BITS or let the
> compiler do it.  This looks easy to fix, but would require thorough vetting
> to be sure all the various paths in the front and back ends can handle it.
>
> BTW, the compiler looks to be padding every record out to a full byte
> length, which won't affect this technique but limits some other things.
>
> Also BTW, in working on the fix of the assertion failure you reported,
> I find there is another assertion failure when trying to pass
> BITS n FOR SetT as an actual parameter to a SetT formal.For VALUE and
> READONLY, this should work, as the types are assignable.
>
>
>>
>> — JC
>>
>>
>>
>> *From:* Rodney M. Bates [mailto:notifications at github.com]
>> *Sent:* Wednesday, September 20, 2017 5:35
>> *To:* modula3/cm3 <cm3 at noreply.github.com>
>> *Cc:* jcchu <jcchu at acm.org>; Author <author at noreply.github.com>
>> *Subject:* Re: [modula3/cm3] Packed set literal generation issue (#21)
>>
>>
>>
>> Fixed on github, commit 12f50b4cde746056b943bd8aaa105c3fcb46b1a8
>>
>> Deleted the assertion, which is neither true nor needed.
>>
>> But see below:
>>
>> On 09/16/2017 11:54 PM, jcchu wrote:
>>> See the fragment below.
>>>
>>> VAR r := SET OF [0..31] { 0 }; (* OK *)
>>> s := BITS 32 FOR SET OF [0..31] { 0 }; (* -- assertion failure in “SetExpr.m3” *)
>>
>> Are you aware that the BITS 32 here has no effect on the size of the global variable?
>>
>> Quoting from 2.2.5, Packed types:
>>
>> TYPE T = BITS n FOR Base
>>
>> where Base is a type and n is an integer-valued constant expression.
>> The values of type T are the same as the values of type Base, but variables of type T
>> that occur in records, objects, or arrays
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> will occupy exactly n bits and be packed adjacent to the preceding field or element.
>>
>> s is not inside a record, object, or array, so the BITS specification has no effect on
>> the memory allocated by the compiler.
>>
>> There are, however some more obscure differences:
>>
>> 1) Although T and BITS n FOR T are are assignable to each other,
>> BITS n FOR T and BITS m FOR T, (n#m) have no subtype or assignability
>> relationship, so to assign one to the other would require two assignment
>> steps, with a T as the intermediate type.
>>
>> 2) Anywhere type equality, as opposed to assignability, is required, T and
>> BITS n FOR T will not match. For example, you cannot pass one of these to
>> the other as a VAR parameter.
>>
>>
>>
>>
>>
>>>
>>> —
>>> You are receiving this because you are subscribed to this thread.
>>> Reply to this email directly, view it on GitHub <https://github.com/modula3/cm3/issues/21>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGVctFuGQY2yHbHNLupX6pnu4MVJDlXgks5sjKYMgaJpZM4PaCl5>.
>>>
>>
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub <https://github.com/modula3/cm3/issues/21#issuecomment-330680504>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AK3PNMFBPWWpfK3zQpmt8ebPZbPlaBQ6ks5skDOIgaJpZM4PaCl5>.Image removed by sender.
>>
>

-- 
Rodney Bates
rodney.m.bates at acm.org