[M3devel] files larger than 2gig

[Mika Nystrom]

Jay I just want to add that I've written a lot of code that "streams"
files greater than 2 GB just fine in Modula-3.  The trick, which I learned
from Blair MacIntyre's rdwr package, is to reset the pointers periodically:


(* $Id: RdWrReset.m3,v 1.1 2008/11/26 18:32:40 mika Exp $ *)

MODULE RdWrReset;
IMPORT Rd AS R, Wr AS W;
IMPORT RdClass, WrClass;
<*NOWARN*>IMPORT UnsafeWr, UnsafeRd;

(* Since we need to use the Mutex properties of Rd.T and Wr.T, we
   should actually import UnsafeWr and UnsafeRd. We need to add the
   following revelations, as the comment in UnsafeRd points out, if we
   want to include both the Unsafe* and *Class interfaces. *)
REVEAL RdClass.Private <: MUTEX;
REVEAL WrClass.Private <: MUTEX;

PROCEDURE Rd (rd: R.T) =
  BEGIN
    LOCK rd DO DEC(rd.cur, rd.lo); DEC(rd.hi, rd.lo); rd.lo := 0; END;
  END Rd;
          
PROCEDURE Wr (wr: W.T) =
  BEGIN
    LOCK wr DO DEC(wr.cur, wr.lo); DEC(wr.hi, wr.lo); wr.lo := 0; END;
  END Wr;

BEGIN END RdWrReset.


Jay writes:
>
>Hm, here is another approach -- leave more code alone, like don't attempt to l
>et FileRd/FileWr work on files larger than 2gig on a 32bit system for example.
> But do fix the lowest level status record to have either LONGINT or LONGREAL,
> preferable LONGREAL for the sake of NT386 (much as I dislike floating point..
>). 
>That is, an intermediate goal would be: 
> - browing to a directory with large files with the Trestle file picker should
> not crash 
> - 32bit Modula-3 code may or may not be able to read/write such files 
> 
> - Jay
>
>
>
>----------------------------------------
>> From: jay.krell at cornell.edu
>> To: m3devel at elegosoft.com
>> Subject: RE: [M3devel] files larger than 2gig
>> Date: Mon, 22 Jun 2009 13:05:13 +0000
>>
>>
>> Hm..I'm not sure.
>> Integers have certain properties, like dividing an integer by an integer yie
>lds an integer,
>> that floating point doesn't. Integer division tends toward zero faster than 
>floating point division.
>> I can try getting it all to compile with LONGINT maybe instead.
>> The current behavior is pretty lame.
>>
>> - Jay
>>
>> ----------------------------------------
>>> From: jay.krell at cornell.edu
>>> To: m3devel at elegosoft.com
>>> Date: Mon, 22 Jun 2009 12:58:17 +0000
>>> Subject: [M3devel] files larger than 2gig
>>>
>>>
>>> C:\dev2\cm3.2\m3-libs\libm3\src\os\Common\File.i3
>>>
>>>
>>> TYPE
>>> Status = RECORD
>>> type: Type;
>>> modificationTime: Time.T;
>>> size: INTEGER;
>>> END;
>>>
>>>
>>> size: INTEGER causes exceptions when you use the Modula-3 gui
>>> and browse to a directory with files larger than 2 gig.
>>>
>>>
>>> I suggest size be changed to LONGREAL, which generally has a 53 bit mantiss
>a
>>> (out 64 bits total) and thus can represent integers very much larger than I
>NTEGER.
>>>
>>>
>>> LONGINT is a tempting option but doesn't help on the current NT386 platform
>,
>>> and I think 53 bits will last a very long time.
>>>
>>>
>>> I'm just trying out such a change and I can see it is not source compatible
>:
>>>
>>>
>>> "../src/rw/FileRd.m3", line 73: incompatible argument types: MIN
>>> "../src/rw/FileRd.m3", line 140: types are not assignable
>>> 2 errors encountered
>>> "../src/rw/FileWr.m3", line 87: incompatible argument types: MIN
>>> "../src/rw/FileWr.m3", line 103: incompatible argument types: MAX
>>> 2 errors encountered
>>>
>>>
>>> Nevertheless I think it should be done, probably even for this release