[M3devel] duplicated code?

[Tony Hosking]

On 16 Nov 2008, at 10:18, Jay wrote:

> Er, the large allocation does come from the gc itself.

What's being allocated?  The side array should not be too huge.

> Tony, is there is an assumption that the heap is contiguous?
> That calls to RTOS.GetMemory return adjacent addresses?

No assumption.

> This code allocates a large array:
>
>     IF desc = NIL OR newSideSpan # NUMBER(desc^) THEN
>       WITH newDesc = NEW(UNTRACED REF ARRAY OF Desc, newSideSpan) DO
>
> I'll know more shortly.
>
> I guess..it looks like the heap can be discontiguous, but
> we do record keeping for what it all spans.

Correct.

> The comments say:
> (* The array desc and the global variables p0, and p1 describe the  
> pages
>    that are part of the traced heap.  Either p0 and p1 are equal to  
> Nil and
>    no pages are allocated; or both are valid pages and page p is  
> allocated
>    iff
> |          p0 <= p < p1
> |      AND desc[p - p0] != Unallocated
>
>
> Hm..
>
> Grow (0x44000) => 0x2b1e45256000   total: 1.5M
> GetUntracedOpenArray(0x1a80)
>    span: 6.6M   density: 24%
> stubgen: Processing RemoteView.T
> GetUntracedOpenArray(0x3f0)
> t1:0xc
> t2:0xa
> t3:0x1
> Grow (0x52000) => 0x2aaaaaaab000   total: 1.8M
> GetUntracedOpenArray(0x1ce69fc8)
>
> I have GetUntracedOpenArray printing how many bytes it is asked for.
> t1,t2,t3 are just the lengths of the strings being concatented.
> Grow(x)=>y means Grow allocated x bytes at address y.
>
> So now, these two addresses 0x2aaaaaaab000 and 0x2b1e45256000  are  
> very far apart, like 400gig.
> And it seems the heap allocator wants to allocate an array to  
> describe the pages.

Ah, yes, that is most unfortunate.


> Hm. Page size is no longer tied to the underlying system -- no  
> longer vm-tied gc.
> Perhaps perhaps blowing it up, to say, 1meg, will address this?

I am working on minimizing the need for the global array, but we do  
need something that can be easily indexed like this.  Perhaps we need  
to pass a hint to RTOS.GetMemory that will try to allocate its regions  
close together.

> But really, an array to describe pages spanning the results of  
> separate memory allocations, seems wrong.
> A sparser data structure would be good, that could describe  
> arbitrary sized runs of pages as being in the same state.

Indeed.  As mentioned above, I am working to eliminate the need for  
this.  Code that starts us on this path will be checked in within a  
day or so.

>  - Jay
>
>
>
> From: jay.krell at cornell.edu
> To: m3devel at elegosoft.com
> Date: Sun, 16 Nov 2008 15:31:25 +0000
> Subject: [M3devel] duplicated code?
>
> Anyone want to clean up this duplicity?
>
>
> D:\dev2\cm3.2>dir /s/b asttotype.m3  StubCode.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\AstToType.m3
> D:\dev2\cm3.2\m3-db\stablegen\src\AstToType.m3
> D:\dev2\cm3.2\m3-comm\sharedobjgen\src\StubCode.m3
> D:\dev2\cm3.2\m3-comm\stubgen\src\StubCode.m3
>
>
> somewhere in there, AMD64_LINUX tries to allocate a lot of memory,  
> and fails
> either there or soon thereafter.
> The garbage collector is working.
>
>  - Jay
>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20081116/e6f957c5/attachment-0002.html>