[M3devel] explanation of CheckLoadTracedRef?

[Jay]

Yes this helps, thank you. Maybe checkin under doc? One thing you didn't understand my wording about and you seemed to contradict is "VAR to VAR".

 

 

PROCEDURE F1() =

VAR i;

BEGIN

 F2(i);

END F1;

 


PROCEDURE F2(VAR a:INTEGER)=
BEGIN

 F3(i);

END F2;

 


PROCEDURE F3(VAR a:INTEGER)=
BEGIN

 a := 1;

END F3;

 


Where are the calls to CheckLoad/StoreTracedRef?

  (Duh, I can try it out..)

It seems redundant to put them "everywhere".
Esp. it seems that F2 shouldn't have do anything.


 

But throw in that F3 might extern and that is unclear.

So I just made up a suggestion that VAR is not good to pass to C code.

That checks are inserted when 1) a pointer is dereferenced -- loaded

or stored in Modula-3 or 2) a

pointer becomes untraced (var becomes untraced ref, among others).

 

 

 > Right.  I think I did see you compute and pass an offset to C code

 

 

I had something like that very recently but think I didn't set it or commit it.

I had something like:

 

 

size_t offset;

 

 

Init(Mutex* root, int* interior)

{

  offset = (size_t)interior - (size_t)root;

}

 

 

DoSomething(Mutex* anotherRoot)

{

  int* interior = (int*)(offset + (size_t)anotherRoot);

 printf("%d\n", *interior);

}

 

 

but I came up with a way to avoid that I think, and then rolled it all back or something anyway.

 

 

 

 - Jay

 


CC: m3devel at elegosoft.com
From: hosking at cs.purdue.edu
To: jay.krell at cornell.edu
Subject: Re: [M3devel] explanation of CheckLoadTracedRef?
Date: Wed, 22 Apr 2009 10:02:52 +1000




On 21 Apr 2009, at 19:37, Jay wrote:


> > How bad/unportable was it the previous way, the VM-synchronized way?
> 
> Not compatible with system threading.

 
Really? NT386 wasn't VM-synchronized back in 3.6, 4.1, etc.?


It was, but every system call had to be wrapped with a call to acquire the global heap lock!



Or only with great cost?


Yes, taking the lock was expensive and prevented scaling on multi-cores.


I have to admit, those old import .libs, kernel32.lib, etc. I didn't realize what was in them when I deleted them. I thought they were just regular import .libs.


There was also the work to wrap all dll symbols to acquire the lock.


I think I got luckly in that the overlap between me deleting them and you removing VM-synchronized GC was small or zero.


You should have seen the mess it was before...  ;-)


> You should *never* access a field in the heap in C code! All 
> accesses to traced fields in the heap must take place in Modula-3. 
> Otherwise things will break! C wrappers should not do anything other 
> than forward calls to C library calls. They should not perform heap 
> accesses.

 
Ok, that makes sense.
Important "out" is the accessing stack is always ok.


Yes, so long as a references to an object is held on the stack then it is safe to pass an address within it to external calls.  Thus, many C functions can take VAR arguments that may end being references to the fields of objects.  The compiler injects the necessary CheckLoad/CheckStore operations when passing  VAR parameters, etc., and the GC maintains the invariant that all stack-referenced objects don't move, stay black (for incremental GC), and remain dirty (for generational GC).


But this is a requirement I didn't keep in mind.
Now, luckily, the C wrappers are all relatively thin and not difficult
to re-review in their entirety.


Right.  I think I did see you compute and pass an offset to C code, but that may have only been in code you e-mailed me for perusal rather than code that got checked in.  Might be worth reviewing...


But, take for example "open".
The first parameter to it is bound to be in the heap.


The ambiguous roots garbage collector we use maintains the invariant that pointers to the interior of heap objects from the stack *pin* that object in the heap: it will not move while the pointer from the stack exists, and invariants will be maintained so that its contenst can be manipulated safely even in the face of incremental and generational GC.


But probably it is untraced or somehow ok, since it does
come from a module used primarily for C interop.


Certainly, C code should never be manipulating the *traced* fields of traced heap objects.  It is fine for it to manipulate the untraced fields of traced heap objects.


And, the line between C wrappers and the "C library" that they forward to
does not exist.
If I, say, pass a VAR to a VAR..no check is made?


Not sure what you mean by this.  Any call that passes traced VAR params will generate a check as necessary before the call.


Important to declare extern/C functions as taking UNTRACED REF and not VAR?


No, VAR is fine.  So long as the VAR value being passed is not traced.


> I think you are confusing incremental and incremental GC.
You assume I understand more than I do (I assume you have a typo. :))


;-)


"generational" -- the concept that most objects die young.
  (aka most objects could have been allocated on the stack...)


Not quite.  The idea is that the likelihood of an object dying is a function of its age.  There is a *weak* generational hypothesis that "most objects die young", and a *strong* generational hypothesis that "the older an object is the less likely it is to die".  Many (but not all) programs support these hypotheses, which permits generational GC to focus effort where it is likely to be profitable (i.e., to free up a lot of space).


But does that imply detailed implementation choices, or is just like a "guiding principle"?
I guess it implies the heap is split into at least two generations, old and young.
Though I guess in reality there is a range of young, less young, lesser young, least young, etc.


Right, many different collectors have exploited age in this way.  For the M3 collector we have just two generations: old and young.


And that has objects age, they should be moved from young to old heaps, and references to them either updated right away, or "caught" upon use and updated then...something like that.


Old-space objects are "clean" if they contain no references to young-space objects.  The Modula-3 compiler injects checks to make sure that whenever we store a reference into a clean old-space object it is marked "dirty".  When a young-space collection occurs we must process the references in dirty old-space objects as roots into the young-space.


"incremental" -- don't pause the world..


Not quite.  The opposite of stop-the-world (stopping all the mutator threads to process their stacks) is on-the-fly.  Incremental refers to the ability to interleave GC work with mutator work.  If the GC work can be interleaved with mutator threads without stopping the mutator threads at each increment then the collector is said to be concurrent (GC work proceeds concurrently with mutator work).
The current M3 collector has a stop-the-world non-moving phase, followed by an concurrent copying phase.  I have some incomplete work that will also make the M3 collector on-the-fly (no STW phase) and parallel (multiple GC threads can operate concurrently).


Hope all this helps!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20090422/4a0fe08e/attachment-0002.html>