[M3devel] Re: RTHooks CheckStoreTraced and CheckLoadTraced

[Tony Hosking]

My immediate reaction is that you probably want to approach reads  
slightly differently.  Instead of using your generic InTypeRead  
routine to access the appropriate bits/bytes of the value, you  
probably want to replace it with some sort of simple addressing  
mechanism that computes the raw address of the field.  Traced  
reference fields must always be aligned within the heap object they  
are stored in so your addressing for these will not need bit  
offsets.  Once you have an ADDRESS "addr" for the traced reference  
field, so long as you use the idiom I gave earlier, you will get the  
correct behavior for reads.  That is,

WITH field = LOOPHOLE(addr, UNTRACED REF REFANY) DO
   .. use field^ to access the traced reference stored at address  
field ..
END;

The alternative is to invoke "RTHooks.CheckLoadTracedRef(r)" on the  
traced reference "r" once you have loopholed it through the raw field  
access into a REFANY, before you return it from InTypeReadRef:

PROCEDURE InTypeReadRef (this: T; obj: REFANY; field: CARDINAL):  
REFANY =
   VAR v: REFANY;
   BEGIN
     this.check(field, Kind.Ref, obj);
     this.read(obj, field, ADR(v), BYTESIZE(v));
     RTHooks.CheckLoadTracedRef(v);
     RETURN v;
   END InTypeReadRef;

This is safe for the current CM3 ambiguous roots collector, but this  
code would probably break with a fully copying/relocating collector  
(if one was ever implemented for Modula-3), since the raw bytes of  
the reference might become stale between the point you load them from  
memory and the point they are placed in the typed REFANY variable  
"v" (effectively, you'd be hiding a reference from the GC for a brief  
period, and it would not be able to update that hidden reference in  
the face of its target being moved).

I note, generally, that your field addressing mechanisms only work  
anyway because we are using an ambiguous roots collector that pins  
all objects referenced from the thread stacks (your "obj" parameters  
in the accessor methods), which allows the use of RTHeap.GetDataAdr  
to provide an address for the object that is valid while the object  
reference is held on the stack.

When writing references to object fields you will need to invoke  
"RTHooks.CheckStoreTraced(o)" on the heap object "o" whose field is  
being stored.  Thus, for InTypeWriteRef:

PROCEDURE InTypeWriteRef(this: T; obj: REFANY; val: REFANY; field:  
CARDINAL) =
   BEGIN
     this.check(field, Kind.Ref, obj);
     WITH f = this.offs.get(field) DO
       IF NOT ISTYPE(f, RefField) OR NOT RTType.IsSubtype(TYPECODE 
(val), NARROW(f, RefField).tc) THEN RAISE WrongType END;
     END;
     this.write(obj, field, ADR(val), BYTESIZE(val));
     RTHooks.CheckStoreTraced(obj);
   END InTypeWriteRef;

If you are performing multiple writes of reference fields on the same  
object (e.g., to an array) then you can invoke CheckStoreTraced once  
for the whole object.

Hope this helps.

On Apr 18, 2007, at 10:49 PM, Darko wrote:

> Thanks, that's very helpful. I agree I shouldn't need it but it's  
> exactly because I'm not using the compiler is why I need to know  
> about it. The code is attached, it's still in development and  
> unfinished but you'll get the idea.
>
> Basically the module allows you to programatically access structure  
> fields. It allows for applications like being able to create  
> indexes (using an adaptation of the Table code) of objects using  
> arbitrary fields as keys. It makes dynamic programming a bit more  
> dynamic. The code is intended to be completely safe, but there's  
> more checks I need to do.
>
>
>
> <InType.m3>
> <InType.i3>
>
>
>
>
> On 19/04/2007, at 4:05 AM, Tony Hosking wrote:
>
>>
>> On Apr 18, 2007, at 9:37 PM, Darko wrote:
>>
>>> Not actually using that code, but I did see your helpful commit  
>>> on it some time ago. It seems that the problem I described  
>>> earlier is due to another bug. The code I'm working on does a lot  
>>> of reading and writing of traced references and I was hoping to  
>>> get a better understanding of situations I need to be aware of  
>>> and how to use those RTHooks calls more efficiently.
>>
>> You SHOULD NOT need to use these calls so long as you are  
>> accessing using properly typed references (as in my example  
>> earlier).  This is not an issue for SAFE code, only in UNSAFE  
>> modules must you be careful how you use LOOPHOLE to cast references.
>>
>>> So say I have two traced objects and I am copying a traced  
>>> reference from a field in one to the other. What set of calls  
>>> would be normally required there?
>>
>> How are you copying the references?  I would like to see your code  
>> for this.  So long as you properly type things then the calls to  
>> RTHooks.CheckLoadTracedRef and RTHooks.CheckStoreTraced will be  
>> generated by the compiler.
>>
>> Anyway...
>>
>> RTHooks.CheckLoadTracedRef is generated by the compiler whenever  
>> you access a traced reference in memory (e.g., via dereference or  
>> from a shared variable).  Its job is to prevent mutators from  
>> acquiring references to "gray" objects (that are reachable but  
>> still to be scanned for the traced references that they contain).   
>> The check turns the object from gray to black by scanning its  
>> references and making sure none of them refer to white objects  
>> (that are not known to be reachable by the collector).
>>
>> RTHooks.CheckStoreTraced is generated by the compiler whenever a  
>> traced reference is stored (or might be stored, e.g., for VAR)  
>> into a traced object.  The check makes sure that the generational  
>> GC knows that the object has been modified.  It needs to know this  
>> for objects in the older genration.
>>
>> Again, I reiterate that it is not my intention that programmers  
>> actually call these runtime hooks explicitly!  I am certain I can  
>> rewrite any code you might have so that you do not need to call  
>> them explicitly.  Instead, properly typed references will result  
>> in the checks being generated for you by the compiler.  Please  
>> feel free to send me code snippets for review.
>>
>>> I'm sure you haven't got time to go into the minutiae of the  
>>> runtime system, but any hints would be appreciated.
>>>
>>> Cheers,
>>> Darko.
>>>
>>>
>>> On 18/04/2007, at 4:34 PM, Tony Hosking wrote:
>>>
>>>> I assume you have an apply method for your RTTypeMap.Visitor  
>>>> that takes "field: ADDRESS" and treats it as "REF REFANY".    
>>>> This is wrong.  When reading a REF field you should use the  
>>>> following idiom:
>>>>
>>>> WITH ref = LOOPHOLE(field, UNTRACED REF REFANY) DO
>>>>   ... access field via ref^ ...
>>>> END;
>>>>
>>>> This will automatically insert a call to the appropriate runtime  
>>>> routines on accessing the reference field.
>>>>
>>>> There should be no need for you to call the runtime routines  
>>>> directly.
>>>>
>>>> On Apr 17, 2007, at 9:51 PM, Darko wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Wondering if you can explain the use of these calls a little  
>>>>> more. I'm currently using type maps to read and write fields  
>>>>> from traced objects. Reading a traced reference from inside a  
>>>>> traced object into a local variable is not working as it  
>>>>> should. Should I use CheckLoadTraced and if so when and how?  
>>>>> Looking at your changes to RTTypeMap, writing references into  
>>>>> objects means you need to call CheckStoreTraced on the object  
>>>>> written inside of, before it is written?
>>>>>
>>>>> Cheers,
>>>>> Darko.
>>>>
>>
>