[M3devel] WeakRef mechanism alive and well? cm3-cvshead

[Tony Hosking]

The thing about conservatism is that even if you NIL *all* references  
to an object that just happens to lie on a pinned page, that object  
(and objects reachable from it) will be retained until the page is no  
longer pinned.

Antony Hosking | Associate Professor | Computer Science | Purdue  
University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484



On Mar 13, 2008, at 1:43 PM, Dragiša Durić wrote:

> I understand transitiveness :), but I've made big effort to escape  
> that.
>
> Lifetime of thread equals lifetime of Client.T instance, and that
> instance is kept in thread's closure, and also in local variable. Both
> are set to NIL before thread termination - all terminations and joins
> happen and are counted and shown.
>
> Pointers to Message objects are kept in two places. Hub.T keeps  
> pointer
> to list tail, and each Client.T contains pointer to it's current
> element. That pointer is forwarded as Message's are added to end. No
> temporary variable, not even WITH substitution is used. And these
> threads are ones dying with  their Client.T's.
>
> "head" is new mechanism used to unlink all elements with id less than
> minimal id used in active Client.T instances. Unlinked totally and all
> as one free for garbage collector to finish them.
>
> I see no transition.
>
> dd
>
> On Thu, 2008-03-13 at 12:45 -0400, Tony Hosking wrote:
>> Realizing that conservative roots collectors like ours can retain  
>> data
>> unnecessarily.  If a thread stack refers to a page that holds a
>> reference that (transitively) refers to some otherwise dead object
>> then that object will be retained.  We say that the page is pinned.
>> Smartening up the collector to ignore dead objects in pinned pages is
>> possible, but not currently implemented.
>>
>> Antony Hosking | Associate Professor | Computer Science | Purdue
>> University
>> 305 N. University Street | West Lafayette | IN 47907 | USA
>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>
>>
>>
>>
>>
>> On Mar 13, 2008, at 12:28 PM, Dragiša Durić wrote:
>>
>>> [17:25:19] <Hubetta> 117 users, 1 ops and 1 unresolved; minId =
>>> 18617; thrs = 120
>>> [17:25:19] <Hubetta> started = 663; cleaned = 452; nilled = 543;
>>> joined = 543
>>> [17:25:19] <Hubetta> Dangling thrs-a-b-n = 1
>>> [17:25:19] <Hubetta> Live queue end is 18621 (head = 18616),
>>> maxClean = 18611, cleaned = 16583
>>>
>>> About one hour uptime, after I added RTCollector.Collect() call
>>> every 20
>>> seconds.
>>>
>>> dd
>>>
>>> On Thu, 2008-03-13 at 10:54 -0400, Tony Hosking wrote:
>>>> Yes.  RTCollectorSRC.FinishCollection;
>>>> RTCollectorSRC.StartCollection.  This will finish the current
>>>> (possibly minor) collection, and start a major collection.  The
>>>> major
>>>> collection must finish before you can guarantee garbage has been
>>>> freed, so you may want to RTCollectorSRC.FinishCollection.  This
>>>> sequence is packaged as RTCollector.Collect.
>>>>
>>>> Antony Hosking | Associate Professor | Computer Science | Purdue
>>>> University
>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mar 13, 2008, at 10:22 AM, Dragiša Durić wrote:
>>>>
>>>>> Can I force "major collection" ?
>>>>>
>>>>> dd
>>>>>
>>>>> On Thu, 2008-03-13 at 10:08 -0400, Tony Hosking wrote:
>>>>>> Indeed! Can you devise a testcase?  Note that the generational
>>>>>> collector will retain old objects for some time until a major
>>>>>> collection occurs,
>>>>>>
>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>> Purdue
>>>>>> University
>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mar 13, 2008, at 9:56 AM, Dragiša Durić wrote:
>>>>>>
>>>>>>> And not only that, their stackbase is also set to NIL,
>>>>>>> meaning
>>>>>>> their
>>>>>>> stack is not regarded in any way during future collections -
>>>>>>> meaning
>>>>>>> all
>>>>>>> local variables are forgotten once apply method returns.
>>>>>>>
>>>>>>> That way, "spuriousness"  is not an issue, once thread
>>>>>>> returns?
>>>>>>>
>>>>>>> dd
>>>>>>>
>>>>>>> On Thu, 2008-03-13 at 09:46 -0400, Tony Hosking wrote:
>>>>>>>> They should get unlinked from the global ring.
>>>>>>>>
>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>> Purdue
>>>>>>>> University
>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mar 13, 2008, at 6:16 AM, Dragiša Durić wrote:
>>>>>>>>
>>>>>>>>> What happens to thread stacks once threads are Join-ed?
>>>>>>>>>
>>>>>>>>> dd
>>>>>>>>>
>>>>>>>>> On Wed, 2008-03-12 at 11:12 -0400, Tony Hosking wrote:
>>>>>>>>>> This is probably a result of conservatism for
>>>>>>>>>> references
>>>>>>>>>> from
>>>>>>>>>> thread
>>>>>>>>>> stacks which results in spurious retention.  It may be
>>>>>>>>>> necessary
>>>>>>>>>> to be
>>>>>>>>>> more careful about what references are held on the
>>>>>>>>>> stack
>>>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>> threads implementation (and elsewhere in your code).
>>>>>>>>>> Until we
>>>>>>>>>> diagnose the place where objects are being retained it
>>>>>>>>>> will be
>>>>>>>>>> hard to
>>>>>>>>>> pinpoint.  You should realize that weak refs are a
>>>>>>>>>> problem
>>>>>>>>>> in
>>>>>>>>>> many
>>>>>>>>>> language implementations, not just Modula-3.  I look
>>>>>>>>>> forward
>>>>>>>>>> to
>>>>>>>>>> hearing more definitively from you.
>>>>>>>>>>
>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>> Science |
>>>>>>>>>> Purdue
>>>>>>>>>> University
>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 |
>>>>>>>>>> USA
>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mar 12, 2008, at 4:21 AM, Dragiša Durić wrote:
>>>>>>>>>>
>>>>>>>>>>> Problem in my case can be thoroughness of GC...
>>>>>>>>>>> Because
>>>>>>>>>>> it's
>>>>>>>>>>> linked
>>>>>>>>>>> list of WeakRef-ed objects, it's always first
>>>>>>>>>>> element in
>>>>>>>>>>> list to
>>>>>>>>>>> become
>>>>>>>>>>> unreachable first - all others are reachable at
>>>>>>>>>>> least
>>>>>>>>>>> through
>>>>>>>>>>> previous
>>>>>>>>>>> (in list) WeakRef-ed object. So, behaviour I have
>>>>>>>>>>> observed
>>>>>>>>>>> can
>>>>>>>>>>> be
>>>>>>>>>>> because of some thousands of freeable objects, that
>>>>>>>>>>> one
>>>>>>>>>>> is
>>>>>>>>>>> missed
>>>>>>>>>>> too
>>>>>>>>>>> often.
>>>>>>>>>>>
>>>>>>>>>>> I have other object being cleaned and counted...
>>>>>>>>>>> There
>>>>>>>>>>> also
>>>>>>>>>>> cleanup
>>>>>>>>>>> invocation looks like it's lagging too much. I'll
>>>>>>>>>>> come
>>>>>>>>>>> with
>>>>>>>>>>> numbers
>>>>>>>>>>> after I have more tests.
>>>>>>>>>>>
>>>>>>>>>>> dd
>>>>>>>>>>>
>>>>>>>>>>> On Tue, 2008-03-11 at 09:50 -0400, Tony Hosking
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Sounds like we need some sort of testcase.  Would
>>>>>>>>>>>> you
>>>>>>>>>>>> be
>>>>>>>>>>>> able
>>>>>>>>>>>> to
>>>>>>>>>>>> devise one?  It will be hard to make it
>>>>>>>>>>>> deterministic,
>>>>>>>>>>>> but
>>>>>>>>>>>> at
>>>>>>>>>>>> least we
>>>>>>>>>>>> should see a non-zero cleanup count.
>>>>>>>>>>>>
>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>> Science |
>>>>>>>>>>>> Purdue
>>>>>>>>>>>> University
>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>> 47907 |
>>>>>>>>>>>> USA
>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mar 11, 2008, at 4:03 AM, Dragiša Durić wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I have single linked list that I use to send
>>>>>>>>>>>>> messages to
>>>>>>>>>>>>> many
>>>>>>>>>>>>> threads.
>>>>>>>>>>>>> It's end is locked for addition, and current end
>>>>>>>>>>>>> is
>>>>>>>>>>>>> given to
>>>>>>>>>>>>> each
>>>>>>>>>>>>> new
>>>>>>>>>>>>> client connecting. This way, all client are
>>>>>>>>>>>>> going
>>>>>>>>>>>>> towards
>>>>>>>>>>>>> end of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> list so it's head becomes unreferenced and goes
>>>>>>>>>>>>> away
>>>>>>>>>>>>> by
>>>>>>>>>>>>> GC.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Except it does not.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've added WeakRef cleanup and sequential id's
>>>>>>>>>>>>> so I
>>>>>>>>>>>>> can
>>>>>>>>>>>>> record
>>>>>>>>>>>>> maximum
>>>>>>>>>>>>> freed id for checking. No single cleanup
>>>>>>>>>>>>> happens.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll cross check my code and count my references
>>>>>>>>>>>>> from
>>>>>>>>>>>>> application
>>>>>>>>>>>>> side,
>>>>>>>>>>>>> but maybe someone else has similar problems?
>>>>>>>>>>>>>
>>>>>>>>>>>>> dd
>>>>>>>>>>>>> -- 
>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> -- 
>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>
>>>>>>
>>>>> -- 
>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>
>>>>
>>> -- 
>>> Dragiša Durić <dragisha at m3w.org>
>>>
>>
> -- 
> Dragiša Durić <dragisha at m3w.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20080313/bdc6ffa7/attachment-0002.html>