[M3devel] WeakRef mechanism alive and well? cm3-cvshead

[Tony Hosking]

I'm not sure I understand what you mean?

On Mar 13, 2008, at 5:46 PM, Dragiša Durić wrote:

> Are objects on pages prepacked/moved?
>
> On Thu, 2008-03-13 at 17:41 -0400, Tony Hosking wrote:
>> The thing about conservatism is that even if you NIL *all* references
>> to an object that just happens to lie on a pinned page, that object
>> (and objects reachable from it) will be retained until the page is no
>> longer pinned.
>>
>> Antony Hosking | Associate Professor | Computer Science | Purdue
>> University
>> 305 N. University Street | West Lafayette | IN 47907 | USA
>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>
>>
>>
>>
>>
>> On Mar 13, 2008, at 1:43 PM, Dragiša Durić wrote:
>>
>>> I understand transitiveness :), but I've made big effort to escape
>>> that.
>>>
>>> Lifetime of thread equals lifetime of Client.T instance, and that
>>> instance is kept in thread's closure, and also in local variable.
>>> Both
>>> are set to NIL before thread termination - all terminations and
>>> joins
>>> happen and are counted and shown.
>>>
>>> Pointers to Message objects are kept in two places. Hub.T keeps
>>> pointer
>>> to list tail, and each Client.T contains pointer to it's current
>>> element. That pointer is forwarded as Message's are added to end. No
>>> temporary variable, not even WITH substitution is used. And these
>>> threads are ones dying with  their Client.T's.
>>>
>>> "head" is new mechanism used to unlink all elements with id less
>>> than
>>> minimal id used in active Client.T instances. Unlinked totally and
>>> all
>>> as one free for garbage collector to finish them.
>>>
>>> I see no transition.
>>>
>>> dd
>>>
>>> On Thu, 2008-03-13 at 12:45 -0400, Tony Hosking wrote:
>>>> Realizing that conservative roots collectors like ours can retain
>>>> data
>>>> unnecessarily.  If a thread stack refers to a page that holds a
>>>> reference that (transitively) refers to some otherwise dead object
>>>> then that object will be retained.  We say that the page is
>>>> pinned.
>>>> Smartening up the collector to ignore dead objects in pinned pages
>>>> is
>>>> possible, but not currently implemented.
>>>>
>>>> Antony Hosking | Associate Professor | Computer Science | Purdue
>>>> University
>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mar 13, 2008, at 12:28 PM, Dragiša Durić wrote:
>>>>
>>>>> [17:25:19] <Hubetta> 117 users, 1 ops and 1 unresolved; minId =
>>>>> 18617; thrs = 120
>>>>> [17:25:19] <Hubetta> started = 663; cleaned = 452; nilled = 543;
>>>>> joined = 543
>>>>> [17:25:19] <Hubetta> Dangling thrs-a-b-n = 1
>>>>> [17:25:19] <Hubetta> Live queue end is 18621 (head = 18616),
>>>>> maxClean = 18611, cleaned = 16583
>>>>>
>>>>> About one hour uptime, after I added RTCollector.Collect() call
>>>>> every 20
>>>>> seconds.
>>>>>
>>>>> dd
>>>>>
>>>>> On Thu, 2008-03-13 at 10:54 -0400, Tony Hosking wrote:
>>>>>> Yes.  RTCollectorSRC.FinishCollection;
>>>>>> RTCollectorSRC.StartCollection.  This will finish the current
>>>>>> (possibly minor) collection, and start a major collection.
>>>>>> The
>>>>>> major
>>>>>> collection must finish before you can guarantee garbage has
>>>>>> been
>>>>>> freed, so you may want to RTCollectorSRC.FinishCollection.
>>>>>> This
>>>>>> sequence is packaged as RTCollector.Collect.
>>>>>>
>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>> Purdue
>>>>>> University
>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mar 13, 2008, at 10:22 AM, Dragiša Durić wrote:
>>>>>>
>>>>>>> Can I force "major collection" ?
>>>>>>>
>>>>>>> dd
>>>>>>>
>>>>>>> On Thu, 2008-03-13 at 10:08 -0400, Tony Hosking wrote:
>>>>>>>> Indeed! Can you devise a testcase?  Note that the
>>>>>>>> generational
>>>>>>>> collector will retain old objects for some time until a
>>>>>>>> major
>>>>>>>> collection occurs,
>>>>>>>>
>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>> Purdue
>>>>>>>> University
>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mar 13, 2008, at 9:56 AM, Dragiša Durić wrote:
>>>>>>>>
>>>>>>>>> And not only that, their stackbase is also set to NIL,
>>>>>>>>> meaning
>>>>>>>>> their
>>>>>>>>> stack is not regarded in any way during future
>>>>>>>>> collections -
>>>>>>>>> meaning
>>>>>>>>> all
>>>>>>>>> local variables are forgotten once apply method returns.
>>>>>>>>>
>>>>>>>>> That way, "spuriousness"  is not an issue, once thread
>>>>>>>>> returns?
>>>>>>>>>
>>>>>>>>> dd
>>>>>>>>>
>>>>>>>>> On Thu, 2008-03-13 at 09:46 -0400, Tony Hosking wrote:
>>>>>>>>>> They should get unlinked from the global ring.
>>>>>>>>>>
>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>> Science |
>>>>>>>>>> Purdue
>>>>>>>>>> University
>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 |
>>>>>>>>>> USA
>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mar 13, 2008, at 6:16 AM, Dragiša Durić wrote:
>>>>>>>>>>
>>>>>>>>>>> What happens to thread stacks once threads are
>>>>>>>>>>> Join-ed?
>>>>>>>>>>>
>>>>>>>>>>> dd
>>>>>>>>>>>
>>>>>>>>>>> On Wed, 2008-03-12 at 11:12 -0400, Tony Hosking
>>>>>>>>>>> wrote:
>>>>>>>>>>>> This is probably a result of conservatism for
>>>>>>>>>>>> references
>>>>>>>>>>>> from
>>>>>>>>>>>> thread
>>>>>>>>>>>> stacks which results in spurious retention.  It
>>>>>>>>>>>> may be
>>>>>>>>>>>> necessary
>>>>>>>>>>>> to be
>>>>>>>>>>>> more careful about what references are held on the
>>>>>>>>>>>> stack
>>>>>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>> threads implementation (and elsewhere in your
>>>>>>>>>>>> code).
>>>>>>>>>>>> Until we
>>>>>>>>>>>> diagnose the place where objects are being
>>>>>>>>>>>> retained it
>>>>>>>>>>>> will be
>>>>>>>>>>>> hard to
>>>>>>>>>>>> pinpoint.  You should realize that weak refs are a
>>>>>>>>>>>> problem
>>>>>>>>>>>> in
>>>>>>>>>>>> many
>>>>>>>>>>>> language implementations, not just Modula-3.  I
>>>>>>>>>>>> look
>>>>>>>>>>>> forward
>>>>>>>>>>>> to
>>>>>>>>>>>> hearing more definitively from you.
>>>>>>>>>>>>
>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>> Science |
>>>>>>>>>>>> Purdue
>>>>>>>>>>>> University
>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>> 47907 |
>>>>>>>>>>>> USA
>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mar 12, 2008, at 4:21 AM, Dragiša Durić wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Problem in my case can be thoroughness of GC...
>>>>>>>>>>>>> Because
>>>>>>>>>>>>> it's
>>>>>>>>>>>>> linked
>>>>>>>>>>>>> list of WeakRef-ed objects, it's always first
>>>>>>>>>>>>> element in
>>>>>>>>>>>>> list to
>>>>>>>>>>>>> become
>>>>>>>>>>>>> unreachable first - all others are reachable at
>>>>>>>>>>>>> least
>>>>>>>>>>>>> through
>>>>>>>>>>>>> previous
>>>>>>>>>>>>> (in list) WeakRef-ed object. So, behaviour I
>>>>>>>>>>>>> have
>>>>>>>>>>>>> observed
>>>>>>>>>>>>> can
>>>>>>>>>>>>> be
>>>>>>>>>>>>> because of some thousands of freeable objects,
>>>>>>>>>>>>> that
>>>>>>>>>>>>> one
>>>>>>>>>>>>> is
>>>>>>>>>>>>> missed
>>>>>>>>>>>>> too
>>>>>>>>>>>>> often.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have other object being cleaned and counted...
>>>>>>>>>>>>> There
>>>>>>>>>>>>> also
>>>>>>>>>>>>> cleanup
>>>>>>>>>>>>> invocation looks like it's lagging too much.
>>>>>>>>>>>>> I'll
>>>>>>>>>>>>> come
>>>>>>>>>>>>> with
>>>>>>>>>>>>> numbers
>>>>>>>>>>>>> after I have more tests.
>>>>>>>>>>>>>
>>>>>>>>>>>>> dd
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 2008-03-11 at 09:50 -0400, Tony Hosking
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Sounds like we need some sort of testcase.
>>>>>>>>>>>>>> Would
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>> able
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> devise one?  It will be hard to make it
>>>>>>>>>>>>>> deterministic,
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> least we
>>>>>>>>>>>>>> should see a non-zero cleanup count.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Antony Hosking | Associate Professor |
>>>>>>>>>>>>>> Computer
>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>> University
>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>>>> 47907 |
>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427
>>>>>>>>>>>>>> 5484
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mar 11, 2008, at 4:03 AM, Dragiša Durić
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have single linked list that I use to send
>>>>>>>>>>>>>>> messages to
>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>> threads.
>>>>>>>>>>>>>>> It's end is locked for addition, and current
>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> given to
>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> client connecting. This way, all client are
>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>> end of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> list so it's head becomes unreferenced and
>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>> away
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> GC.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Except it does not.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've added WeakRef cleanup and sequential
>>>>>>>>>>>>>>> id's
>>>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> record
>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>> freed id for checking. No single cleanup
>>>>>>>>>>>>>>> happens.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'll cross check my code and count my
>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>> application
>>>>>>>>>>>>>>> side,
>>>>>>>>>>>>>>> but maybe someone else has similar problems?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>> -- 
>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> -- 
>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> -- 
>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>
>>>>>>>>
>>>>>>> -- 
>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>
>>>>>>
>>>>> -- 
>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>
>>>>
>>> -- 
>>> Dragiša Durić <dragisha at m3w.org>
>>>
>>
> -- 
> Dragiša Durić <dragisha at m3w.org>