[M3devel] WeakRef mechanism alive and well? cm3-cvshead
Tony Hosking
hosking at cs.purdue.edu
Thu Mar 13 23:15:59 CET 2008
No, when a page is pinned because one of its objects is directly
referenced from a thread stack then all objects on that page, both
live and dead, are retained. The collector does not move objects to
or from pinned pages. The objects on the page can only be freed when
the page is no longer pinned by a direct reference from any thread
stack. Moral of the story is to NIL references aggressively in local
variables.
On Mar 13, 2008, at 6:09 PM, Dragiša Durić wrote:
> When object becomes unreachable, and it sits on page with other
> reachable objects, it's pinned?
>
> What I asked is... As garbage collector moves objects around, would
> it
> pack reachable objects over unreachable ones in these pages? Thus
> kicking them out of pages where they're pinned?
>
> dd
>
> On Thu, 2008-03-13 at 18:03 -0400, Tony Hosking wrote:
>> I'm not sure I understand what you mean?
>>
>> On Mar 13, 2008, at 5:46 PM, Dragiša Durić wrote:
>>
>>> Are objects on pages prepacked/moved?
>>>
>>> On Thu, 2008-03-13 at 17:41 -0400, Tony Hosking wrote:
>>>> The thing about conservatism is that even if you NIL *all*
>>>> references
>>>> to an object that just happens to lie on a pinned page, that object
>>>> (and objects reachable from it) will be retained until the page
>>>> is no
>>>> longer pinned.
>>>>
>>>> Antony Hosking | Associate Professor | Computer Science | Purdue
>>>> University
>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mar 13, 2008, at 1:43 PM, Dragiša Durić wrote:
>>>>
>>>>> I understand transitiveness :), but I've made big effort to escape
>>>>> that.
>>>>>
>>>>> Lifetime of thread equals lifetime of Client.T instance, and that
>>>>> instance is kept in thread's closure, and also in local variable.
>>>>> Both
>>>>> are set to NIL before thread termination - all terminations and
>>>>> joins
>>>>> happen and are counted and shown.
>>>>>
>>>>> Pointers to Message objects are kept in two places. Hub.T keeps
>>>>> pointer
>>>>> to list tail, and each Client.T contains pointer to it's current
>>>>> element. That pointer is forwarded as Message's are added to
>>>>> end. No
>>>>> temporary variable, not even WITH substitution is used. And these
>>>>> threads are ones dying with their Client.T's.
>>>>>
>>>>> "head" is new mechanism used to unlink all elements with id less
>>>>> than
>>>>> minimal id used in active Client.T instances. Unlinked totally and
>>>>> all
>>>>> as one free for garbage collector to finish them.
>>>>>
>>>>> I see no transition.
>>>>>
>>>>> dd
>>>>>
>>>>> On Thu, 2008-03-13 at 12:45 -0400, Tony Hosking wrote:
>>>>>> Realizing that conservative roots collectors like ours can retain
>>>>>> data
>>>>>> unnecessarily. If a thread stack refers to a page that holds a
>>>>>> reference that (transitively) refers to some otherwise dead
>>>>>> object
>>>>>> then that object will be retained. We say that the page is
>>>>>> pinned.
>>>>>> Smartening up the collector to ignore dead objects in pinned
>>>>>> pages
>>>>>> is
>>>>>> possible, but not currently implemented.
>>>>>>
>>>>>> Antony Hosking | Associate Professor | Computer Science | Purdue
>>>>>> University
>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mar 13, 2008, at 12:28 PM, Dragiša Durić wrote:
>>>>>>
>>>>>>> [17:25:19] <Hubetta> 117 users, 1 ops and 1 unresolved; minId =
>>>>>>> 18617; thrs = 120
>>>>>>> [17:25:19] <Hubetta> started = 663; cleaned = 452; nilled = 543;
>>>>>>> joined = 543
>>>>>>> [17:25:19] <Hubetta> Dangling thrs-a-b-n = 1
>>>>>>> [17:25:19] <Hubetta> Live queue end is 18621 (head = 18616),
>>>>>>> maxClean = 18611, cleaned = 16583
>>>>>>>
>>>>>>> About one hour uptime, after I added RTCollector.Collect() call
>>>>>>> every 20
>>>>>>> seconds.
>>>>>>>
>>>>>>> dd
>>>>>>>
>>>>>>> On Thu, 2008-03-13 at 10:54 -0400, Tony Hosking wrote:
>>>>>>>> Yes. RTCollectorSRC.FinishCollection;
>>>>>>>> RTCollectorSRC.StartCollection. This will finish the current
>>>>>>>> (possibly minor) collection, and start a major collection.
>>>>>>>> The
>>>>>>>> major
>>>>>>>> collection must finish before you can guarantee garbage has
>>>>>>>> been
>>>>>>>> freed, so you may want to RTCollectorSRC.FinishCollection.
>>>>>>>> This
>>>>>>>> sequence is packaged as RTCollector.Collect.
>>>>>>>>
>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>> Purdue
>>>>>>>> University
>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mar 13, 2008, at 10:22 AM, Dragiša Durić wrote:
>>>>>>>>
>>>>>>>>> Can I force "major collection" ?
>>>>>>>>>
>>>>>>>>> dd
>>>>>>>>>
>>>>>>>>> On Thu, 2008-03-13 at 10:08 -0400, Tony Hosking wrote:
>>>>>>>>>> Indeed! Can you devise a testcase? Note that the
>>>>>>>>>> generational
>>>>>>>>>> collector will retain old objects for some time until a
>>>>>>>>>> major
>>>>>>>>>> collection occurs,
>>>>>>>>>>
>>>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>>>> Purdue
>>>>>>>>>> University
>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mar 13, 2008, at 9:56 AM, Dragiša Durić wrote:
>>>>>>>>>>
>>>>>>>>>>> And not only that, their stackbase is also set to NIL,
>>>>>>>>>>> meaning
>>>>>>>>>>> their
>>>>>>>>>>> stack is not regarded in any way during future
>>>>>>>>>>> collections -
>>>>>>>>>>> meaning
>>>>>>>>>>> all
>>>>>>>>>>> local variables are forgotten once apply method returns.
>>>>>>>>>>>
>>>>>>>>>>> That way, "spuriousness" is not an issue, once thread
>>>>>>>>>>> returns?
>>>>>>>>>>>
>>>>>>>>>>> dd
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 2008-03-13 at 09:46 -0400, Tony Hosking wrote:
>>>>>>>>>>>> They should get unlinked from the global ring.
>>>>>>>>>>>>
>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>> Science |
>>>>>>>>>>>> Purdue
>>>>>>>>>>>> University
>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 |
>>>>>>>>>>>> USA
>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mar 13, 2008, at 6:16 AM, Dragiša Durić wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> What happens to thread stacks once threads are
>>>>>>>>>>>>> Join-ed?
>>>>>>>>>>>>>
>>>>>>>>>>>>> dd
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, 2008-03-12 at 11:12 -0400, Tony Hosking
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> This is probably a result of conservatism for
>>>>>>>>>>>>>> references
>>>>>>>>>>>>>> from
>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>> stacks which results in spurious retention. It
>>>>>>>>>>>>>> may be
>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>> more careful about what references are held on the
>>>>>>>>>>>>>> stack
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> threads implementation (and elsewhere in your
>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>> Until we
>>>>>>>>>>>>>> diagnose the place where objects are being
>>>>>>>>>>>>>> retained it
>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>> hard to
>>>>>>>>>>>>>> pinpoint. You should realize that weak refs are a
>>>>>>>>>>>>>> problem
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> many
>>>>>>>>>>>>>> language implementations, not just Modula-3. I
>>>>>>>>>>>>>> look
>>>>>>>>>>>>>> forward
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> hearing more definitively from you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>> University
>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>>>> 47907 |
>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mar 12, 2008, at 4:21 AM, Dragiša Durić wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Problem in my case can be thoroughness of GC...
>>>>>>>>>>>>>>> Because
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>> list of WeakRef-ed objects, it's always first
>>>>>>>>>>>>>>> element in
>>>>>>>>>>>>>>> list to
>>>>>>>>>>>>>>> become
>>>>>>>>>>>>>>> unreachable first - all others are reachable at
>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>> previous
>>>>>>>>>>>>>>> (in list) WeakRef-ed object. So, behaviour I
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> observed
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> because of some thousands of freeable objects,
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> missed
>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>> often.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have other object being cleaned and counted...
>>>>>>>>>>>>>>> There
>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>> cleanup
>>>>>>>>>>>>>>> invocation looks like it's lagging too much.
>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>> come
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> numbers
>>>>>>>>>>>>>>> after I have more tests.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 2008-03-11 at 09:50 -0400, Tony Hosking
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Sounds like we need some sort of testcase.
>>>>>>>>>>>>>>>> Would
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> devise one? It will be hard to make it
>>>>>>>>>>>>>>>> deterministic,
>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> least we
>>>>>>>>>>>>>>>> should see a non-zero cleanup count.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Antony Hosking | Associate Professor |
>>>>>>>>>>>>>>>> Computer
>>>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>>>> University
>>>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>>>>>> 47907 |
>>>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427
>>>>>>>>>>>>>>>> 5484
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mar 11, 2008, at 4:03 AM, Dragiša Durić
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have single linked list that I use to send
>>>>>>>>>>>>>>>>> messages to
>>>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>> threads.
>>>>>>>>>>>>>>>>> It's end is locked for addition, and current
>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> given to
>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> client connecting. This way, all client are
>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>> end of
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> list so it's head becomes unreferenced and
>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>> away
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> GC.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Except it does not.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've added WeakRef cleanup and sequential
>>>>>>>>>>>>>>>>> id's
>>>>>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> record
>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>> freed id for checking. No single cleanup
>>>>>>>>>>>>>>>>> happens.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'll cross check my code and count my
>>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> application
>>>>>>>>>>>>>>>>> side,
>>>>>>>>>>>>>>>>> but maybe someone else has similar problems?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>
>>>>>>
>>>>> --
>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>
>>>>
>>> --
>>> Dragiša Durić <dragisha at m3w.org>
>>
> --
> Dragiša Durić <dragisha at m3w.org>
More information about the M3devel
mailing list