[M3devel] WeakRef mechanism alive and well? cm3-cvshead
Tony Hosking
hosking at cs.purdue.edu
Fri Mar 14 02:16:16 CET 2008
http://www.cs.purdue.edu/homes/hosking/papers/ismm06.pdf
On Mar 13, 2008, at 7:05 PM, Dragiša Durić wrote:
> Is your work on GC covered somewhere else except source, comments and
> this list?
>
> dd
>
> On Thu, 2008-03-13 at 18:35 -0400, Tony Hosking wrote:
>> Ah, the fact that you needed to be aggressive suggests that
>> conservatism may be the culprit. I keep meaning one day to make sure
>> that we do not can objects on pinned pages unless we know that object
>> is definitely alive.
>>
>> On Mar 13, 2008, at 6:21 PM, Dragiša Durić wrote:
>>
>>> I see... What bugs me now is a fact about my flow... I am already
>>> doing
>>> this, and there is always few thousand objects uncleaned from of
>>> type
>>> Message... I'll follow this through, to crosscheck everything again.
>>>
>>> What is nice, after I made this agressive RTCollector.Collect
>>> "policy",
>>> my memory footprint remains same for many hours... It was swelling
>>> before.
>>>
>>> Thanks,
>>> dd
>>>
>>> On Thu, 2008-03-13 at 18:15 -0400, Tony Hosking wrote:
>>>> No, when a page is pinned because one of its objects is directly
>>>> referenced from a thread stack then all objects on that page, both
>>>> live and dead, are retained. The collector does not move objects
>>>> to
>>>> or from pinned pages. The objects on the page can only be freed
>>>> when
>>>> the page is no longer pinned by a direct reference from any thread
>>>> stack. Moral of the story is to NIL references aggressively in
>>>> local
>>>> variables.
>>>>
>>>> On Mar 13, 2008, at 6:09 PM, Dragiša Durić wrote:
>>>>
>>>>> When object becomes unreachable, and it sits on page with other
>>>>> reachable objects, it's pinned?
>>>>>
>>>>> What I asked is... As garbage collector moves objects around,
>>>>> would
>>>>> it
>>>>> pack reachable objects over unreachable ones in these pages? Thus
>>>>> kicking them out of pages where they're pinned?
>>>>>
>>>>> dd
>>>>>
>>>>> On Thu, 2008-03-13 at 18:03 -0400, Tony Hosking wrote:
>>>>>> I'm not sure I understand what you mean?
>>>>>>
>>>>>> On Mar 13, 2008, at 5:46 PM, Dragiša Durić wrote:
>>>>>>
>>>>>>> Are objects on pages prepacked/moved?
>>>>>>>
>>>>>>> On Thu, 2008-03-13 at 17:41 -0400, Tony Hosking wrote:
>>>>>>>> The thing about conservatism is that even if you NIL *all*
>>>>>>>> references
>>>>>>>> to an object that just happens to lie on a pinned page, that
>>>>>>>> object
>>>>>>>> (and objects reachable from it) will be retained until the page
>>>>>>>> is no
>>>>>>>> longer pinned.
>>>>>>>>
>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>> Purdue
>>>>>>>> University
>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mar 13, 2008, at 1:43 PM, Dragiša Durić wrote:
>>>>>>>>
>>>>>>>>> I understand transitiveness :), but I've made big effort to
>>>>>>>>> escape
>>>>>>>>> that.
>>>>>>>>>
>>>>>>>>> Lifetime of thread equals lifetime of Client.T instance, and
>>>>>>>>> that
>>>>>>>>> instance is kept in thread's closure, and also in local
>>>>>>>>> variable.
>>>>>>>>> Both
>>>>>>>>> are set to NIL before thread termination - all terminations
>>>>>>>>> and
>>>>>>>>> joins
>>>>>>>>> happen and are counted and shown.
>>>>>>>>>
>>>>>>>>> Pointers to Message objects are kept in two places. Hub.T
>>>>>>>>> keeps
>>>>>>>>> pointer
>>>>>>>>> to list tail, and each Client.T contains pointer to it's
>>>>>>>>> current
>>>>>>>>> element. That pointer is forwarded as Message's are added to
>>>>>>>>> end. No
>>>>>>>>> temporary variable, not even WITH substitution is used. And
>>>>>>>>> these
>>>>>>>>> threads are ones dying with their Client.T's.
>>>>>>>>>
>>>>>>>>> "head" is new mechanism used to unlink all elements with id
>>>>>>>>> less
>>>>>>>>> than
>>>>>>>>> minimal id used in active Client.T instances. Unlinked totally
>>>>>>>>> and
>>>>>>>>> all
>>>>>>>>> as one free for garbage collector to finish them.
>>>>>>>>>
>>>>>>>>> I see no transition.
>>>>>>>>>
>>>>>>>>> dd
>>>>>>>>>
>>>>>>>>> On Thu, 2008-03-13 at 12:45 -0400, Tony Hosking wrote:
>>>>>>>>>> Realizing that conservative roots collectors like ours can
>>>>>>>>>> retain
>>>>>>>>>> data
>>>>>>>>>> unnecessarily. If a thread stack refers to a page that
>>>>>>>>>> holds a
>>>>>>>>>> reference that (transitively) refers to some otherwise dead
>>>>>>>>>> object
>>>>>>>>>> then that object will be retained. We say that the page is
>>>>>>>>>> pinned.
>>>>>>>>>> Smartening up the collector to ignore dead objects in pinned
>>>>>>>>>> pages
>>>>>>>>>> is
>>>>>>>>>> possible, but not currently implemented.
>>>>>>>>>>
>>>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>>>> Purdue
>>>>>>>>>> University
>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mar 13, 2008, at 12:28 PM, Dragiša Durić wrote:
>>>>>>>>>>
>>>>>>>>>>> [17:25:19] <Hubetta> 117 users, 1 ops and 1 unresolved;
>>>>>>>>>>> minId =
>>>>>>>>>>> 18617; thrs = 120
>>>>>>>>>>> [17:25:19] <Hubetta> started = 663; cleaned = 452; nilled =
>>>>>>>>>>> 543;
>>>>>>>>>>> joined = 543
>>>>>>>>>>> [17:25:19] <Hubetta> Dangling thrs-a-b-n = 1
>>>>>>>>>>> [17:25:19] <Hubetta> Live queue end is 18621 (head = 18616),
>>>>>>>>>>> maxClean = 18611, cleaned = 16583
>>>>>>>>>>>
>>>>>>>>>>> About one hour uptime, after I added RTCollector.Collect()
>>>>>>>>>>> call
>>>>>>>>>>> every 20
>>>>>>>>>>> seconds.
>>>>>>>>>>>
>>>>>>>>>>> dd
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 2008-03-13 at 10:54 -0400, Tony Hosking wrote:
>>>>>>>>>>>> Yes. RTCollectorSRC.FinishCollection;
>>>>>>>>>>>> RTCollectorSRC.StartCollection. This will finish the
>>>>>>>>>>>> current
>>>>>>>>>>>> (possibly minor) collection, and start a major collection.
>>>>>>>>>>>> The
>>>>>>>>>>>> major
>>>>>>>>>>>> collection must finish before you can guarantee garbage has
>>>>>>>>>>>> been
>>>>>>>>>>>> freed, so you may want to RTCollectorSRC.FinishCollection.
>>>>>>>>>>>> This
>>>>>>>>>>>> sequence is packaged as RTCollector.Collect.
>>>>>>>>>>>>
>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>>>>>> Purdue
>>>>>>>>>>>> University
>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 | USA
>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mar 13, 2008, at 10:22 AM, Dragiša Durić wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Can I force "major collection" ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> dd
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 2008-03-13 at 10:08 -0400, Tony Hosking wrote:
>>>>>>>>>>>>>> Indeed! Can you devise a testcase? Note that the
>>>>>>>>>>>>>> generational
>>>>>>>>>>>>>> collector will retain old objects for some time until a
>>>>>>>>>>>>>> major
>>>>>>>>>>>>>> collection occurs,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer Science |
>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>> University
>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 |
>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mar 13, 2008, at 9:56 AM, Dragiša Durić wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And not only that, their stackbase is also set to NIL,
>>>>>>>>>>>>>>> meaning
>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>> stack is not regarded in any way during future
>>>>>>>>>>>>>>> collections -
>>>>>>>>>>>>>>> meaning
>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>> local variables are forgotten once apply method returns.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That way, "spuriousness" is not an issue, once thread
>>>>>>>>>>>>>>> returns?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, 2008-03-13 at 09:46 -0400, Tony Hosking wrote:
>>>>>>>>>>>>>>>> They should get unlinked from the global ring.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>>>> University
>>>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN 47907 |
>>>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mar 13, 2008, at 6:16 AM, Dragiša Durić wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What happens to thread stacks once threads are
>>>>>>>>>>>>>>>>> Join-ed?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, 2008-03-12 at 11:12 -0400, Tony Hosking
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> This is probably a result of conservatism for
>>>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>> stacks which results in spurious retention. It
>>>>>>>>>>>>>>>>>> may be
>>>>>>>>>>>>>>>>>> necessary
>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>> more careful about what references are held on the
>>>>>>>>>>>>>>>>>> stack
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> threads implementation (and elsewhere in your
>>>>>>>>>>>>>>>>>> code).
>>>>>>>>>>>>>>>>>> Until we
>>>>>>>>>>>>>>>>>> diagnose the place where objects are being
>>>>>>>>>>>>>>>>>> retained it
>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>> hard to
>>>>>>>>>>>>>>>>>> pinpoint. You should realize that weak refs are a
>>>>>>>>>>>>>>>>>> problem
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>> language implementations, not just Modula-3. I
>>>>>>>>>>>>>>>>>> look
>>>>>>>>>>>>>>>>>> forward
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> hearing more definitively from you.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Antony Hosking | Associate Professor | Computer
>>>>>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>>>>>> University
>>>>>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>>>>>>>> 47907 |
>>>>>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427 5484
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mar 12, 2008, at 4:21 AM, Dragiša Durić wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Problem in my case can be thoroughness of GC...
>>>>>>>>>>>>>>>>>>> Because
>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>>>>>> list of WeakRef-ed objects, it's always first
>>>>>>>>>>>>>>>>>>> element in
>>>>>>>>>>>>>>>>>>> list to
>>>>>>>>>>>>>>>>>>> become
>>>>>>>>>>>>>>>>>>> unreachable first - all others are reachable at
>>>>>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>> previous
>>>>>>>>>>>>>>>>>>> (in list) WeakRef-ed object. So, behaviour I
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> observed
>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> because of some thousands of freeable objects,
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> missed
>>>>>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>>>>>> often.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have other object being cleaned and counted...
>>>>>>>>>>>>>>>>>>> There
>>>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>> cleanup
>>>>>>>>>>>>>>>>>>> invocation looks like it's lagging too much.
>>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>>> come
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> numbers
>>>>>>>>>>>>>>>>>>> after I have more tests.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, 2008-03-11 at 09:50 -0400, Tony Hosking
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> Sounds like we need some sort of testcase.
>>>>>>>>>>>>>>>>>>>> Would
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> devise one? It will be hard to make it
>>>>>>>>>>>>>>>>>>>> deterministic,
>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> least we
>>>>>>>>>>>>>>>>>>>> should see a non-zero cleanup count.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Antony Hosking | Associate Professor |
>>>>>>>>>>>>>>>>>>>> Computer
>>>>>>>>>>>>>>>>>>>> Science |
>>>>>>>>>>>>>>>>>>>> Purdue
>>>>>>>>>>>>>>>>>>>> University
>>>>>>>>>>>>>>>>>>>> 305 N. University Street | West Lafayette | IN
>>>>>>>>>>>>>>>>>>>> 47907 |
>>>>>>>>>>>>>>>>>>>> USA
>>>>>>>>>>>>>>>>>>>> Office +1 765 494 6001 | Mobile +1 765 427
>>>>>>>>>>>>>>>>>>>> 5484
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mar 11, 2008, at 4:03 AM, Dragiša Durić
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I have single linked list that I use to send
>>>>>>>>>>>>>>>>>>>>> messages to
>>>>>>>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>>>>> threads.
>>>>>>>>>>>>>>>>>>>>> It's end is locked for addition, and current
>>>>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> given to
>>>>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>>> client connecting. This way, all client are
>>>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>> towards
>>>>>>>>>>>>>>>>>>>>> end of
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> list so it's head becomes unreferenced and
>>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>>> away
>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>> GC.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Except it does not.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I've added WeakRef cleanup and sequential
>>>>>>>>>>>>>>>>>>>>> id's
>>>>>>>>>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> record
>>>>>>>>>>>>>>>>>>>>> maximum
>>>>>>>>>>>>>>>>>>>>> freed id for checking. No single cleanup
>>>>>>>>>>>>>>>>>>>>> happens.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'll cross check my code and count my
>>>>>>>>>>>>>>>>>>>>> references
>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>> application
>>>>>>>>>>>>>>>>>>>>> side,
>>>>>>>>>>>>>>>>>>>>> but maybe someone else has similar problems?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> dd
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>>>
>>>>> --
>>>>> Dragiša Durić <dragisha at m3w.org>
>>>>
>>> --
>>> Dragiša Durić <dragisha at m3w.org>
>>
> --
> Dragiša Durić <dragisha at m3w.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20080313/f661453d/attachment-0002.html>
More information about the M3devel
mailing list