[M3devel] WeakRef mechanism alive and well? cm3-cvshead

Thu Mar 13 18:43:19 CET 2008

I understand transitiveness :), but I've made big effort to escape that.

Lifetime of thread equals lifetime of Client.T instance, and that
instance is kept in thread's closure, and also in local variable. Both
are set to NIL before thread termination - all terminations and joins
happen and are counted and shown.

Pointers to Message objects are kept in two places. Hub.T keeps pointer
to list tail, and each Client.T contains pointer to it's current
element. That pointer is forwarded as Message's are added to end. No
temporary variable, not even WITH substitution is used. And these
threads are ones dying with  their Client.T's.

"head" is new mechanism used to unlink all elements with id less than
minimal id used in active Client.T instances. Unlinked totally and all
as one free for garbage collector to finish them.

I see no transition.

dd

On Thu, 2008-03-13 at 12:45 -0400, Tony Hosking wrote:
> Realizing that conservative roots collectors like ours can retain data
> unnecessarily.  If a thread stack refers to a page that holds a
> reference that (transitively) refers to some otherwise dead object
> then that object will be retained.  We say that the page is pinned.
>  Smartening up the collector to ignore dead objects in pinned pages is
> possible, but not currently implemented.
> 
> Antony Hosking | Associate Professor | Computer Science | Purdue
> University
> 305 N. University Street | West Lafayette | IN 47907 | USA
> Office +1 765 494 6001 | Mobile +1 765 427 5484
> 
> 
> 
> 
> 
> On Mar 13, 2008, at 12:28 PM, Dragiša Durić wrote:
> 
> > [17:25:19] <Hubetta> 117 users, 1 ops and 1 unresolved; minId =
> > 18617; thrs = 120
> > [17:25:19] <Hubetta> started = 663; cleaned = 452; nilled = 543;
> > joined = 543
> > [17:25:19] <Hubetta> Dangling thrs-a-b-n = 1
> > [17:25:19] <Hubetta> Live queue end is 18621 (head = 18616),
> > maxClean = 18611, cleaned = 16583
> > 
> > About one hour uptime, after I added RTCollector.Collect() call
> > every 20
> > seconds.
> > 
> > dd
> > 
> > On Thu, 2008-03-13 at 10:54 -0400, Tony Hosking wrote:
> > > Yes.  RTCollectorSRC.FinishCollection;
> > > RTCollectorSRC.StartCollection.  This will finish the current
> > > (possibly minor) collection, and start a major collection.  The
> > > major
> > > collection must finish before you can guarantee garbage has been
> > > freed, so you may want to RTCollectorSRC.FinishCollection.  This
> > > sequence is packaged as RTCollector.Collect.
> > > 
> > > Antony Hosking | Associate Professor | Computer Science | Purdue
> > > University
> > > 305 N. University Street | West Lafayette | IN 47907 | USA
> > > Office +1 765 494 6001 | Mobile +1 765 427 5484
> > > 
> > > 
> > > 
> > > 
> > > 
> > > On Mar 13, 2008, at 10:22 AM, Dragiša Durić wrote:
> > > 
> > > > Can I force "major collection" ?
> > > > 
> > > > dd
> > > > 
> > > > On Thu, 2008-03-13 at 10:08 -0400, Tony Hosking wrote:
> > > > > Indeed! Can you devise a testcase?  Note that the generational
> > > > > collector will retain old objects for some time until a major
> > > > > collection occurs,
> > > > > 
> > > > > Antony Hosking | Associate Professor | Computer Science |
> > > > > Purdue
> > > > > University
> > > > > 305 N. University Street | West Lafayette | IN 47907 | USA
> > > > > Office +1 765 494 6001 | Mobile +1 765 427 5484
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > On Mar 13, 2008, at 9:56 AM, Dragiša Durić wrote:
> > > > > 
> > > > > > And not only that, their stackbase is also set to NIL,
> > > > > > meaning
> > > > > > their
> > > > > > stack is not regarded in any way during future collections -
> > > > > > meaning
> > > > > > all
> > > > > > local variables are forgotten once apply method returns.
> > > > > > 
> > > > > > That way, "spuriousness"  is not an issue, once thread
> > > > > > returns?
> > > > > > 
> > > > > > dd
> > > > > > 
> > > > > > On Thu, 2008-03-13 at 09:46 -0400, Tony Hosking wrote:
> > > > > > > They should get unlinked from the global ring.
> > > > > > > 
> > > > > > > Antony Hosking | Associate Professor | Computer Science |
> > > > > > > Purdue
> > > > > > > University
> > > > > > > 305 N. University Street | West Lafayette | IN 47907 | USA
> > > > > > > Office +1 765 494 6001 | Mobile +1 765 427 5484
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > On Mar 13, 2008, at 6:16 AM, Dragiša Durić wrote:
> > > > > > > 
> > > > > > > > What happens to thread stacks once threads are Join-ed?
> > > > > > > > 
> > > > > > > > dd
> > > > > > > > 
> > > > > > > > On Wed, 2008-03-12 at 11:12 -0400, Tony Hosking wrote:
> > > > > > > > > This is probably a result of conservatism for
> > > > > > > > > references
> > > > > > > > > from
> > > > > > > > > thread
> > > > > > > > > stacks which results in spurious retention.  It may be
> > > > > > > > > necessary
> > > > > > > > > to be
> > > > > > > > > more careful about what references are held on the
> > > > > > > > > stack
> > > > > > > > > in
> > > > > > > > > the
> > > > > > > > > threads implementation (and elsewhere in your code).
> > > > > > > > > Until we
> > > > > > > > > diagnose the place where objects are being retained it
> > > > > > > > > will be
> > > > > > > > > hard to
> > > > > > > > > pinpoint.  You should realize that weak refs are a
> > > > > > > > > problem
> > > > > > > > > in
> > > > > > > > > many
> > > > > > > > > language implementations, not just Modula-3.  I look
> > > > > > > > > forward
> > > > > > > > > to
> > > > > > > > > hearing more definitively from you.
> > > > > > > > > 
> > > > > > > > > Antony Hosking | Associate Professor | Computer
> > > > > > > > > Science |
> > > > > > > > > Purdue
> > > > > > > > > University
> > > > > > > > > 305 N. University Street | West Lafayette | IN 47907 |
> > > > > > > > > USA
> > > > > > > > > Office +1 765 494 6001 | Mobile +1 765 427 5484
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On Mar 12, 2008, at 4:21 AM, Dragiša Durić wrote:
> > > > > > > > > 
> > > > > > > > > > Problem in my case can be thoroughness of GC...
> > > > > > > > > > Because
> > > > > > > > > > it's
> > > > > > > > > > linked
> > > > > > > > > > list of WeakRef-ed objects, it's always first
> > > > > > > > > > element in
> > > > > > > > > > list to
> > > > > > > > > > become
> > > > > > > > > > unreachable first - all others are reachable at
> > > > > > > > > > least
> > > > > > > > > > through
> > > > > > > > > > previous
> > > > > > > > > > (in list) WeakRef-ed object. So, behaviour I have
> > > > > > > > > > observed
> > > > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > > because of some thousands of freeable objects, that
> > > > > > > > > > one
> > > > > > > > > > is
> > > > > > > > > > missed
> > > > > > > > > > too
> > > > > > > > > > often.
> > > > > > > > > > 
> > > > > > > > > > I have other object being cleaned and counted...
> > > > > > > > > > There
> > > > > > > > > > also
> > > > > > > > > > cleanup
> > > > > > > > > > invocation looks like it's lagging too much. I'll
> > > > > > > > > > come
> > > > > > > > > > with
> > > > > > > > > > numbers
> > > > > > > > > > after I have more tests.
> > > > > > > > > > 
> > > > > > > > > > dd
> > > > > > > > > > 
> > > > > > > > > > On Tue, 2008-03-11 at 09:50 -0400, Tony Hosking
> > > > > > > > > > wrote:
> > > > > > > > > > > Sounds like we need some sort of testcase.  Would
> > > > > > > > > > > you
> > > > > > > > > > > be
> > > > > > > > > > > able
> > > > > > > > > > > to
> > > > > > > > > > > devise one?  It will be hard to make it
> > > > > > > > > > > deterministic,
> > > > > > > > > > > but
> > > > > > > > > > > at
> > > > > > > > > > > least we
> > > > > > > > > > > should see a non-zero cleanup count.
> > > > > > > > > > > 
> > > > > > > > > > > Antony Hosking | Associate Professor | Computer
> > > > > > > > > > > Science |
> > > > > > > > > > > Purdue
> > > > > > > > > > > University
> > > > > > > > > > > 305 N. University Street | West Lafayette | IN
> > > > > > > > > > > 47907 |
> > > > > > > > > > > USA
> > > > > > > > > > > Office +1 765 494 6001 | Mobile +1 765 427 5484
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > On Mar 11, 2008, at 4:03 AM, Dragiša Durić wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > I have single linked list that I use to send
> > > > > > > > > > > > messages to
> > > > > > > > > > > > many
> > > > > > > > > > > > threads.
> > > > > > > > > > > > It's end is locked for addition, and current end
> > > > > > > > > > > > is
> > > > > > > > > > > > given to
> > > > > > > > > > > > each
> > > > > > > > > > > > new
> > > > > > > > > > > > client connecting. This way, all client are
> > > > > > > > > > > > going
> > > > > > > > > > > > towards
> > > > > > > > > > > > end of
> > > > > > > > > > > > the
> > > > > > > > > > > > list so it's head becomes unreferenced and goes
> > > > > > > > > > > > away
> > > > > > > > > > > > by
> > > > > > > > > > > > GC.
> > > > > > > > > > > > 
> > > > > > > > > > > > Except it does not.
> > > > > > > > > > > > 
> > > > > > > > > > > > I've added WeakRef cleanup and sequential id's
> > > > > > > > > > > > so I
> > > > > > > > > > > > can
> > > > > > > > > > > > record
> > > > > > > > > > > > maximum
> > > > > > > > > > > > freed id for checking. No single cleanup
> > > > > > > > > > > > happens.
> > > > > > > > > > > > 
> > > > > > > > > > > > I'll cross check my code and count my references
> > > > > > > > > > > > from
> > > > > > > > > > > > application
> > > > > > > > > > > > side,
> > > > > > > > > > > > but maybe someone else has similar problems?
> > > > > > > > > > > > 
> > > > > > > > > > > > dd
> > > > > > > > > > > > -- 
> > > > > > > > > > > > Dragiša Durić <dragisha at m3w.org>
> > > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > -- 
> > > > > > > > > > Dragiša Durić <dragisha at m3w.org>
> > > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > -- 
> > > > > > > > Dragiša Durić <dragisha at m3w.org>
> > > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > -- 
> > > > > > Dragiša Durić <dragisha at m3w.org>
> > > > > > 
> > > > > 
> > > > -- 
> > > > Dragiša Durić <dragisha at m3w.org>
> > > > 
> > > 
> > -- 
> > Dragiša Durić <dragisha at m3w.org>
> > 
> 
-- 
Dragiša Durić <dragisha at m3w.org>