[M3devel] cvsup

Sun Mar 21 10:33:11 CET 2010

Few versions ago (May or July, when I last tried) cvsup started to
behave ill - it just stops giving any output while memory usage is
skyrocketing.

With that version, I've synced "incrementally". When it locks, I kill
it, then start over.

With latest version, one from Feb 21, this does not help anymore. It
just stops working (I've not observed if it leaks memory as former
version).

Half an hour ago I've reinstalled rpm's for one of this former versions
I've had on my system. cvsup again sort-of-works. At least it finishes
syncing after few restarts. This is what I get with -v:

faramir:dragisha/pts/6: ~% cvsup -v
CVSup client, non-GUI version
Copyright 1996-2003 John D. Polstra
Software version: 2009-08-03 16:02:04
Protocol version: 17.0
Operating system: LINUXLIBC6

It looks, for me, like client is culprit. Someone maybe can follow this
even further back to find when and where. I don't think kernel threads
are so guilty of this behaviour.

I will build RC cvs head now.

On Sun, 2010-03-21 at 08:54 +0000, Jay K wrote:
> Dragisha, it's still not clear to me if client is still working for
> not or not,
> on which platform, and if it doesn't work, what does it do?
>  (Well, NPTL implies Linux.)
> Yes, mostly we are talking about server here.
>   The server would always hang with kernel threads.
> But I'm going to look at client more due to this "EOF expected"
> warning I'm getting.
> 
> 
> > Or is my situation typical?
> 
> 
> What is your situation?
> We have very few reports of any kind, positive or negative.
> 
> 
> "Dead" could mean you stopped using it.
> But if it means it stopped working, please, what does it to?
> If it hangs, put in PutText+Flush calls to find out where
> it gets to before it hangs?
> See if it works with user threads (edit m3core/src/thread.quake)?
> 
> 
>  - Jay
> 
> > Subject: RE: [M3devel] cvsup
> > From: dragisha at m3w.org
> > To: jay.krell at cornell.edu
> > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com
> > Date: Sun, 21 Mar 2010 08:19:08 +0100
> > 
> > cvsup client was working for me all the time until latest build -
> RC4
> > pulled from local CVS (mirrored by cvsup) on Feb 21 2010.
> > 
> > It also worked 5 yrs ago when I used it with my NPTL implementation.
> > 
> > What little I understood glimpsing over this discussion, it affects
> > cvsup server? Or is my situation typical?
> > 
> > I'll try m3gdb it.
> > 
> > Thanks
> > 
> > On Sat, 2010-03-20 at 03:17 +0000, Jay K wrote:
> > > Dead as in was working, but no longer?
> > > Please elaborate if not working. What does it do?
> > > What platform? 
> > > Can you please debug it?
> > > I been doing mostly RTIO.PutText/Flush anyway.
> > > I tried several "debuggers": gdb, m3gdb, dbx, couldn't get much
> out of
> > > any. 
> > > 
> > > - Jay
> > > 
> > > > Subject: RE: [M3devel] cvsup
> > > > From: dragisha at m3w.org
> > > > To: jay.krell at cornell.edu
> > > > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com
> > > > Date: Thu, 18 Mar 2010 13:56:11 +0100
> > > > 
> > > > Client, it's what I am using. And it's dead as we speak.
> > > > 
> > > > Last worked before I pulled/built RC4
> > > > 
> > > > 
> > > > On Thu, 2010-03-18 at 10:09 +0000, Jay K wrote:
> > > > > Dragisha, Really? The server? With the -C flag and/or
> > > -f?
> > > > > 
> > > > > Evidence is very very very very very good as to what is going
> on
> > > here.
> > > > > It is all related to "fork1" vs. "forkall".
> > > > > fork1 is the overwhelming usual behavior (Solaris has an
> option),
> > > > > and basically *any* library that uses pthreads is obligated to
> use
> > > > > pthread_atfork, be it a C library or the Modula-3 library,
> etc..
> > > > > 
> > > > > The application cannot do things, unless
> > > > > the library exposes its internal locks. The Posix spec for
> > > > > pthread_atfork explains the problem.
> > > > > 
> > > > > Consider C code that links in Modula-3 code.
> > > > > C code is fairly free to use the fork + do work model. It
> isn't
> > > very
> > > > > common,
> > > > > but it does exist, and people have gone to extra length to
> keep it
> > > > > viable.
> > > > > bash and ccache I believe both use this model.
> > > > > 
> > > > > 
> > > > > Granted, if you had your own kernel thread implementation, it
> > > might
> > > > > have addressed this.
> > > > > 
> > > > > 
> > > > > I have a version now that has served over 1000 requests,
> similar
> > > to
> > > > > what I sent, but a bit reduced. The main change was I just
> called
> > > > > pthread_mutex_lock/unlock over the locks in ThreadPThread.m3,
> > > instead
> > > > > of using SuspendOthers. Haven't tested on Solaris/Darwin yet,
> just
> > > > > Linux. This version also doesn't expose the
> > > ForkProcessAndAllThreads
> > > > > functions.
> > > > > The client has to restart its threads, or in the cvsupd case,
> > > don't
> > > > > depend on the dispatcher thread to survive fork.
> > > > > I want to still try out the "bigger" change, but with the
> simpler
> > > > > lock/unlock.
> > > > > And test on Solaris and Darwin.
> > > > > 
> > > > > 
> > > > > I think for SuspendOthers to not work is not surprising.
> > > > > The threads can still hold a few locks even if suspended, I'm
> > > pretty
> > > > > sure.
> > > > > The point is to fork in a "controlled" fashion -- all locks
> held,
> > > and
> > > > > in
> > > > > the right order, so that both parent and child can release
> them.
> > > > > 
> > > > > 
> > > > > Taking "all" the locks in Prepare is more reliable.
> > > > > 
> > > > > 
> > > > > I do wonder if really "all" locks need to be taken.
> > > > > I only take the static ones declared in PThreadThread.c.
> > > > > 
> > > > > - Jay
> > > > > 
> > > > > 
> > > > > > Subject: Re: [M3devel] cvsup
> > > > > > From: dragisha at m3w.org
> > > > > > To: jay.krell at cornell.edu
> > > > > > CC: hosking at cs.purdue.edu; m3devel at elegosoft.com
> > > > > > Date: Wed, 17 Mar 2010 13:29:43 +0100
> > > > > > 
> > > > > > Yes, I can. I am building it regularly, from version to
> version,
> > > at
> > > > > > least few times a year. And it also worked with my ancient
> > > kernel
> > > > > thread
> > > > > > implementation. Was one of stress tests.
> > > > > > 
> > > > > > On Tue, 2010-03-16 at 16:10 +0000, Jay K wrote:
> > > > > > > 
> > > > > > > (Can anyone claim to have seen cvsup server work with
> kernel
> > > > > threads?)
> > > > > > -- 
> > > > > > Dragiša Durić <dragisha at m3w.org>
> > > > > > 
> > > > -- 
> > > > Dragiša Durić <dragisha at m3w.org>
> > > > 
> > -- 
> > Dragiša Durić <dragisha at m3w.org>
> > 
-- 
Dragiša Durić <dragisha at m3w.org>