[M3devel] pthread in cm3...

[Tony Hosking]

On Apr 22, 2007, at 10:47 AM, Dragiša Durić wrote:

> Hello,
>
> Have been skimming through source of PTHREAD code... And I think  
> job can
> be done without so much relying on how-they-did-it-before, esp with
> regard to list of waiters and similar internal and global structures.
> Also, I see number of global locks and I am sure they are congestion
> generators every now and while, esp in heavy threading situations.
>
> Of course, there is number of approaches to this multi-thread
> situations. Mine being one of very nonconservative use of threads, I
> think it is important to remain open to possibly very big number of
> threads running in single process - meaning scalability is one of
> primary objections... As global locks don't do well with  
> scalability, I
> don't like "cm" and similar global congestion points.

Yes, there are tensions between a thin/absent veneer between language  
threads and system threads.  Most important are issues of preserving  
a reasonable memory model for programmers (see Hans Boehm's paper  
http://portal.acm.org/citation.cfm?doid=1065010.1065042).  There are  
also questions of portability and debuggability.  I agree that global  
locks are to be avoided where they cause known contention, but there  
are tradeoffs there too.  For large numbers of threads (as you appear  
to need) I think we would need to adopt some other implementation  
approach, possibly by multiplexing multiple lightweight user-level  
threads on some smaller number of heavyweight system-level threads,  
but then you run into scheduling and load-balancing problems.

> We talked about this at least once before, and I think I remember you
> insisted on more compatibility than can be read from SPwM3. Do you  
> think
> best idea would be to integrate mine NPTL code into CM3 for people to
> trash and test, and let everyone select what is best for their
> situation?

What I wanted was a situation where programs would be able to run  
with the same tools (e.g., showheap, showthread) under both user- 
level and system threading.  This goal has been achieved with the  
current pthread-based implementation.  Moreover, I wanted something  
where variations in thread support from one system to another could  
be exploited most easily (such as for systems where thread suspend/ 
resume is provided as a primitive).  Again, the current  
implementation achieves this, and runs with minimal target-specific  
code on Darwin, Solaris, and Linux.  Ports to other targets should be  
relatively straightforward.

> Problems I had with my pthread implementation were all related to VM
> hell of earlier GC implementation... After you did that piece of art
> with new approach to GC, I expect infinite uptimes from my servers and
> bots :). Big thanks for that!

Any native threading implementation is going to have problems with VM- 
based memory management.  I'm surprised to were able to get anything  
going with the VM-based GC.

>
> dd
> -- 
> Dragiša Durić <dragisha at m3w.org>