[M3devel] userthreads vs. pthreads performance?

Jay K jay.krell at cornell.edu
Mon Mar 29 10:24:47 CEST 2010


 > We can optimize this somewhat on most systems with __thread. I had that in briefly

 

 

I did a little bit of testing here.

 

 

__thread takes a few forms, depending on which of -fPIC and -shared you use.

 

Once you do throw in -fPIC and -shared, I have found __thread to be significantly slower on Solaris/sparc and Linux/powerpc32, slower or a wash on Linux/amd64, and twice as fast as pthread_getspecific on Linux/x86.

I doesn't appear supported at all on Darwin, though pthread_getspecific are very fast there (albeit not inlined).

I didn't test *BSD.

My testing was not very scientific.

However -fPIC and/or -shared imply a function call to access __thread variables.

 That's probably the big factor. Without -fPIC/-shared, there is no function call.

 

 

If you are going to access them multiple times in a function then there is a probably an optimization to be had -- caching their address. If the variables are larger than a pointer, probably then also an optimization to be had.

 

 

We could do that first thing for certain -- PushFrame could return the address and PopFrame would be much faster.

However another angle here is to eliminate PushFrame/PopFrame, by using libunwind. I think we should look into libunwind for the next release. The thread locals will (mostly) remain but accesses to them greatly decline.

 

 

We compile everything -fPIC and -shared.

Some systems (libtool) compile things once that way and once not, providing pairs of libraries, depending on intended use.

 

 

I should point out also that userthreads have been greatly deoptimized in the current tree (by me, Tony approved), because they used to inline PushFrame/PopFrame, but they don't any longer.

 

 

(Historically on NT, __declspec(thread) only worked in code in the .exe or statically loaded by the .exe -- that is, not in .dlls loaded with LoadLibrary. However that limitation was removed in Vista. I expect __declspec(thread) is much faster than TlsGetValue, but I assume we'll support pre-Vista for a while longer so not interesting..)

 

 

 - Jay

 


From: jay.krell at cornell.edu
To: dragisha at m3w.org; mika at async.async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] userthreads vs. pthreads performance?
Date: Mon, 29 Mar 2010 03:15:41 +0000



 > Getting thread locals should not require a kernel call
 
Indeed, on Linux/x86 it does not, looks pretty ok:
 
00000380 <__pthread_getspecific>:
 380:   55                      push   %ebp
 381:   89 e5                   mov    %esp,%ebp
 383:   8b 55 08                mov    0x8(%ebp),%edx
 386:   81 fa ff 03 00 00       cmp    $0x3ff,%edx
 38c:   76 04                   jbe    392 <__pthread_getspecific+0x12>
 38e:   5d                      pop    %ebp
 38f:   31 c0                   xor    %eax,%eax
 391:   c3                      ret

 392:   89 d0                   mov    %edx,%eax
 394:   c1 e8 05                shr    $0x5,%eax
 397:   8d 0c 85 1c 01 00 00    lea    0x11c(,%eax,4),%ecx
 39e:   65 8b 01                mov    %gs:(%ecx),%eax
 3a1:   85 c0                   test   %eax,%eax
 3a3:   74 e9                   je     38e <__pthread_getspecific+0xe>
 3a5:   8b 04 d5 00 00 00 00    mov    0x0(,%edx,8),%eax
 3ac:   85 c0                   test   %eax,%eax
 3ae:   74 de                   je     38e <__pthread_getspecific+0xe>
 3b0:   65 8b 01                mov    %gs:(%ecx),%eax
 3b3:   83 e2 1f                and    $0x1f,%edx
 3b6:   8b 04 90                mov    (%eax,%edx,4),%eax
 3b9:   5d                      pop    %ebp
 3ba:   c3                      ret


> Entering an uncontended pthread mutex should not be expensive

Linux/x86:
 
00001020 <__pthread_self>:
    1020:       55                      push   %ebp
    1021:       89 e5                   mov    %esp,%ebp
    1023:       65 a1 50 00 00 00       mov    %gs:0x50,%eax
    1029:       5d                      pop    %ebp
    102a:       c3                      ret
    102b:       90                      nop
    102c:       8d 74 26 00             lea    0x0(%esi),%esi
 
 
pretty lame, five instructions were only two are needed.
 

000004f0 <__pthread_mutex_lock>:

.. too much to read through..but I think no kernel call..
 
 - Jay
 


From: jay.krell at cornell.edu
To: dragisha at m3w.org; mika at async.async.caltech.edu
CC: m3devel at elegosoft.com
Subject: RE: [M3devel] userthreads vs. pthreads performance?
Date: Sun, 28 Mar 2010 20:46:01 +0000



O(1) scheduling is not a new idea. Just look at NT and probably Solaris and probably all the other non-free systems (AIX, Irix, HP-UX, Tru64, VMS, etc.)

Getting thread locals should not require a kernel call. It doesn't on NT. We can optimize this somewhat on most systems with __thread. I had that in briefly.

Entering an uncontended pthread mutex should not be expensive -- at least no kernel call, but granted a call and atomic op. Two calls because of the C layer.
But user threads pay for a call too of course.

Maybe I should profile some of this..

- Jay

> From: dragisha at m3w.org
> To: mika at async.async.caltech.edu
> Date: Sun, 28 Mar 2010 21:14:57 +0200
> CC: m3devel at elegosoft.com
> Subject: Re: [M3devel] userthreads vs. pthreads performance?
> 
> I remember reading (long time ago) about how these (FUTEXes) are
> efficient in LINUX... Can I have your test code to try?
> 
> On Sun, 2010-03-28 at 12:11 -0700, Mika Nystrom wrote:
> > Well I have run programs on PPC_DARWIN and FreeBSD<X> and seen these sorts of things...
> > 
> > =?UTF-8?Q?Dragi=C5=A1a_Duri=C4=87?= writes:
> > >Which platform?
> > >
> > >On Sun, 2010-03-28 at 11:57 -0700, Mika Nystrom wrote:
> > >> Yep, sounds right. 
> > >> 
> > >> I was profiling some other thread-using code that slowed down
> > >> enormously
> > >> because of pthreads and it turned out the program was spending ~95%
> > >> of its time in accessing the thread locals via one of the pthread_
> > >> functions.
> > >> (The overhead of entering the kernel.)
> > >-- 
> > >Dragiša Durić <dragisha at m3w.org>
> -- 
> Dragiša Durić <dragisha at m3w.org>
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20100329/16111798/attachment-0002.html>


More information about the M3devel mailing list