[M3devel] dynamic chose between user/kernel threads?

[Jay]

> [Jay] Some programs "better" with user threads
> [Tony] Is this still true for modern kernel thread systems?
 
I really don't know.
Olaf /kind of/ indicated so (just in saying that it'd be a nice option to have).
 
 > [Jay] small stacks for many threads vs. larger stacks for calls to "the system"
 > [Tony] In a systems programming language like Modula-3 shouldn't threads have a relationship to the system?
 
 
Some might just be doing computation?


 > This will need to be a link-time distinction. 
 > I would hate to have both kernel and user thread code live in the same compiled run-time system.
 
I understand the principle, and I don't have a working implementation to argue with, but I do find this surprising. In my mind, you take
MODULE ThreadPThread EXPORTS Scheduler, Thread, etc.
change it to
MODULE ThreadPThread Exports SchedulerPThread, ThreadPThread, etc.
 
and introduce
MODULE Thread EXPORTS Scheduler, Thread, etc.
 
PROCEDURE Fork()=
BEGIN  IF KernelThreads
     RETURN ThreadPThread.Fork();
  ELSE
     RETURN ThreadPosix.Fork();
          END;
        END Fork.
 
 
      And so on. The potential bloat would be, you get both copies of code, both copies of globals, possibly initialization (but this most likely be avoided) and if any of the types are transparent, such that caller can allocate them on the stack, you end up either with the larger of the two sizes, or using a heap allocation with a dynamically chosen size.
   It seems to be roughly one source file each of bloat.
 
 

>[Tony]I see no problem with factoring out the threads subsystem as a separate library and linking accordingly.
 
That would prevent the bloat, indeed.
However, if you mean, like m3coreuserthreads.dll, m3corekernelthreads.dll, well, the vast bulk of each of those two files would be identical. You could maybe factor out m3kernelthreads.dll m3userthreads.dll out of m3core.dll and have m3core.dll dynamically chose one of them. That might be good.
 
But personally..I'd like fewer .dlls, even if that means larger .dlls.


>I would argue for user vs. kernel.
 
I think you are right. Kind of like the NT386 platform argument. There are many real implementation choices, but they need not be all exposed to the user, esp. unless/until someone asks. If Cygwin pthreads made to work, and then if there are proponents of it over NT threads, and vice versa, add a switch, pick a default. In the meantime, I only made one of three options work.


>And managing the state-space explosion that multiple choices might entail.  I would like to avoid that.
 
What do you mean? Growing types to the union? The globals and code bloat? Globals could all be pushed into a heap allocated pointer anyway, paying a perf cost that would unlikely be noticed.
 
Again, I kind'a'thunk linking both in would be pretty ok, it's basically just one source file, but I haven't done it yet. Before I do this I need to be working on a platform with user threads, and then make them work, and then make them "switchable", and I want to bring up more platforms before I work on any user threads. AMD64_FREEBSD I think will be next.


 > I am not so sure, since it would entail some fairly low-level
 > and expensive decisions that would best instead be done at link-time.
 
Again this doesn't agree with my /suspicion/ but I didn't do enough research/work to be all that informed..


 > Actually, for non-standalone (dynamically-linked) code
 > we could make the choice at dynamic load time using the OS's support for dynamic library choice.
 
I'd rather not use dlopen et. al. at least not until I understand various details of dynamic loading across more platforms. I always though Win32 did it all just about right, and then I find a surprising array of other implementation choices and options on other systems..
  > I note that Solaris (but not Solaris Modula-3) currently lets
  > dynamically linked C programs choose among 3 different threads implementations dynamically.


Too much of a good thing imho...
At least in Windows, there are a lot of "in-process plugins", and often times, interoperating code has to agree on the same underlying pieces, so wherever there is a choice, is an opportunity to fail.
For example, if I call malloc and give you the result to free, or I call fopen and give it to you to fwrite,
we need to be using the same C runtime. Even Linux has a few to chose from -- glibc, dietlibc, uclibc, newlib, etc. If you push "plugins" out of process, things become usually easier. Then folks either interoperate through carefully formatted files / pipes, network protocols, or RPC, and not via function calls.
 
 
 I only knew of two threading libraries on Solaris, "pre-pthreads" and "pthreads".
Oh, maybe "lwp" and "thr"?
I was wondering -- these are all built on the same kernel threads, right?
So if you use a mix of them in a process, you still get reasonable global scheduling, right?
ie: There's no point in writing ThreadSolarisThis.m3 or ThreadSolarisThat.m3, ThreadPThread.m3 suffices plenty?
 
 
Coming back to the bloat issue.
If you make the transform I show, you could also have a variant of Thread.m3 that just calls ThreadPThread.m3 directly. That is, it should be easy (still not having done it) to alter the code to build both of m3coreboth.dll and m3corejustkernel.dll. They can both have the interface that accepts the request or mandate for user threads, and m3corejustkernel.dll could either ignore or error on it.
Users could install one or the other .dll as m3core.dll.
Of course you could alos build m3corejustuser.dll.
(These are stupid names, I know.)
 
I really need to try it and see, it's fairly hypothetical right now.
It sounds like I might be missing something.
But first some other things..
 - Jay



CC: wagner at elegosoft.com; m3devel at elegosoft.comFrom: hosking at cs.purdue.eduTo: jay.krell at cornell.eduSubject: Re: [M3devel] dynamic chose between user/kernel threads?Date: Thu, 8 Jan 2009 13:51:15 +1100
Jay, perhaps you can provide some context and motivation for your proposals.  I am unable to evaluate your proposals in the form of a simple laundry-list.


On 8 Jan 2009, at 01:34, Jay wrote:

I looked into this a bit..and I think it's actually pretty easy.It hardly churns the code even. I have other things I want to do first. In the meantime, specify the directive?There are actually a large number of subtle options.  Can I ask for any of n libraries -- NT, pthreads, user threads? setjmp vs. context? Or is it just a boolean, user vs. kernel?  Is the option to "favor" or "require" a certain library?  Can I set it on libraries or only programs? What if the requests clash?  Is it "Favor" or "Require"?Is it a function call or setting a global variable?I favor function call, but TARGET, WORD_SIZE, BUILD_DIR establish the other precedent,and either can work. Is it flags in the moduleinfo like incgc, gengc, or separate data thatthere is just one of in RTLinker? Probably separate data, unless it is allowed in libraries.  What are the names of the libraries? "posix" and "pthreads"  ?  "user" and "kernel" ?  "true" and "false" (or vice versa) ?  Should NT implement user threads, using fibers?  Presumably it works with both standalone and shared/dynamic programs.Presumably it is ok to always bloat up m3core.dll with both libraries.Presumably it is ok to have everyone pay a teeny tiny perf.  That is, there is a microscopic dispatching layer, that everyone  ends up going through, not chosing to link in one library or the other.  And as part of this, whenever it happens, can we bump up the minimumbootstrap to a version that includes SchedulerPosix.DoesWaitPidYield()?Or does it become VAR Scheduler.UsingKerneThreads?(no, it should be a function; but naming matter remains).  (ie. as I said, the decision currently is baked into m3core.dll, butnow it is also baked into sysutils.dll, but these should change together;m3core.dll should manage the knowledge).  - Jay> Date: Sat, 3 Jan 2009 01:12:27 +0100> From: wagner at elegosoft.com> To: m3devel at elegosoft.com> Subject: Re: [M3devel] dynamic chose between user/kernel threads?> > An option to cm3 or a quake directive to use in the m3makefiles> would suffice in my opinion (and be a great improvement).> > Olaf> > Quoting Jay <jay.krell at cornell.edu>:> > >> > Should the user/kernel thread chose be available:> >> >> > On the command line to a Modula-3 executable (or even an executable > > where main is in another language, but which static or dynamic > > Modula-3 libs are used)?> >> >> > Via a quake directive when building programs?> >> >> > You know, imagine you have a bunch of Modula-3 programs and some but > > not all use a very large number of threads and benefit from > > userthreads.> >> >> > Currently the chose is locked into m3core when it is built.> >> >> > - Jay> > > -- > Olaf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20090108/570b4780/attachment-0002.html>