[M3devel] threading on Windows?

Fri Feb 11 22:26:52 CET 2011

Tony:

Thank you very much for this explanation.

So, it appears that the changes introduced to support the new "heap-based jmpbuf allocation" have introduced at least 2 problems that must be solved if we are to move forward on the Windows platform, namely:

1.       Space leak problem.

2.       Binding to appropriate "alloca" routine on Windows platform.

Assuming we want to retain this new heap-based jmpbuf allocation capability, someone needs to solve these 2 problems for the Windows platform.

I will begin looking at the code and see if I can make some progress on these 2 problems.  Before I "do" anything, I'll run my proposed solution thru the m3devel group first.  Any further insights you or Jay can give in helping me work on the solution are welcome and appreciated.

Finally, let me also suggest that we all may need to revisit the conversation we had on this forum many months back about contributors making sure their changes are tested not to cause major breaks before committing them to HEAD branch.

Seems to me that if the "heap based jumpbuf allocation scheme" was known not to be compatible with Windows, that it should have been checked in on a different experimental branch until all the kinks could be worked out for all platforms.

Otherwise, without some mechanism to warn everyone of known problems on the HEAD branch, folks can easily download stuff that renders their implementation broken.

If my understanding of HEAD is not the prevailing one, I for one would like clarification as to what is STABLE for checkout and use and what is UNSTABLE.

Regards,
Randy Coleburn

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Thursday, February 10, 2011 4:26 PM
To: Coleburn, Randy
Cc: m3devel
Subject: Re: [M3devel] threading on Windows?

Hi Randy,

A far as I know, the changes are as follows.

TRY blocks using setjmp/longjmp to implement the exception handling each require statically one instance of a jmpbuf.  Historically, the storage for this was reserved as stack local storage (using CG.Declare_param) of the target-dependent jmpbuf size.  This meant having to declare the jmp_buf accurately for the target.  What Jay did was to replace the stack local with a scalar pointer local variable (also using Declare_param), and to call alloca to obtain a pointer to stack allocated jmpbuf storage.  The idea was to add a level of indirection, and have the size of the jmpbuf obtained from a C variable initialized appropriately,  so that targets need no longer declare their jmpbuf_size.   This was intended to make porting easier.  There are several problems here.  First, the local pointer variable needs to be initialized *once* on entry to the procedure to the value NIL, so that we check for NIL on the first invocation of the TRY and alloca only in that case.  The problem is that currently that initialization is not being performed *once* but instead on each invocation of the TRY so loops with TRY blocks in them currently have a space leak.

The other problem was how to bind to the appropriate alloca routine.  Ideally it should be inlined, and on Windows this is achieved by calling a *different* function, not called alloca.  My suggestion is that making the alloca call a runtime hook will allow targets to easily redirect to the correct function.  On non-Windows targets using the gcc-based backend all works out because alloca is a gcc intrinsic function that gets inlined.

To fix the space leak problem I proposed that Marker in the M3 compiler front end m3front should track, per-procedure, the jmpbuf pointer variables as it parses or checks the program, and have a callback to initialize them at the top of the procedure during code generation.  I don't think this will involve much work, but I don't currently have any spare cycles to spend on it.

-- Tony
Antony Hosking | Associate Professor | Computer Science | Purdue University
305 N. University Street | West Lafayette | IN 47907 | USA
Office +1 765 494 6001 | Mobile +1 765 427 5484

On Feb 10, 2011, at 3:59 PM, Coleburn, Randy wrote:

Jay / Tony:

I am seeing in this thread that the approach to fixing the problem is not yet agreed upon.

I will be glad to help in any way I can, but right now, I am "in the dark", both as to what change was introduced that caused the problem, and the various potential solution paths.  All I know is that it has been broken for a number of weeks now.

I know Jay has asked for help, and I am willing to help, but it may cost him or you some time/effort in getting me up to speed on what went wrong and the pros/cons of potential solutions.  If you choose to help enlighten me, I pledge to try and put that knowledge to good use both now and in the future.  Otherwise, I will have to wait for one of you to solve the problem.

BTW, when I look at the linker error reports for the various packages, I'm seeing several different symbols that don't resolve, not just the one dealing with memory allocation.

Regards,
Randy

From: Tony Hosking [mailto:hosking at cs.purdue.edu]
Sent: Thursday, February 10, 2011 3:33 PM
To: jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>
Cc: Coleburn, Randy; m3devel
Subject: Re: [M3devel] threading on Windows?

Why is it better?  Because rebinding to a different function is just a matter of changing an INTERFACE in the libraries, rather than hacking the compiler.
On Feb 10, 2011, at 6:01 AM, jay.krell at cornell.edu<mailto:jay.krell at cornell.edu> wrote:

1. I never understood why this any better.
2. It how things are currently.
3. Why not m3cg.alloca? (but see #2)

 - Jay/iPad

On Feb 9, 2011, at 9:48 PM, Tony Hosking <hosking at cs.purdue.edu<mailto:hosking at cs.purdue.edu>> wrote:
What happened to fixing this with runtime hooks?

On Feb 9, 2011, at 11:40 PM, Jay K wrote:

Of course not.
It is a made-up function that the frontend generates a call to.
That it never previously did.
Using the general function call interface.
You need to add a special case, in the general function call code, to do something different and specific
for this specific function.

 - Jay

________________________________
From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
To: jay.krell at cornell.edu<mailto:jay.krell at cornell.edu>; m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
Date: Wed, 9 Feb 2011 22:28:45 -050
Subject: Re: [M3devel] threading on Windows?
Jay:

I looked thru the 4,618 lines of M3x86.m3, but I don't see any reference to "m3_alloca" or even "alloc" in this file.

Regards,
Randy

From: jayk123 at hotmail.com<mailto:jayk123 at hotmail.com> [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Wednesday, February 09, 2011 9:01 PM
To: Coleburn, Randy; m3devel
Subject: RE: [M3devel] threading on Windows?

In m3back/src/M3x86.m3.

 - Jay

________________________________

From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
To: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
Date: Wed, 9 Feb 2011 18:22:56 -0500
Subject: Re: [M3devel] threading on Windows?
I am certainly willing to work on the problem, but need more context info about what caused it in order to know how to resolve.
All I know is that everything was working fine until I checked out the HEAD repository.
When you say "special case" calls "m3_alloca", where do I go about finding this "special case"?
Are we talking Modula-3 code, C code, Assembler, what?  What package/module?
Regards,
Randy

From: jayk123 at hotmail.com<mailto:jayk123 at hotmail.com> [mailto:jayk123 at hotmail.com] On Behalf Of Jay K
Sent: Wednesday, February 09, 2011 5:59 PM
To: Coleburn, Randy; Mika Nystrom; m3devel
Subject: RE: [M3devel] threading on Windows?

Maybe someone else can do it?
The fix is: special case calls to the function "m3_alloca".
Change it to call "_chkstk" (or maybe "chkstk", whatever works).
The one parameter is an unsigned 32bit quantity, passed in eax, and the return value is a pointer, returned in esp.
To some extent I drag my feet hoping anyone else might become motivated enough to do it and start learning how.

 - Jay

> From: rcolebur at SCIRES.COM<mailto:rcolebur at SCIRES.COM>
> To: mika at async.caltech.edu<mailto:mika at async.caltech.edu>; m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
> Date: Wed, 9 Feb 2011 17:25:20 -0500
> Subject: Re: [M3devel] threading on Windows?
>
> Mika:
>
> Sorry, but my Windows build is broken. Jay seems to indicate he is working on a "fix" and that this fix is relatively simple, but so far no solution has been checked in.
>
> I've been wanting to run your program ever since you first checked it in, but that was about same time the HEAD branch update introduced a build problem. Problem is an unresolved symbol during link.
>
> As soon as I can get the build problem resolved, I'll try out your test program.
>
> Regards,
> Randy Coleburn
>
> -----Original Message-----
> From: Mika Nystrom [mailto:mika at async.caltech.edu]
> Sent: Wednesday, February 09, 2011 5:01 PM
> To: m3devel at elegosoft.com<mailto:m3devel at elegosoft.com>
> Subject: [M3devel] threading on Windows?
>
> Hi m3devel,
>
> I'm just curious if anyone out there who's running CM3 on Windows has had
> a chance to try my thread testing program yet. Current status on Unix
> (Linux, FreeBSD) appears to be that there are problems in pthreads (but
> I think only under heavy load) and that user threading works perfectly.
> So I now wonder how things are on the third threading platform (Windows).
> The thread testing program is at m3-libs/m3core/tests/thread .
>
> (Note however that there is a memory leak in TRY-EXCEPT in the current
> head so don't try to update everything to the CVS head. The previous
> release should be OK.)
>
> Also has anyone else noticed that debugging information has recently
> broken? m3gdb is very unhappy on all platforms for me...
>
> Mika

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20110211/9f86ae19/attachment-0002.html>