[M3devel] trampolines and uplevel locals declared after function starts..

[Jay K]

Are you going to use trampolines? Runtime generated code? On the stack? I kind of hope not.There are advantages and disadvantages either way.Actually..hm..if LLVM makes it easy, well, then, that is a big advantage..given how much I have wrestled with this stuff...  For example, try writing a nested function using gcc on MacOSX -- you get a warning/error and you have to throw an extra flag to enable nested functions.  Another concern here is object code compatibility between the backends? Maybe?What I'm not generating isn't compatible, I'm afraid, and making it compatible might be difficult.It might work to generate C++ and use "this" for the static link. That might be compatible -- if static link and "this" coincide. There is evidence they do (NT/x86) but I haven't researched the various architectures.  You know, a few times already I haven't been careful about mixing object code across backends, and I got crashes.. considering how static links work, not surprising..    - Jay From: hosking at cs.purdue.edu
Date: Fri, 21 Sep 2012 09:50:21 -0400
To: jay.krell at cornell.edu
CC: m3devel at elegosoft.com; antony.hosking at gmail.com
Subject: Re: [M3devel] trampolines and uplevel locals declared after	function starts..

I think we should leave the CG interfaces as they are already (for static links).  I can see how to generate LLVM IR easily with things the way they are.  And you are already working to it.  I wasn’t suggesting that you use trampolines, though I think it would be easy enough to do.  But I am going to use the LLVM nest approach to pass frame pointers.  That way I’ll get the efficient native static link passing code.
On Sep 21, 2012, at 1:03 AM, Jay K <jay.krell at cornell.edu> wrote: >  Did you take a look at the LLVM IR I sent? T

Sorry, I couldn't quickly make sense of it. :(
I have to read up on LLVM or study your code more.
(This is why I favor a C backend -- I'm already an expert and there is far more C expertise in the world than LLVM expertise..I think.. and other reasons like portability..)
 
 
Mention of "trampolines" was worrisome though.
That usually means "runtime generated code, often on the stack". True here?
 
 
If not, nevermind
If so:
 
 
 
There is no good solution here, but I think runtime generated code is mostly a losing battle..at least on the stack. If you dedicate a little bit of special purpose platform-specific code to it, and you don't mind it being slow, and you deal with the lifetime management, then it becomes viable.
e.g. on Windows, you use VirtualAlloc(MEM_EXECUTE).
On other platforms you use mprotect I think. And/or mmap? mmap(PROT_EXEC)?
But you end up allocating memory in large chunks, like at least 4K or 64K.
(You can also write out little temporary .so/.dll files, but that is unnecessarily inefficient.)
 
 
I've seen code that calls mprotect against the stack. Yuck.
The stack is nonexecutable on purpose -- for security (assuming there is any C or C++ code in the process).
 
 
If you read the libffi mailing list, you see they are constantly in this arms race.
There is a slow steady trickle of problems making the libffi code executable.
For Darwin/ARM they came up with a trick to avoid generating code, since the platform doesn't allow unsigned code to run.
 
 
 
Pluses/minuses:
 
 
runtime generated code:
Producing the code is fast, on historical systems that don't make it difficult to execute memory, e.g. on the stack.
Running the code is fast.
Pointers to closures become compatible with C -- you can pass a pointer to a closure as a callback to or C++ and it works.
Highly target-dependent.
 
 
The unusual Modula-3 method:
Also fast.
Calling function pointers is different and larger and slower.
C code cannot call function pointers to nested functions. That is the main downside.
Only slightly target-dependent -- in its choice of closure marker.
I think we should change the closure marker slightly -- its size should be target-dependent, it should only be 4 bytes on SPARC64, MIPS64, PPC64, Alpha, AMD64 -- so the alignment check can be skipped. And it should possibly be larger than 4 or 8 bytes on IA64.
But someone needs to read up on all the various instruction encodings and verify that 4 or 8 0xFF bytes are really invalid and won't occur.
But most likely if 4 bytes of 0xFF are a good marker on MIPS32, PPC32, SPARC32, I386, they are also probably a good marker on MIPS64, PPC64, SPARC64, AMD64. And Alpha.
Another option is to use something like "jmp marker". i.e. a jump to an address that is used just be the marker. It might not be a constant though. Maybe it is relocated. But you want it to be the same across all dynamic linked code, actually.
 
 
We might want to have calls through a function pointer go through a helper function, to keep it small.
 
 
<tangent>
Hm.. I wonder..maybe a closure can be generated fairly portably actually.
Maybe you just need one little wrapper per function signature. And most would really be the same.
Let me think on this..hm..well..you could do it if you have a special static link register probably. I need to think about this more. Do "all" the calling conventions provide a special extra register for "static link"? And is that often used for C++ "this"? In that case, there is a way..
 
 
<tangent>
Ultimately, I should point out, the idea of passing a closure to C is actually very useful.
One should be able to pass an "object method pointer".
So some language construct or pragma might be nice there -- to force down the runtime generated code path.
But usually the C function that wants a function pointer also takes a "void* context", so maybe not so needed.
 
 
Or maybe folks would just use "libffi" for that.
 
 
 
 - Jay
 
CC: m3devel at elegosoft.com
From: antony.hosking at gmail.com
Subject: Re: [M3devel] uplevel locals declared after function starts..
Date: Thu, 20 Sep 2012 19:38:27 -0400
To: jay.krell at cornell.edu

Agreed. You need to backpatch the uplevel locals. I would accumulate all of them. Did you take a look at the LLVM IR I sent?  That's how I plan to so it. 

Sent from my iPhone
On Sep 20, 2012, at 19:18, Jay K <jay.krell at cornell.edu> wrote:

We get uplevel locals declared while within a function.So I think I'll bite the bullet and make multiple passes now.Otherwise I build up the frame struct "too early".I need to see all the locals -- all the uplevels, early in a function.

There is also a chance they don't all have unique names.

I do NOT expect I should have separate frames/static_links per-scope,just per-function.

I could put structs inside the frame, named after the line they starton or such.. that might be prettier and more readable, or not.

The structs could then be unioned at some point -- parallel scopescan/should share storage.

M3C.m3:PROCEDURE declare_local(...)BEGIN...IF u.in_proc THEN        ExtraScope_Open(u);        print(u, var.Declare() & " M3_INIT;");        <* ASSERT up_level = FALSE *>  this fails    ELSE        IF up_level THEN            u.current_proc.uplevels := TRUE;        END;        u.current_proc.locals.addhi(var);    END;    RETURN var;END declare_local;

== package /Users/jay/dev2/cm3/m3-libs/m3tk-misc ==
...new source -> compiling Args.m3

****** runtime error:***    <*ASSERT*> failed.***    file "../src/M3C.m3", line 1778***
  m3_backend => 1536m3cc (aka cm3cg) failed compiling: Args.mc

I'm guessing the code is here:

PROCEDURE Errors(h: Handle; indent: CARDINAL := 0): Text.T RAISES {}=  BEGIN    IF h.errors = 0 THEN      <*FATAL Fatal*> BEGIN RAISE Fatal; END;    ELSE
      VAR        texts := NEW(REF ARRAY OF TEXT, h.errors * 2);            (* allocates space for all the error messages + padding *)        pos: CARDINAL := 0;        padding := Fmt.Pad("", indent);        fmt: Text.T;
      <*INLINE*> PROCEDURE Add(t: Text.T) RAISES {}=          BEGIN texts[pos] := t; INC(pos) END Add;      <*INLINE*> PROCEDURE PaddedAdd(t: Text.T) RAISES {}=          BEGIN Add(padding); Add(t) END PaddedAdd;


see how "texts" is both uplevel and "not at the start" of the function.

This is going to take a few days to deal with.

Compiler can compile itself, and a large swath of the system now.I didn't use the self-built compiler, but I will do that next. It probably works.Very good.

 - Jay
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20120921/76b04f27/attachment-0002.html>