[M3devel] trampolines and uplevel locals declared after function starts..

Fri Sep 21 16:40:44 CEST 2012

Backend compatibility is not something we need to aim for.  Different backends will generate different object code.
I do plan to use the LLVM mechanisms to pass static links as they are the only way to do so.  They use a trampoline to grab the static link for function pointers, but I won’t do that.  I’ll compute the static link explicitly so that closures will be constructed in the usual way (triple comprising −1, function pointer, static link).

On Sep 21, 2012, at 10:27 AM, Jay K <jay.krell at cornell.edu> wrote:

> Are you going to use trampolines? Runtime generated code? On the stack? I kind of hope not.
> There are advantages and disadvantages either way.
> Actually..hm..if LLVM makes it easy, well, then, that is a big advantage..given how much I have wrestled with this stuff...
>  
>  
> For example, try writing a nested function using gcc on MacOSX -- you get a warning/error and you have to throw an extra flag to enable nested functions.
>  
>  
> Another concern here is object code compatibility between the backends? Maybe?
> What I'm not generating isn't compatible, I'm afraid, and making it compatible might be difficult.
> It might work to generate C++ and use "this" for the static link. That might be compatible -- if static link and "this" coincide. There is evidence they do (NT/x86) but I haven't researched the various architectures.
>  
>  
> You know, a few times already I haven't been careful about mixing object code across backends, and I got crashes.. considering how static links work, not surprising..
>  
>  
>  
>  - Jay 
> From: hosking at cs.purdue.edu
> Date: Fri, 21 Sep 2012 09:50:21 -0400
> To: jay.krell at cornell.edu
> CC: m3devel at elegosoft.com; antony.hosking at gmail.com
> Subject: Re: [M3devel] trampolines and uplevel locals declared after	function starts..
> 
> I think we should leave the CG interfaces as they are already (for static links).  I can see how to generate LLVM IR easily with things the way they are.  And you are already working to it.  I wasn’t suggesting that you use trampolines, though I think it would be easy enough to do.  But I am going to use the LLVM nest approach to pass frame pointers.  That way I’ll get the efficient native static link passing code.
> 
> On Sep 21, 2012, at 1:03 AM, Jay K <jay.krell at cornell.edu> wrote:
> 
>  >  Did you take a look at the LLVM IR I sent? T
> 
> Sorry, I couldn't quickly make sense of it. :(
> I have to read up on LLVM or study your code more.
> (This is why I favor a C backend -- I'm already an expert and there is far more C expertise in the world than LLVM expertise..I think.. and other reasons like portability..)
>  
>  
> Mention of "trampolines" was worrisome though.
> That usually means "runtime generated code, often on the stack". True here?
>  
>  
> If not, nevermind
> If so:
>  
>  
>  
> There is no good solution here, but I think runtime generated code is mostly a losing battle..at least on the stack. If you dedicate a little bit of special purpose platform-specific code to it, and you don't mind it being slow, and you deal with the lifetime management, then it becomes viable.
> e.g. on Windows, you use VirtualAlloc(MEM_EXECUTE).
> On other platforms you use mprotect I think. And/or mmap? mmap(PROT_EXEC)?
> But you end up allocating memory in large chunks, like at least 4K or 64K.
> (You can also write out little temporary .so/.dll files, but that is unnecessarily inefficient.)
>  
>  
> I've seen code that calls mprotect against the stack. Yuck.
> The stack is nonexecutable on purpose -- for security (assuming there is any C or C++ code in the process).
>  
>  
> If you read the libffi mailing list, you see they are constantly in this arms race.
> There is a slow steady trickle of problems making the libffi code executable.
> For Darwin/ARM they came up with a trick to avoid generating code, since the platform doesn't allow unsigned code to run.
>  
>  
>  
> Pluses/minuses:
>  
>  
> runtime generated code:
> Producing the code is fast, on historical systems that don't make it difficult to execute memory, e.g. on the stack.
> Running the code is fast.
> Pointers to closures become compatible with C -- you can pass a pointer to a closure as a callback to or C++ and it works.
> Highly target-dependent.
>  
>  
> The unusual Modula-3 method:
> Also fast.
> Calling function pointers is different and larger and slower.
> C code cannot call function pointers to nested functions. That is the main downside.
> Only slightly target-dependent -- in its choice of closure marker.
> I think we should change the closure marker slightly -- its size should be target-dependent, it should only be 4 bytes on SPARC64, MIPS64, PPC64, Alpha, AMD64 -- so the alignment check can be skipped. And it should possibly be larger than 4 or 8 bytes on IA64.
> But someone needs to read up on all the various instruction encodings and verify that 4 or 8 0xFF bytes are really invalid and won't occur.
> But most likely if 4 bytes of 0xFF are a good marker on MIPS32, PPC32, SPARC32, I386, they are also probably a good marker on MIPS64, PPC64, SPARC64, AMD64. And Alpha.
> Another option is to use something like "jmp marker". i.e. a jump to an address that is used just be the marker. It might not be a constant though. Maybe it is relocated. But you want it to be the same across all dynamic linked code, actually.
>  
>  
> We might want to have calls through a function pointer go through a helper function, to keep it small.
>  
>  
> <tangent>
> Hm.. I wonder..maybe a closure can be generated fairly portably actually.
> Maybe you just need one little wrapper per function signature. And most would really be the same.
> Let me think on this..hm..well..you could do it if you have a special static link register probably. I need to think about this more. Do "all" the calling conventions provide a special extra register for "static link"? And is that often used for C++ "this"? In that case, there is a way..
>  
>  
> <tangent>
> Ultimately, I should point out, the idea of passing a closure to C is actually very useful.
> One should be able to pass an "object method pointer".
> So some language construct or pragma might be nice there -- to force down the runtime generated code path.
> But usually the C function that wants a function pointer also takes a "void* context", so maybe not so needed.
>  
>  
> Or maybe folks would just use "libffi" for that.
>  
>  
>  
>  - Jay
>  
> CC: m3devel at elegosoft.com
> From: antony.hosking at gmail.com
> Subject: Re: [M3devel] uplevel locals declared after function starts..
> Date: Thu, 20 Sep 2012 19:38:27 -0400
> To: jay.krell at cornell.edu
> 
> Agreed. You need to backpatch the uplevel locals. I would accumulate all of them. Did you take a look at the LLVM IR I sent?  That's how I plan to so it. 
> 
> Sent from my iPhone
> 
> On Sep 20, 2012, at 19:18, Jay K <jay.krell at cornell.edu> wrote:
> 
> We get uplevel locals declared while within a function.
> So I think I'll bite the bullet and make multiple passes now.
> Otherwise I build up the frame struct "too early".
> I need to see all the locals -- all the uplevels, early in a function.
> 
> 
> There is also a chance they don't all have unique names.
> 
> 
> I do NOT expect I should have separate frames/static_links per-scope,
> just per-function.
> 
> 
> I could put structs inside the frame, named after the line they start
> on or such.. that might be prettier and more readable, or not.
> 
> 
> The structs could then be unioned at some point -- parallel scopes
> can/should share storage.
> 
> 
> M3C.m3:
> PROCEDURE declare_local(...)
> BEGIN
> ...
> IF u.in_proc THEN
>         ExtraScope_Open(u);
>         print(u, var.Declare() & " M3_INIT;");
>         <* ASSERT up_level = FALSE *>  this fails
>     ELSE
>         IF up_level THEN
>             u.current_proc.uplevels := TRUE;
>         END;
>         u.current_proc.locals.addhi(var);
>     END;
>     RETURN var;
> END declare_local;
> 
> 
> == package /Users/jay/dev2/cm3/m3-libs/m3tk-misc ==
> 
> ...
> new source -> compiling Args.m3
> 
> 
> ***
> *** runtime error:
> ***    <*ASSERT*> failed.
> ***    file "../src/M3C.m3", line 1778
> ***
> 
>   m3_backend => 1536
> m3cc (aka cm3cg) failed compiling: Args.mc
> 
> 
> I'm guessing the code is here:
> 
> 
> PROCEDURE Errors(h: Handle; indent: CARDINAL := 0): Text.T RAISES {}=
>   BEGIN
>     IF h.errors = 0 THEN
>       <*FATAL Fatal*> BEGIN RAISE Fatal; END;
>     ELSE
> 
>       VAR
>         texts := NEW(REF ARRAY OF TEXT, h.errors * 2);
>             (* allocates space for all the error messages + padding *)
>         pos: CARDINAL := 0;
>         padding := Fmt.Pad("", indent);
>         fmt: Text.T;
> 
>       <*INLINE*> PROCEDURE Add(t: Text.T) RAISES {}=
>           BEGIN texts[pos] := t; INC(pos) END Add;
>       <*INLINE*> PROCEDURE PaddedAdd(t: Text.T) RAISES {}=
>           BEGIN Add(padding); Add(t) END PaddedAdd;
> 
> 
> 
> see how "texts" is both uplevel and "not at the start" of the function.
> 
> 
> This is going to take a few days to deal with.
> 
> 
> Compiler can compile itself, and a large swath of the system now.
> I didn't use the self-built compiler, but I will do that next. It probably works.
> Very good.
> 
> 
>  - Jay

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://m3lists.elegosoft.com/pipermail/m3devel/attachments/20120921/04570f2b/attachment-0002.html>