<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"><base href="x-msg://869/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Backend compatibility is not something we need to aim for. Different backends will generate different object code.<div>I do plan to use the LLVM mechanisms to pass static links as they are the only way to do so. They use a trampoline to grab the static link for function pointers, but I won’t do that. I’ll compute the static link explicitly so that closures will be constructed in the usual way (triple comprising −1, function pointer, static link).<br>
<br><div><div>On Sep 21, 2012, at 10:27 AM, Jay K <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div class="hmmessage" style="font-size: 12pt; font-family: Calibri; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div dir="ltr">Are you going to use trampolines? Runtime generated code? On the stack? I kind of hope not.<br>There are advantages and disadvantages either way.<br>Actually..hm..if LLVM makes it easy, well, then, that is a big advantage..given how much I have wrestled with this stuff...<br> <br> <br>For example, try writing a nested function using gcc on MacOSX -- you get a warning/error and you have to throw an extra flag to enable nested functions.<br> <br> <br>Another concern here is object code compatibility between the backends? Maybe?<br>What I'm not generating isn't compatible, I'm afraid, and making it compatible might be difficult.<br>It might work to generate C++ and use "this" for the static link. That might be compatible -- if static link and "this" coincide. There is evidence they do (NT/x86) but I haven't researched the various architectures.<br> <br> <br>You know, a few times already I haven't been careful about mixing object code across backends, and I got crashes.. considering how static links work, not surprising..<br> <br> <br> <br> - Jay <br><div><div id="SkyDrivePlaceholder"></div><hr id="stopSpelling">From: <a href="mailto:hosking@cs.purdue.edu">hosking@cs.purdue.edu</a><br>Date: Fri, 21 Sep 2012 09:50:21 -0400<br>To: <a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br>CC: <a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a>; <a href="mailto:antony.hosking@gmail.com">antony.hosking@gmail.com</a><br>Subject: Re: [M3devel] trampolines and uplevel locals declared after function starts..<br><br>I think we should leave the CG interfaces as they are already (for static links). I can see how to generate LLVM IR easily with things the way they are. And you are already working to it. I wasn’t suggesting that you use trampolines, though I think it would be easy enough to do. But I am going to use the LLVM nest approach to pass frame pointers. That way I’ll get the efficient native static link passing code.<div><br></div><div>On Sep 21, 2012, at 1:03 AM, Jay K <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:</div><div><br class="ecxApple-interchange-newline"><blockquote><div class="ecxhmmessage" style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 12pt; line-height: normal; font-family: Calibri; text-transform: none; text-indent: 0px; letter-spacing: normal; word-spacing: 0px; white-space: normal; orphans: 2; widows: 2; "><div dir="ltr"> > Did you take a look at the LLVM IR I sent? T<br><br>Sorry, I couldn't quickly make sense of it. :(<br>I have to read up on LLVM or study your code more.<br>(This is why I favor a C backend -- I'm already an expert and there is far more C expertise in the world than LLVM expertise..I think.. and other reasons like portability..)<br> <br> <br>Mention of "trampolines" was worrisome though.<br>That usually means "runtime generated code, often on the stack". True here?<br> <br> <br>If not, nevermind<br>If so:<br> <br> <br> <br>There is no good solution here, but I think runtime generated code is mostly a losing battle..at least on the stack. If you dedicate a little bit of special purpose platform-specific code to it, and you don't mind it being slow, and you deal with the lifetime management, then it becomes viable.<br>e.g. on Windows, you use VirtualAlloc(MEM_EXECUTE).<br>On other platforms you use mprotect I think. And/or mmap? mmap(PROT_EXEC)?<br>But you end up allocating memory in large chunks, like at least 4K or 64K.<br>(You can also write out little temporary .so/.dll files, but that is unnecessarily inefficient.)<br> <br> <br>I've seen code that calls mprotect against the stack. Yuck.<br>The stack is nonexecutable on purpose -- for security (assuming there is any C or C++ code in the process).<br> <br> <br>If you read the libffi mailing list, you see they are constantly in this arms race.<br>There is a slow steady trickle of problems making the libffi code executable.<br>For Darwin/ARM they came up with a trick to avoid generating code, since the platform doesn't allow unsigned code to run.<br> <br> <br> <br>Pluses/minuses:<br> <br> <br>runtime generated code:<br>Producing the code is fast, on historical systems that don't make it difficult to execute memory, e.g. on the stack.<br>Running the code is fast.<br>Pointers to closures become compatible with C -- you can pass a pointer to a closure as a callback to or C++ and it works.<br>Highly target-dependent.<br> <br> <br>The unusual Modula-3 method:<br>Also fast.<br>Calling function pointers is different and larger and slower.<br>C code cannot call function pointers to nested functions. That is the main downside.<br>Only slightly target-dependent -- in its choice of closure marker.<br>I think we should change the closure marker slightly -- its size should be target-dependent, it should only be 4 bytes on SPARC64, MIPS64, PPC64, Alpha, AMD64 -- so the alignment check can be skipped. And it should possibly be larger than 4 or 8 bytes on IA64.<br>But someone needs to read up on all the various instruction encodings and verify that 4 or 8 0xFF bytes are really invalid and won't occur.<br>But most likely if 4 bytes of 0xFF are a good marker on MIPS32, PPC32, SPARC32, I386, they are also probably a good marker on MIPS64, PPC64, SPARC64, AMD64. And Alpha.<br>Another option is to use something like "jmp marker". i.e. a jump to an address that is used just be the marker. It might not be a constant though. Maybe it is relocated. But you want it to be the same across all dynamic linked code, actually.<br> <br> <br>We might want to have calls through a function pointer go through a helper function, to keep it small.<br> <br> <br><tangent><br>Hm.. I wonder..maybe a closure can be generated fairly portably actually.<br>Maybe you just need one little wrapper per function signature. And most would really be the same.<br>Let me think on this..hm..well..you could do it if you have a special static link register probably. I need to think about this more. Do "all" the calling conventions provide a special extra register for "static link"? And is that often used for C++ "this"? In that case, there is a way..<br> <br> <br><tangent><br>Ultimately, I should point out, the idea of passing a closure to C is actually very useful.<br>One should be able to pass an "object method pointer".<br>So some language construct or pragma might be nice there -- to force down the runtime generated code path.<br>But usually the C function that wants a function pointer also takes a "void* context", so maybe not so needed.<br> <br> <br>Or maybe folks would just use "libffi" for that.<br> <br> <br> <br> - Jay<br> <br><div><div id="ecxSkyDrivePlaceholder"></div><hr id="ecxstopSpelling">CC:<span class="Apple-converted-space"> </span><a href="mailto:m3devel@elegosoft.com">m3devel@elegosoft.com</a><br>From:<span class="Apple-converted-space"> </span><a href="mailto:antony.hosking@gmail.com">antony.hosking@gmail.com</a><br>Subject: Re: [M3devel] uplevel locals declared after function starts..<br>Date: Thu, 20 Sep 2012 19:38:27 -0400<br>To:<span class="Apple-converted-space"> </span><a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a><br><br><div>Agreed. You need to backpatch the uplevel locals. I would accumulate all of them. Did you take a look at the LLVM IR I sent? That's how I plan to so it. <br><br>Sent from my iPhone</div><div><br>On Sep 20, 2012, at 19:18, Jay K <<a href="mailto:jay.krell@cornell.edu">jay.krell@cornell.edu</a>> wrote:<br><br></div><blockquote><div dir="ltr"><div><font face="Calibri, sans-serif">We get uplevel locals declared while within a function.</font></div><div><font face="Calibri, sans-serif">So I think I'll bite the bullet and make multiple passes now.</font></div><div><font face="Calibri, sans-serif">Otherwise I build up the frame struct "too early".</font></div><div><font face="Calibri, sans-serif">I need to see all the locals -- all the uplevels, early in a function.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">There is also a chance they don't all have unique names.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">I do NOT expect I should have separate frames/static_links per-scope,</font></div><div><font face="Calibri, sans-serif">just per-function.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">I could put structs inside the frame, named after the line they start</font></div><div><font face="Calibri, sans-serif">on or such.. that might be prettier and more readable, or not.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">The structs could then be unioned at some point -- parallel scopes</font></div><div><font face="Calibri, sans-serif">can/should share storage.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">M3C.m3:</font></div><div><font face="Calibri, sans-serif">PROCEDURE declare_local(...)</font></div><div><font face="Calibri, sans-serif">BEGIN</font></div><div><font face="Calibri, sans-serif">...</font></div><div><font face="Calibri, sans-serif">IF u.in_proc THEN</font></div><div><font face="Calibri, sans-serif"> ExtraScope_Open(u);</font></div><div><font face="Calibri, sans-serif"> print(u, var.Declare() & " M3_INIT;");</font></div><div><font face="Calibri, sans-serif"> <* ASSERT up_level = FALSE *> this fails</font></div><div><font face="Calibri, sans-serif"> ELSE</font></div><div><font face="Calibri, sans-serif"> IF up_level THEN</font></div><div><font face="Calibri, sans-serif"> u.current_proc.uplevels := TRUE;</font></div><div><font face="Calibri, sans-serif"> END;</font></div><div><font face="Calibri, sans-serif"> u.current_proc.locals.addhi(var);</font></div><div><font face="Calibri, sans-serif"> END;</font></div><div><font face="Calibri, sans-serif"> RETURN var;</font></div><div><font face="Calibri, sans-serif">END declare_local;</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">== package /Users/jay/dev2/cm3/m3-libs/m3tk-misc ==</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">...</font></div><div><font face="Calibri, sans-serif">new source -> compiling Args.m3</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">***</font></div><div><font face="Calibri, sans-serif">*** runtime error:</font></div><div><font face="Calibri, sans-serif">*** <*ASSERT*> failed.</font></div><div><font face="Calibri, sans-serif">*** file "../src/M3C.m3", line 1778</font></div><div><font face="Calibri, sans-serif">***</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"> m3_backend => 1536</font></div><div><font face="Calibri, sans-serif">m3cc (aka cm3cg) failed compiling: Args.mc</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">I'm guessing the code is here:</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">PROCEDURE Errors(h: Handle; indent: CARDINAL := 0): Text.T RAISES {}=</font></div><div><font face="Calibri, sans-serif"> BEGIN</font></div><div><font face="Calibri, sans-serif"> IF h.errors = 0 THEN</font></div><div><font face="Calibri, sans-serif"> <*FATAL Fatal*> BEGIN RAISE Fatal; END;</font></div><div><font face="Calibri, sans-serif"> ELSE</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"> VAR</font></div><div><font face="Calibri, sans-serif"> texts := NEW(REF ARRAY OF TEXT, h.errors * 2);</font></div><div><font face="Calibri, sans-serif"> (* allocates space for all the error messages + padding *)</font></div><div><font face="Calibri, sans-serif"> pos: CARDINAL := 0;</font></div><div><font face="Calibri, sans-serif"> padding := Fmt.Pad("", indent);</font></div><div><font face="Calibri, sans-serif"> fmt: Text.T;</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"> <*INLINE*> PROCEDURE Add(t: Text.T) RAISES {}=</font></div><div><font face="Calibri, sans-serif"> BEGIN texts[pos] := t; INC(pos) END Add;</font></div><div><font face="Calibri, sans-serif"> <*INLINE*> PROCEDURE PaddedAdd(t: Text.T) RAISES {}=</font></div><div><font face="Calibri, sans-serif"> BEGIN Add(padding); Add(t) END PaddedAdd;</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">see how "texts" is both uplevel and "not at the start" of the function.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">This is going to take a few days to deal with.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif">Compiler can compile itself, and a large swath of the system now.</font></div><div><font face="Calibri, sans-serif">I didn't use the self-built compiler, but I will do that next. It probably works.</font></div><div><font face="Calibri, sans-serif">Very good.</font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"><br></font></div><div><font face="Calibri, sans-serif"> - Jay</font></div></div></blockquote></div></div></div></blockquote></div></div></div></div></blockquote></div><br></div></body></html>