[M3devel] Modula-3 llvm backend

[Jay]

I'm kind of impatient, sorry, that will be reflected here:

 - if you have a known working version, stick with it for now?

 - maintain multiple bindings?

 - patches: good & bad: it can be fun to fork, but also wasteful; maintaining diff/patch files may be an option.

 - Proceed without dwarf? If much else to do? You certain the problem is in LLVM? Your patches totally fix the problem.

 - Aomewhat interesting, m3x86 & m3cc for *some* platforms, e.g. Darwin, produce a minimum of debugging info, kinda surprising, & people don't seem to notice; I have largely fixed this with the C backend though & intend to fix it more. 

- Presumably I can/should expand beyond AMD_LINUX. I was going to use I386_DARWIN, but until I convince LLVM to be that, probably AMD64_DARWIN.

 - Jay

On Aug 30, 2015, at 10:52 AM, "Rodney M. Bates" <rodney_bates at lcwb.coop> wrote:

> 
> 
> On 08/29/2015 05:20 PM, Jay K wrote:
>> I understand your dilemas.
>> Static linking vs. dynamic.
>> Cloning the source or not.
>> Significantly growing our repository or not.
>> Forking or not -- do we need any patches? Hopefully not but ok if we do.
>> 
>> 
>> Clearly the C backend cheats -- the C compiler appears on the system
>> as if by magic, or various external means.
>> 
>> 
>> I am somewhat keen on seeing this LLVM stuff work.
>> And somewhat interested in helping.
>> 
>> 
>> First, you know, there are many sayings and their redundancies and opposites:
>> 
>>  1a too many cooks spoil the broth
>>  1b 9 women cannot have a baby in one month (but they can have 9 in 9)
>> 
>>  2 more/many hands make less/little work
>> Which is it here, 1 or 2?
>> i.e. do you need/want help?
> 
> Help is surely needed/wanted, as long as we don't start too much tripping
> over each other.  Just exactly where needs some thought.
> 
>> I have built LLVM now a few times.
>> Yes it is large and takes time.
>> Do we have any required patches for it?
> 
> I am *really* hoping not.  Jay, you know well from gcc, and I from gdb, what a royal pain
> it is to retrofit patches to new versions.  But that may be unrealistic.  The last I
> checked, llvm assert-fails on the dwarf language code for Modula3, and this has to be
> fixed in the llvm source tree.  Maybe we can get them to do that for us.
> 
>> I'm still on the fence as to if we should "import" it, vs. depend on its presence
>> "somehow".
>> 
>> If there really is version sensitivity, that argues for importing.
> 
> There is definitely version sensitivity.  There were big changes in the
> data structure for debug info and the calls to construct it, somewhere
> between 3.5.0 and 3.6.1, require big rework of bindings (which they do
> not provide.)  They do not hesitate to change interface things.  There
> has been a roaring discussion about whether/how to start keeping the
> "Core" interface stable--something apparently has not be a goal so far.
> Core is a C interface they provide to certain things, notably not
> debug info.
> 
>> While "apt-get install llvm" might be in widespread availability,
>> the exact "apt-get install llvm-3.5.0" I imagine isn't.
>> 
>> 
>> The current setup obviously needs work.
>> I'll likely commit something at least a little better.
>> 
>> Do we really need 3.5.0, or can I use 3.5.2?
> 
> It should be either 3.5.0 or 3.6.1, since those are the only ones we have anything
> pretending to be a binding for.  I prefer to at least start with 3.6.1, since it is
> later and significantly different, so will require just that much less retrofitting.
> 
>> 
>>  > So I will be upgrading asap. I am always a bit wary of upgrading seeing the
>>  > problems other people have had, guess I will have to trust upgrade.py.
>> 
>>  Really, I run this stuff all the time.
>>  On various hosts.
>>  What you/we/I do need though is a safe way to rebuild the compiler
>>  when it might be buggy. I have ways, but they aren't great/known/easy/incrmental.
>>  In particular, if you have an ABI change, you need two different m3cores.
>>  The one for the working compiler and the one for the new compiler.
>>  Maybe I just need to use "buildlocal" more.
>>  It doesn't help that the C backend isn't ABI compatible with cm3cg, due to
>>  nested functions and static links. cm3cg uses a target-dependent extra
>>  register not accessible to C as far as I know. C backend uses an extra
>>  last parameter.
>> 
>>   - Jay
>> 
>> 
>> 
>> 
>> > Date: Sat, 29 Aug 2015 13:54:46 -0500
>> > From: rodney_bates at lcwb.coop
>> > To: peter.mckinna at gmail.com; m3devel at elegosoft.com
>> > Subject: Re: [M3devel] Modula-3 llvm backend
>> >
>> >
>> >
>> > On 08/29/2015 06:02 AM, Peter McKinna wrote:
>> > > Hi Rodney,
>> > > No I havent done much lately. I've reached a point where I was waiting for your stuff to be integrated so I can do some more testing. Which is really good. I think we need to track down any lurking bugs by building first the test suite and then the runtime and compiler and see if they work.
>> > >
>> >
>> > If nobody objects, I would like to start working on the 3.6.1 version (see below) now,
>> > instead of the 3.5.0.
>> >
>> > > So I will be upgrading asap. I am always a bit wary of upgrading seeing the problems other people have had, guess I will have to trust upgrade.py.
>> > >
>> >
>> > upgrade.py worked OK for me after I went back to the release compiler and did it
>> > from there, which is the case we really care about. Llvm changes themselves should
>> > not affect the upgrade process, since no llvm mode is used in building the distribution.
>> >
>> > > As you know llvm 3.7 is on the verge of being released. I would like to upgrade to that release soon as well. How did your attempt to build debug bindings go ? That will be the sticking point in upgrading I think.
>> > >
>> >
>> > I did new bindings for Core and DIBuilder for llvm 3.6.1. This was a big pain. llvm
>> > changed the debug info data structure design to make it not derived from Value. Not only
>> > did it mean lots of binding work, but the client code in M3CG_LLVM needed many changes,
>> > though systematic and simple. These are in separate M3 package llvmbindings. In the
>> > case of DIBuilder, I made the binding pretty much complete for what is in llvm, rather
>> > than just what is currently used.
>> >
>> > I still am uncomfortable about how the complex C++ type hierarchies should be
>> > reflected in the bindings. I doesn't look feasible to reflect these into true
>> > M3 hierarcharies. But clients need some kind of conversions. Some functions
>> > return results that need to be passed back in later, but to a parameter having
>> > a static subtype or supertype thereof. The unwrap functions used in llvm appear
>> > to sometimes just silently convert (the equivalent of) failing NARROWs into NIL
>> > pointers, rather than runtime errors. This will inevitably make some bugs much
>> > harder to track down. I'm not even sure dynamic type checks alway happen. Maybe
>> > some undetected type errors can propagate into C++ and give inscrutable segfaults.
>> >
>> > I am hoping future llvm releases will not affect this so much. They do change things
>> > a lot. I also like to avoid chasing the new releases too often. Just as I was getting
>> > llvmbindings to link with 3.6.1, out came 3.6.2, and now 3.7 is looming.
>> >
>> > I have committed an entirely new package named llvm3.6.1, that uses llvmbindings.
>> > This all compiled and linked, as last I checked, but I have done very little execution
>> > checking. These do require getting some llvm stuff in the right places and states.
>> > I have had similar frustrations as Jay has with trying to use git branches, so
>> > for now, I just made this a separate package. They generate alternative executables
>> > with comparable function, so should be easy to switch between.
>> >
>> > Eventually, we will have to decide what to do about getting needed llvm stuff in,
>> > in a standard place and also, how much work will be done outside of the cm3 build
>> > system. Getting a stable copy of llvm stuff inside cm3, as we now do with gcc/m2cc
>> > would be cleanest for users, but it is very big, takes hours to compile, and maybe
>> > not everybody will want to use it badly enough to pay that price. Right now, you
>> > have to do building and installing of llvm on the side, then edit a few paths in
>> > m3makefiles.
>> >
>> > Keeping m3llvm as a completely separate executable means those who don't want to use
>> > it can be unbothered. With its still being in major development, this seems best
>> > for now. I originally set out on the path of having the functions of m3llvm linked
>> > in to cm3, as with the C backend, but am skeptical about all the llvm libraries that
>> > have to be built and linked in.
>> >
>> >
>> >
>> > > As far as debug support goes we are still lacking support for globals , sets, open arrays and hence TEXT and I was planning on asking the llvm people if they had any suggestions.
>> > >
>> >
>> > Those things will take some work, but should not be conceptually difficult.
>> >
>> > > We need to get a few parameters into this backend including the data representation string and target triple and maybe others. Also it would be good if one could set whether debug info was generated. Its easier to read assembler with dwarf annotations than stabs however sometimes its nice to remove the clutter. Last I looked you could set the level of debug but not turn it off entirely in the builder.
>> > >
>> > > Also how does one test the atomic instructions? I have never seen them generated. I think the llvm C api needs support for these as well so the point may be moot. Actually there has been a lot of discussion about the C api recently on the llvm lists so hopefully any changes wont be too radical.
>> > >
>> > > Anyway great work. I'm trying to upgrade the opengl bindings to 4.5 and build some test programs. All good fun.
>> > >
>> > > Regards Peter
>> > >
>> >
>> >
>> >
>> > >
>> > > On Sat, Aug 29, 2015 at 5:52 AM, Rodney M. Bates <rodney_bates at lcwb.coop <mailto:rodney_bates at lcwb.coop>> wrote:
>> > >
>> > > Peter,
>> > >
>> > > How actively are you working on the llvm back end? I don't see a lot of updates to
>> > > the github repository. I have checked in some minor things and think I am about
>> > > ready to do more. But I think it best we keep things merged as much as reasonable.
>> > >
>> > > One change is adding specialized support for calls on alloca. These are very
>> > > recently being generated as library calls by the front end, and come through
>> > > the whole process as link failures. I think it best to detect these and convert
>> > > to llvm alloca instructions. m3cc does something like this now.
>> > >
>> > >
>> > > --
>> > > Rodney Bates
>> > > rodney.m.bates at acm.org <mailto:rodney.m.bates at acm.org>
>> > >
>> > >
>> >
>> > --
>> > Rodney Bates
>> > rodney.m.bates at acm.org
>> > _______________________________________________
>> > M3devel mailing list
>> > M3devel at elegosoft.com
>> > https://mail.elegosoft.com/cgi-bin/mailman/listinfo/m3devel
> 
> -- 
> Rodney Bates
> rodney.m.bates at acm.org